Skip to content

Commit 70bf7d7

Browse files
committed
feat(batch): pre-fetch file versions for cache-enabled batch runs
Add optimization that fetches Figma file metadata once per unique fileId before parallel config processing. This avoids redundant API calls when multiple configs reference the same Figma files. - Add FileIdExtractor to parse configs and collect unique fileIds - Add PreFetchedFileVersions storage with @TaskLocal injection - Add FileVersionPreFetcher for parallel metadata fetching - Update ImageTrackingManager to check pre-fetched storage first - Add preFetchPartialFailure warning type for partial failures
1 parent eba76a2 commit 70bf7d7

14 files changed

Lines changed: 725 additions & 19 deletions

.claude/EXFIG.toon

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,7 @@ keyDirectories:
120120
input: Sources/ExFig/Input/
121121
output: Sources/ExFig/Output/
122122
cache: Sources/ExFig/Cache/
123+
batch: Sources/ExFig/Batch/
123124
terminalUI: Sources/ExFig/TerminalUI/
124125
templates: Sources/*/Resources/
125126
tests: Tests/
@@ -136,6 +137,9 @@ keyFiles:
136137
outputManager: Sources/ExFig/TerminalUI/TerminalOutputManager.swift
137138
spinner: Sources/ExFig/TerminalUI/Spinner.swift
138139
progressBar: Sources/ExFig/TerminalUI/ProgressBar.swift
140+
preFetchedVersions: Sources/ExFig/Batch/PreFetchedFileVersions.swift
141+
fileIdExtractor: Sources/ExFig/Batch/FileIdExtractor.swift
142+
fileVersionPreFetcher: Sources/ExFig/Batch/FileVersionPreFetcher.swift
139143

140144
faultTolerance:
141145
defaults:
@@ -149,6 +153,16 @@ faultTolerance:
149153
rateLimitHandling: respects Retry-After header
150154
checkpointSystem: saves progress for resumption
151155

156+
batchPreFetch:
157+
purpose: Pre-fetch file metadata for unique fileIds before parallel processing
158+
trigger: "--cache flag enabled on batch command"
159+
pattern: "@TaskLocal injection (PreFetchedVersionsStorage)"
160+
files:
161+
PreFetchedFileVersions: Storage struct with TaskLocal
162+
FileIdExtractor: Extracts unique fileIds from YAML configs
163+
FileVersionPreFetcher: Parallel pre-fetching with spinner
164+
fallback: Individual configs fetch own metadata if pre-fetch fails
165+
152166
terminalUI:
153167
classes:
154168
TerminalUI: Main facade for terminal operations

CLAUDE.md

Lines changed: 35 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -177,13 +177,13 @@ RetryLogger.formatRetryMessage(context)
177177

178178
**Warning types:**
179179

180-
| Category | Cases |
181-
| --------------- | ---------------------------------------------------------------------------------------------------------- |
182-
| Configuration | `configMissing`, `composeRequirementMissing` |
183-
| Asset Discovery | `noAssetsFound` |
184-
| Xcode | `xcodeProjectUpdateFailed` |
185-
| Batch | `noConfigsFound`, `invalidConfigsSkipped`, `noValidConfigs`, `checkpointExpired`, `checkpointPathMismatch` |
186-
| Retry | `retrying(attempt:maxAttempts:error:delay:)` |
180+
| Category | Cases |
181+
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
182+
| Configuration | `configMissing`, `composeRequirementMissing` |
183+
| Asset Discovery | `noAssetsFound` |
184+
| Xcode | `xcodeProjectUpdateFailed` |
185+
| Batch | `noConfigsFound`, `invalidConfigsSkipped`, `noValidConfigs`, `checkpointExpired`, `checkpointPathMismatch`, `preFetchPartialFailure` |
186+
| Retry | `retrying(attempt:maxAttempts:error:delay:)` |
187187

188188
**Adding new warnings:**
189189

@@ -279,6 +279,34 @@ let data = try await client.request(endpoint)
279279
- `Sources/FigmaAPI/Client/RetryPolicy.swift` - Retry with exponential backoff
280280
- `Sources/ExFig/Cache/CheckpointTracker.swift` - Checkpoint management for resumable exports
281281

282+
### Batch Pre-fetch Optimization
283+
284+
When `--cache` is enabled, batch processing pre-fetches file metadata for all unique Figma file IDs before parallel
285+
config processing. This avoids redundant API calls when multiple configs reference the same files.
286+
287+
**Key files:**
288+
289+
- `Sources/ExFig/Batch/PreFetchedFileVersions.swift` - Storage struct with `@TaskLocal` injection
290+
- `Sources/ExFig/Batch/FileIdExtractor.swift` - Extracts unique fileIds from YAML configs
291+
- `Sources/ExFig/Batch/FileVersionPreFetcher.swift` - Parallel pre-fetching with spinner
292+
- `Sources/ExFig/Cache/ImageTrackingManager.swift` - Checks `PreFetchedVersionsStorage` before API call
293+
294+
**Pattern:**
295+
296+
```swift
297+
// Pre-fetched versions are injected via @TaskLocal (same pattern as InjectedClientStorage)
298+
let result = await PreFetchedVersionsStorage.$versions.withValue(preFetchedVersions) {
299+
await executor.execute(configs: configs) { ... }
300+
}
301+
302+
// ImageTrackingManager checks TaskLocal first, falls back to API
303+
if let preFetched = PreFetchedVersionsStorage.versions,
304+
let metadata = preFetched.metadata(for: fileId) {
305+
return metadata // Use pre-fetched
306+
}
307+
// ... fall back to API request
308+
```
309+
282310
## Figma API Reference
283311

284312
**Official Documentation:** <https://www.figma.com/developers/api>

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
[![Swift-versions](https://img.shields.io/endpoint?url=https%3A%2F%2Fswiftpackageindex.com%2Fapi%2Fpackages%2Falexey1312%2FExFig%2Fbadge%3Ftype%3Dswift-versions)](https://swiftpackageindex.com/alexey1312/ExFig)
55
[![CI](https://github.com/alexey1312/ExFig/actions/workflows/ci.yml/badge.svg)](https://github.com/alexey1312/ExFig/actions/workflows/ci.yml)
66
[![Release](https://github.com/alexey1312/ExFig/actions/workflows/release.yml/badge.svg)](https://github.com/alexey1312/ExFig/actions/workflows/release.yml)
7-
![Coverage](https://img.shields.io/badge/coverage-58.44%25-yellow)
7+
![Coverage](https://img.shields.io/badge/coverage-57.76%25-yellow)
88
[![License](https://img.shields.io/github/license/alexey1312/ExFig.svg)](LICENSE)
99

1010
Command-line utility to export colors, typography, icons, and images from Figma to Xcode, Android Studio, and Flutter
Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
import Foundation
2+
import Yams
3+
4+
/// Extracts unique Figma file IDs from config files.
5+
///
6+
/// Used by batch processing to collect all unique file IDs before pre-fetching
7+
/// their metadata. This allows a single API call per unique file instead of
8+
/// one per config.
9+
struct FileIdExtractor {
10+
/// Extract all unique file IDs from a list of config URLs.
11+
///
12+
/// Parses each config file and extracts:
13+
/// - `figma.lightFileId` (required)
14+
/// - `figma.darkFileId` (optional)
15+
/// - `common.variablesColors.tokensFileId` (optional)
16+
///
17+
/// - Parameter configURLs: URLs to config files.
18+
/// - Returns: Set of unique file IDs found across all configs.
19+
func extractUniqueFileIds(from configURLs: [URL]) -> Set<String> {
20+
var fileIds = Set<String>()
21+
22+
for configURL in configURLs {
23+
let ids = extractFileIds(from: configURL)
24+
fileIds.formUnion(ids)
25+
}
26+
27+
return fileIds
28+
}
29+
30+
/// Extract file IDs from a single config file.
31+
///
32+
/// - Parameter configURL: URL to the config file.
33+
/// - Returns: Array of file IDs found in this config.
34+
private func extractFileIds(from configURL: URL) -> [String] {
35+
do {
36+
let data = try Data(contentsOf: configURL)
37+
guard let content = String(data: data, encoding: .utf8) else {
38+
return []
39+
}
40+
41+
let decoder = YAMLDecoder()
42+
let config = try decoder.decode(PartialConfig.self, from: content)
43+
44+
var ids: [String] = []
45+
46+
// Extract figma.lightFileId (required)
47+
ids.append(config.figma.lightFileId)
48+
49+
// Extract figma.darkFileId (optional)
50+
if let darkFileId = config.figma.darkFileId {
51+
ids.append(darkFileId)
52+
}
53+
54+
// Extract common.variablesColors.tokensFileId (optional)
55+
if let tokensFileId = config.common?.variablesColors?.tokensFileId {
56+
ids.append(tokensFileId)
57+
}
58+
59+
return ids
60+
} catch {
61+
// Config parsing failed, skip this file
62+
// The actual batch processing will report this error later
63+
return []
64+
}
65+
}
66+
}
67+
68+
// MARK: - Partial Config Models
69+
70+
/// Minimal config structure for extracting file IDs.
71+
/// Only decodes the fields we need, ignoring everything else.
72+
private struct PartialConfig: Decodable {
73+
let figma: FigmaConfig
74+
let common: CommonConfig?
75+
76+
struct FigmaConfig: Decodable {
77+
let lightFileId: String
78+
let darkFileId: String?
79+
}
80+
81+
struct CommonConfig: Decodable {
82+
let variablesColors: VariablesColorsConfig?
83+
}
84+
85+
struct VariablesColorsConfig: Decodable {
86+
let tokensFileId: String
87+
}
88+
}
Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
import FigmaAPI
2+
import Foundation
3+
4+
/// Configuration for pre-fetch operation.
5+
struct PreFetchConfiguration {
6+
let configs: [ConfigFile]
7+
let cacheEnabled: Bool
8+
let noCacheFlag: Bool
9+
let verbose: Bool
10+
let rateLimiter: SharedRateLimiter
11+
let retryPolicy: RetryPolicy
12+
}
13+
14+
/// Pre-fetches file metadata for multiple Figma files in parallel.
15+
///
16+
/// Used by batch processing to fetch all unique file versions upfront,
17+
/// avoiding redundant API calls when multiple configs reference the same files.
18+
struct FileVersionPreFetcher: Sendable {
19+
let client: Client
20+
let ui: TerminalUI
21+
22+
// MARK: - Static Factory
23+
24+
/// Pre-fetches file versions for all unique file IDs if cache is enabled.
25+
///
26+
/// This optimization fetches file metadata once per unique fileId before parallel
27+
/// config processing, avoiding redundant API calls when multiple configs reference
28+
/// the same Figma files.
29+
///
30+
/// - Parameters:
31+
/// - configuration: Pre-fetch configuration.
32+
/// - ui: Terminal UI for progress output.
33+
/// - Returns: PreFetchedFileVersions if successful, nil otherwise.
34+
static func preFetchIfNeeded(
35+
configuration: PreFetchConfiguration,
36+
ui: TerminalUI
37+
) async -> PreFetchedFileVersions? {
38+
// Only pre-fetch when cache is enabled
39+
guard configuration.cacheEnabled, !configuration.noCacheFlag else {
40+
return nil
41+
}
42+
43+
// Extract unique file IDs from all configs
44+
let extractor = FileIdExtractor()
45+
let configURLs = configuration.configs.map(\.url)
46+
let uniqueFileIds = extractor.extractUniqueFileIds(from: configURLs)
47+
48+
guard !uniqueFileIds.isEmpty else {
49+
return nil
50+
}
51+
52+
if configuration.verbose {
53+
ui.info("Found \(uniqueFileIds.count) unique Figma file(s) to pre-fetch")
54+
}
55+
56+
// Get access token
57+
guard let token = ProcessInfo.processInfo.environment["FIGMA_PERSONAL_TOKEN"] else {
58+
// Token missing, let individual configs handle this error
59+
return nil
60+
}
61+
62+
// Create rate-limited client for pre-fetch
63+
let baseClient = FigmaClient(accessToken: token, timeout: nil)
64+
let client = RateLimitedClient(
65+
client: baseClient,
66+
rateLimiter: configuration.rateLimiter,
67+
configID: ConfigID("prefetch"),
68+
retryPolicy: configuration.retryPolicy
69+
)
70+
71+
// Pre-fetch file versions
72+
let preFetcher = FileVersionPreFetcher(client: client, ui: ui)
73+
do {
74+
return try await preFetcher.preFetch(fileIds: uniqueFileIds)
75+
} catch {
76+
// Pre-fetch failed, proceed without optimization
77+
// Individual configs will fetch their own metadata
78+
if configuration.verbose {
79+
ui.warning("Pre-fetch failed: \(error.localizedDescription)")
80+
}
81+
return nil
82+
}
83+
}
84+
85+
// MARK: - Instance Methods
86+
87+
/// Pre-fetch file metadata for all unique file IDs.
88+
///
89+
/// - Parameter fileIds: Set of unique file IDs to fetch.
90+
/// - Returns: PreFetchedFileVersions containing all successfully fetched metadata.
91+
/// - Throws: Error if all fetches fail. Partial failures are handled gracefully.
92+
func preFetch(fileIds: Set<String>) async throws -> PreFetchedFileVersions {
93+
guard !fileIds.isEmpty else {
94+
return PreFetchedFileVersions(versions: [:])
95+
}
96+
97+
let fileIdArray = Array(fileIds)
98+
99+
let result = try await ui.withSpinner(
100+
"Pre-fetching file versions (\(fileIdArray.count) unique files)..."
101+
) {
102+
try await fetchAllMetadata(fileIds: fileIdArray)
103+
}
104+
105+
// Report partial failures if any
106+
let failedCount = fileIdArray.count - result.count
107+
if failedCount > 0 {
108+
ui.warning(.preFetchPartialFailure(failed: failedCount, total: fileIdArray.count))
109+
}
110+
111+
return result
112+
}
113+
114+
/// Fetch metadata for all file IDs in parallel.
115+
private func fetchAllMetadata(fileIds: [String]) async throws -> PreFetchedFileVersions {
116+
try await withThrowingTaskGroup(of: (String, FileMetadata?).self) { group in
117+
for fileId in fileIds {
118+
group.addTask { [client] in
119+
do {
120+
let endpoint = FileMetadataEndpoint(fileId: fileId)
121+
let metadata = try await client.request(endpoint)
122+
return (fileId, metadata)
123+
} catch {
124+
// Individual file fetch failed, return nil
125+
// Will be handled as partial failure
126+
return (fileId, nil)
127+
}
128+
}
129+
}
130+
131+
var versions: [String: FileMetadata] = [:]
132+
for try await (fileId, metadata) in group {
133+
if let metadata {
134+
versions[fileId] = metadata
135+
}
136+
}
137+
138+
// If all fetches failed, throw error
139+
if versions.isEmpty, !fileIds.isEmpty {
140+
throw PreFetchError.allFetchesFailed(count: fileIds.count)
141+
}
142+
143+
return PreFetchedFileVersions(versions: versions)
144+
}
145+
}
146+
}
147+
148+
// MARK: - Errors
149+
150+
/// Errors that can occur during pre-fetching.
151+
enum PreFetchError: Error, LocalizedError {
152+
/// All file metadata fetches failed.
153+
case allFetchesFailed(count: Int)
154+
155+
var errorDescription: String? {
156+
switch self {
157+
case let .allFetchesFailed(count):
158+
"Failed to pre-fetch all \(count) file versions"
159+
}
160+
}
161+
162+
var recoverySuggestion: String? {
163+
switch self {
164+
case .allFetchesFailed:
165+
"Check your FIGMA_PERSONAL_TOKEN and network connection"
166+
}
167+
}
168+
}

0 commit comments

Comments
 (0)