feat(synthesis): cache per-article map extracts to skip redundant LLM calls#192
Merged
Conversation
… calls Corpus synthesis re-ran the entire map (one LLM call per member) on every "Analyze corpus", because nothing persisted the per-article extracts — only the final brief was saved. Three runs over a 129-member case paid for ~3x the map, which is the bulk of the cost. Adds a corpus-extracts store (xray-audits v3) keyed by corpusExtractKey: a SHA-256 over the exact map inputs (sent text + claim digest + case framing + CORPUS_PROMPT_VERSION). The runner checks the cache before the confirm (spend preview shows "N of M cached — reused for free"), short-circuits cached members with no LLM call, and persists each miss. Invalidation falls out of the key: a body edit changes the text; a Suggest pass changes the claim digest (so that article re-maps); a prompt bump changes the version. The reduce still runs each time (one call over the full extract set). After run 1, an unchanged corpus costs only the reduce; adding one article costs one map call, not 129. - keyed on inputs, not article hash, so a claims change re-maps correctly - cache rides the export-included audit DB; reconcilable, never auto-dropped - no wire kind, no prompt change — identical outputs, brief quality unchanged - tests: store CRUD + v1/v2 coexistence; key determinism + invalidation Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Corpus synthesis re-ran the entire map (one LLM call per member) on every "Analyze corpus" click. Nothing persisted the per-article extracts —
xray-auditsstored onlyruns/predictions/resolutions/case-briefs, and only the final brief was saved. So running synthesis 3× over a 129-member case paid for ~3× the map, which is the bulk of the cost (129 map calls vs 1 reduce). (The audit runs cache, keyed by article hash, is a separate feature; the corpus map path was uncached.)Fix
A
corpus-extractsstore (xray-audits v3), keyed bycorpusExtractKey— a SHA-256 over the exact map inputs: sent text + claim digest + case framing +CORPUS_PROMPT_VERSION.Effect: after run 1, a re-run over an unchanged corpus costs only the reduce call; adding one article costs one map call, not 129.
Invalidation falls out of the key
memberTextchanges → new key.Quality
No wire kind, no prompt change — identical outputs, brief quality unchanged. This is pure cost reuse. (Pairs with #191: the claim-join fix enriches inputs; this makes re-running cheap enough to A/B the brief.)
Verification
corpus-extractsstore CRUD + v1/v2 coexistence across the v3 upgrade;corpusExtractKeydeterminism + each invalidation trigger. Updated the backup dump-shape assertion for the new store.npm test→ 1973 pass ·npm run buildclean ·npm run lint0 errors.🤖 Generated with Claude Code