Skip to content

feat(synthesis): representative claim digest + map/reduce version split#193

Merged
bryanmatthewsimonson merged 1 commit into
mainfrom
feat/corpus-brief-breadth
Jul 18, 2026
Merged

feat(synthesis): representative claim digest + map/reduce version split#193
bryanmatthewsimonson merged 1 commit into
mainfrom
feat/corpus-brief-breadth

Conversation

@bryanmatthewsimonson

Copy link
Copy Markdown
Owner

Real-use problem

After #191 attached ~2,335 claims across ~109 articles, an Opus brief came back deeper but narrower than the earlier 2-claim brief: position holders collapsed to 1–3 each, and central cruxes (furin cleavage site, DEFUSE proposal) vanished. The brief said why itself — "13 with extracted claims in the digest."

Cause: DIGEST_CLAIM_CAP = 150 filled the reduce's claim index first-come by article order, so ~13 claim-dense articles consumed all 150 slots and the rest of the corpus never reached the reduce — starving cross-article proposals and claim-anchored cruxes of the whole tail.

Three moves (all reduce-side — the map cache is untouched)

  1. Representative claim index (selectDigestClaims): keep every is_key claim, then round-robin one-per-article so the 150 span the whole corpus instead of clustering in its densest few. claimIndex now carries is_key.
  2. Map/reduce version split — the important structural fix. corpusExtractKey (the map-extract cache key) now keys on MAP_PROMPT_VERSION (held at corpus-v2), not the overall CORPUS_PROMPT_VERSION (bumped to corpus-v3). So this reduce change and all future reduce-prompt tuning stale briefs via corpusInputHash without orphaning the expensive cached extracts. The two start equal and diverge exactly when a reduce-only change lands. (Without this, every reduce tweak would blow away the map cache from feat(synthesis): cache per-article map extracts to skip redundant LLM calls #192 — caching and tuning were at war.)
  3. Reduce-prompt completeness nudge: list EVERY holder (the list is the corpus map, not a sample) and enumerate ALL major cruxes including scientific disputes surfaced in the extracts, not only those with a claim in the index.

Why this is safe

Reduce-side only — the map prompt is unchanged, so your 100 cached extracts survive (verified: corpusExtractKey keys on MAP_PROMPT_VERSION, pinned at corpus-v2). Briefs go stale and re-run cheaply. No wire kind, no schema change, no fused score.

Verification

  • Tests: representative selection (breadth over 200 articles + is_key priority + dense-article can't monopolize); the map/overall version decoupling (a reduce bump doesn't move the cache key); the prompt breadth lines.
  • npm test → 1976 pass · npm run build clean · npm run lint 0 errors.

After merge + reload: stay on Opus, re-run — the 100 extracts reuse for free, and the reduce should produce a brief with full holder lists and the furin/DEFUSE cruxes restored. Compare against the current brief; if a crux still reads thin, the remaining lever is a modest DIGEST_CLAIM_CAP raise.

🤖 Generated with Claude Code

The claim-join fix attached ~2,335 claims across ~109 articles, but
DIGEST_CLAIM_CAP=150 filled the reduce's claim index first-come by article
order — so ~13 dense articles ate all 150 slots and the rest of the corpus
never reached the reduce. Result: deep proposals but a narrowed brief
(holders collapsed to 1-3 each; furin/DEFUSE cruxes dropped).

- selectDigestClaims: keep every is_key claim, then round-robin
  one-per-article so the 150 span the whole corpus, not its densest few.
  claimIndex now carries is_key.
- Split the version: corpusExtractKey (map-extract cache) keys on
  MAP_PROMPT_VERSION (held at 'corpus-v2'); CORPUS_PROMPT_VERSION bumped
  to 'corpus-v3' for staleness/provenance. Reduce-side changes now stale
  briefs WITHOUT orphaning the cached map extracts — the caching and
  reduce-prompt tuning were otherwise at war.
- Reduce prompt: list EVERY holder and enumerate ALL major cruxes
  (incl. scientific disputes not in the claims index), not a sample.

Reduce-side only: the map prompt is untouched, so existing cached
extracts survive. No wire kind. Tests: representative selection (breadth
+ is_key priority), the map/overall version decoupling, prompt breadth.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@bryanmatthewsimonson
bryanmatthewsimonson merged commit a8bc17c into main Jul 18, 2026
1 check passed
@bryanmatthewsimonson
bryanmatthewsimonson deleted the feat/corpus-brief-breadth branch July 18, 2026 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant