Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions docs/JOURNAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,47 @@ or files, and the "so-what" for future readers.

---

## 2026-07-18 — Case synthesis saw ~2 of ~1,900 claims: membership was a union, claim-attachment wasn't

**Tags:** bug, design

A real corpus run reported only **2 claims** present across a 129-article
case. Root cause: `buildMemberUnits` fed the map/reduce from
`deriveArticleRows`' `row.claims`, and those rows attach only the
**orbit** claims — `orbitClaims = allClaims.filter(c => c.about.includes(caseEntityId))`
([case-dossier.js](../src/shared/case-dossier.js)). Claims are authored
`about` their SUBJECTS (SARS-CoV-2, EcoHealth, Fauci…), not the case
container, so exactly the 2 claims someone had tagged onto the case
qualified. Phase 20.1 made ARTICLE membership a union (tag ∪
claim-about-case) but left claim-attachment on the old orbit-only
definition, so every tag-member article joined with `claims: []` — the
whole point of atomizing claims, silently dropped before the LLM ever
saw them.

**Fix (surgical, synthesis-only):** `buildMemberUnits` now joins each
member's claims by normalized `source_url` against the full registry
(`data.claimsById`), key-first/oldest-first for determinism. On the
real corpus this restores **~1,882 claims across 91 articles**. The
deterministic dossier's own `deriveArticleRows` attachment is left
untouched (it's governed separately by CASE_DOSSIER_DESIGN.md; its
article-row claim display is a distinct concern). The URL join also
sidesteps the claim-hash-freeze gotcha — a claim's `article_hash` is
stamped at extraction (`buildClaimEvent`) and lags a re-publish, but
inclusion now keys on URL and re-keys the unit to the member's CURRENT
hash, so a stale claim hash never gates it in or out.

Also: `synthesis-block.js` now derives the corpus staleness hash from
the ACTUAL member-claim ids sent, not `dossier.orbit.claim_ids` — so
adding/removing a claim on a member invalidates the stored brief (it
did not before). Stored briefs re-hash once and show "stale," which is
correct: the corpus definition changed.

Not fixed here (surfaced, deferred): the 30 members with no LOCAL
archive text can't feed the corpus (you can't synthesize bytes you
don't have — `buildMemberUnits` skips non-archive-backed rows), and
`DIGEST_CLAIM_CAP = 150` still bounds the reduce stage's cross-article
claim index (the per-article map stage sees each article's full set).

## 2026-07-17 — Phase 18 C5: the LLM extraction assist, and the reversed-table attack

Tags: `design`, `bug`.
Expand Down
8 changes: 6 additions & 2 deletions src/portal/synthesis-block.js
Original file line number Diff line number Diff line change
Expand Up @@ -209,8 +209,12 @@ export function renderSynthesisBlock(host, { data, dossier, callbacks = {} }) {
if (cid && typeof a.stance === 'number') assessmentsByClaim[cid] = a.stance;
}
const members = await buildMemberUnits(data, { assessmentsByClaim });
const orbitClaimIds = (dossier.orbit && dossier.orbit.claim_ids) || [];
const liveHash = await corpusInputHash(members, orbitClaimIds);
// Staleness must track the ACTUAL claim set sent — the
// member-article claims joined by URL in buildMemberUnits — not
// the orbit's about-the-case subset, or adding/removing a claim
// on a member would never invalidate the stored brief.
const memberClaimIds = members.flatMap((m) => m.claims.map((c) => c.id));
const liveHash = await corpusInputHash(members, memberClaimIds);

const claimsById = {};
// The claim index handed to the reduce stage — id + text +
Expand Down
30 changes: 29 additions & 1 deletion src/shared/case-synthesis.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
// runner (synthesis-block.js) drives the actual LLM calls + storage.

import { Crypto } from './crypto.js';
import { Utils } from './utils.js';
import { EventBuilder } from './event-builder.js';
import { createGroundingIndex } from './quote-grounding.js';
import { CLAIM_RELATIONSHIPS } from './assessment-taxonomy.js';
Expand All @@ -30,28 +31,55 @@ async function sha16(s) { return (await Crypto.sha256(String(s || ''))).slice(0,
* hash covers (so quotes ground against exactly what was sent),
* truncated to the budget with the flag surfaced. `assessmentsByClaim`
* (claim id → stance) is joined in by the caller from AssessmentModel.
*
* A member's `claims` are ALL claims captured from its URL — joined by
* normalized `source_url` against the full registry (`data.claimsById`),
* NOT the orbit filter (`about` names the case). Union membership (20.1)
* makes an article a member by TAG, and its atomized claims must ride
* along, or a corpus of hundreds of claims collapses to the handful
* authored directly about the case entity — the map/reduce would then
* "see" almost none of the corpus. `deriveArticleRows` still supplies
* the member URL set (its orbit-scoped `row.claims` is intentionally
* bypassed here); the deterministic dossier keeps its own attachment.
* The URL join also means a claim's frozen `article_hash` lagging a
* re-publish never gates inclusion — the unit is keyed to the member's
* CURRENT hash regardless.
*/
export async function buildMemberUnits(data, { assessmentsByClaim = {} } = {}) {
const { rows } = deriveArticleRows(data);
const recByUrl = new Map();
for (const rec of data.articles || []) {
if (rec && rec.url) recByUrl.set(rec.url, rec);
}
const claimsByUrl = new Map();
for (const c of Object.values(data.claimsById || {})) {
if (!c || !c.source_url) continue;
const u = Utils.normalizeUrl(c.source_url) || c.source_url;
if (!claimsByUrl.has(u)) claimsByUrl.set(u, []);
claimsByUrl.get(u).push(c);
}
const units = [];
for (const row of rows) {
const rec = recByUrl.get(row.url) || null;
if (!rec || !rec.article) continue; // only archive-backed members feed the corpus
const full = EventBuilder.assembleArticleBody(rec.article) || '';
const text = full.slice(0, MAX_MEMBER_INPUT_CHARS);
const id = rec.articleHash || (`url:${await sha16(row.url)}`);
// Key-first then oldest-first then id (the case-export order),
// so a truncating consumer keeps the key claims and the set is
// deterministic regardless of registry iteration order.
const rowClaims = (claimsByUrl.get(row.url) || []).slice().sort((a, b) =>
(b.is_key ? 1 : 0) - (a.is_key ? 1 : 0)
|| (a.created || 0) - (b.created || 0)
|| (a.id < b.id ? -1 : a.id > b.id ? 1 : 0));
units.push({
article_hash: id,
url: row.url,
title: row.title || null,
text,
truncated: full.length > MAX_MEMBER_INPUT_CHARS,
total_chars: full.length,
claims: (row.claims || []).map((c) => ({
claims: rowClaims.map((c) => ({
id: c.id, text: c.text, quote: c.quote || null, is_key: !!c.is_key,
stance: (c.id in assessmentsByClaim) ? assessmentsByClaim[c.id] : null
}))
Expand Down
41 changes: 41 additions & 0 deletions tests/case-synthesis.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -187,3 +187,44 @@ test('case-synthesis: orchestrateModuleRuns drives the map with injected send (r
assert.equal(failures.length, 1);
assert.equal(failures[0].module, 'C');
});

test('case-synthesis: buildMemberUnits joins ALL a member article\'s claims by source_url, not the orbit (about-case) subset', async () => {
const CASE = 'entity_case';
const SUBJ = 'entity_subject';
// Two member articles (tagged with the case). Their claims are
// `about` the SUBJECT, never the case entity — so the old orbit
// filter (`about` includes the case) would attach ZERO of them. A
// third article is NOT a member (untagged); its claim must not leak.
const data = {
case: { id: CASE, name: 'Test case' },
membership_ids: [CASE],
orbit: { claims: [] }, // nothing authored about the case itself
wire: { articles: [] },
claimsById: {
c1: { id: 'c1', text: 'Claim one', source_url: 'https://ex.com/a', about: [SUBJ], created: 100 },
c2: { id: 'c2', text: 'Key claim', source_url: 'https://ex.com/a', about: [SUBJ], is_key: true, created: 90 },
c3: { id: 'c3', text: 'From B', source_url: 'https://ex.com/b', about: [SUBJ], created: 50 },
cX: { id: 'cX', text: 'Non-member', source_url: 'https://other.com/z', about: [SUBJ], created: 10 }
},
articles: [
{ url: 'https://ex.com/a', articleHash: 'hashA', article: { title: 'A', content: 'Body A', entities: [{ entity_id: CASE }] } },
{ url: 'https://ex.com/b', articleHash: 'hashB', article: { title: 'B', content: 'Body B', entities: [{ entity_id: CASE }] } },
{ url: 'https://other.com/z', articleHash: 'hashZ', article: { title: 'Z', content: 'Body Z', entities: [] } }
]
};
const units = await CS.buildMemberUnits(data);
const byUrl = Object.fromEntries(units.map((u) => [u.url, u]));

assert.deepEqual(units.map((u) => u.url).sort(), ['https://ex.com/a', 'https://ex.com/b'],
'both tagged members present; the untagged article is not a member');
// Article A carries BOTH its claims, joined by URL though neither
// names the case; key-first ordering (c2 is_key) then oldest-first.
assert.deepEqual(byUrl['https://ex.com/a'].claims.map((c) => c.id), ['c2', 'c1']);
assert.equal(byUrl['https://ex.com/a'].claims[0].is_key, true);
assert.deepEqual(byUrl['https://ex.com/b'].claims.map((c) => c.id), ['c3']);
// The unit is keyed to the member's CURRENT hash, not any claim's.
assert.equal(byUrl['https://ex.com/a'].article_hash, 'hashA');
// The non-member article's claim never appears anywhere.
assert.ok(!units.some((u) => u.claims.some((c) => c.id === 'cX')),
'a claim from an untagged, non-member article stays out of the corpus');
});
Loading