diff --git a/docs/COUNTERFACTUAL_DESIGN.md b/docs/COUNTERFACTUAL_DESIGN.md new file mode 100644 index 0000000..242bf3b --- /dev/null +++ b/docs/COUNTERFACTUAL_DESIGN.md @@ -0,0 +1,145 @@ +# Structural counterfactual — "what depends on this claim" (design) + +> **Status:** design draft (2026-07-16), **not approved** — written +> overnight for maintainer review (Phase 26 prep). No code exists yet. +> This is the maintainer's "Monte Carlo / if a claim were true or false, +> what happens to the rest" ask, **reframed to clear the epistemic +> firewall** (their explicit decision: *structural, counts not +> probabilities*). Where this and the constitution +> ([`TRUTH_ADJUDICATION_DESIGN.md`](TRUTH_ADJUDICATION_DESIGN.md) §1, +> [`PHILOSOPHY.md`](PHILOSOPHY.md), [`CASE_DOSSIER_DESIGN.md`](CASE_DOSSIER_DESIGN.md) +> §2.2) disagree, the constitution governs. + +## §1. The reframing (read this first) + +The maintainer asked for a "Monte Carlo" simulator: *if claim X were +true or false, what would it mean for the other arguments and the +entities involved?* A simulator that answers with a **probability** +("X is 73% likely; the case is 60% lab-leak") is the **one shape the +project's constitution was written to exclude** — on this exact case: + +> `EPISTACK_ENTRY.md` — "The COVID record's defining fact is that six +> independent Bayesian analyses of the same evidence span **23 orders of +> magnitude**. X-Ray deliberately **does not average, weight, or roll +> up** … the honest headline is the *distribution*." +> `TRUTH_ADJUDICATION §1` — "A proposition is true, false, contested, or +> unresolved — **not '73% true'** … **Quantities are measurements, never +> estimations. Every number shows its derivation from evidence, or it +> does not appear.**" + +So this design builds the **structural** counterfactual: given a claim, +report **what in the case graph structurally changes if that claim is +removed or negated** — as **counts that show their derivation**, never +an estimated probability. That is a *measurement* (the constitution's +admissible category), not a *simulation of belief*. It answers "what +depends on this claim" honestly without ever asserting how likely +anything is. + +## §2. What it computes + +Input: one claim in a case. Substrate: the per-case graph that already +exists — `collectCaseDossierData` → `orbit.claims`, `propositions`, +canonicalized `links.{contradicts, attestations}` (+ the T1.3 +`supports`/`updates`/`duplicates` edges), `digestDossier`'s claim +index, and (if the hypothesis map ships) its claim→hypothesis edges. + +**Remove the claim** (or **negate** it — flip which side of a +contradiction/support it sits on) and report the *structural delta*, each +line a count with its derivation: + +- **Contradiction knots** (`buildKnots` union-find over `contradicts`): + "removing this claim leaves knot K with one fewer node / **dissolves** + knot K (it was the only bridge)." +- **Support / attestation:** "M claims lose their only `supports` edge"; + "proposition P loses baseline attestation — `origin_count` drops from + X to Y (the derivation: these N attestations traced to this origin)." +- **Hypothesis edges** (if the map ships): "hypothesis H loses S of its + supporting claims / U of its undermining claims" — as neutral counts, + never a recomputed strength. +- **Entities:** "entities E1, E2 lose their only claim in this case / + their only claim tying them to entity E3." +- **Timeline / coverage:** "this was the only source dated before + ; removing it empties that world-time band." + +Every output is a diff over deterministic graph structure. **No output +is a probability, a likelihood, a confidence, or a fused score.** The +negate variant reports the same structural deltas for the flipped edge +direction — still counts, never "the case is now more likely X." + +## §3. Shape (pure, computed-on-read, no wire kind, no LLM) + +A pure module — `src/shared/case-counterfactual.js` (proposed) — +`traceClaimDependencies(dossierData, claimRef, { mode: 'remove' | +'negate' })` → a structured delta object of the counts above, each +carrying its `derivation` (the specific edges/claims/attestations that +produced it). Deterministic ⇒ unit-testable against fixture graphs (the +`case-dossier.test.mjs` style). **No LLM** — this is a graph walk, not a +generation; it needs no gate, no key, no grounding. Computed on read, +nothing persisted, no new wire kind (the dossier posture). + +Render: a "Trace dependencies" affordance on a claim in the case view +(or its inspector) that expands the structural delta as a plain list +("Removing this claim would: dissolve 1 contradiction knot · leave 2 +claims with no support · drop proposition P's attestation origins 3→2"). +Optionally a short narrative rendering of the same counts — but the +counts and their derivations are the substance; prose only restates +them. + +## §4. The red line this must not cross + +The feature is admissible **only** as a structural measurement. It +crosses a red line the instant it: + +- emits a **probability / likelihood / confidence** for the claim, the + case, or an outcome (`TRUTH_ADJUDICATION §1` "never '73% true'"); +- produces a **fused case-strength number** or a recomputed hypothesis + score (`CASE_DOSSIER §2.2` "no case-level score, ever"; the Hypothesis + Map's no-scoreboard rule); +- **averages** competing analyses into one figure (P8; Red Line #1). + +Guards, matching the `CASE_SYNTHESIS §5` discipline: + +- the output object's **key set is grep-tested** to contain no + `probability` / `likelihood` / `confidence` / `score` / `weight`; +- every numeric field is a **count with a `derivation`** array (the + edges/claims it came from) — a number with no derivation is a bug, + not a feature; +- copy review: no output string may read as "more/less likely", + "stronger/weaker case", or "% chance". + +If the maintainer later wants the **probabilistic** version (a real +Monte Carlo over analyst-supplied priors, Rootclaim-style), that is a +**separate feature and a separate, constitution-amending decision** — +it would require editing `PHILOSOPHY.md` / `TRUTH_ADJUDICATION §1` +first, and by their own framing (23 orders of magnitude from six honest +Bayesian analyses) the project's answer to date is a deliberate *no*. +This doc flags that fork; it does not take it. + +## §5. Non-goals (v1) + +No probabilities/likelihoods/confidences anywhere; no fused case or +hypothesis score; no averaging of analyses; no LLM (deterministic graph +walk); no persistence, no wire kind; no claim about *truth* — only about +*structure* ("what connects to what"), leaving the truth judgment to the +verdict layer and the reader. + +## §6. Slice ladder (for review — not yet approved) + +| Slice | Content | +|---|---| +| CF.0 | This doc, approved + a ROADMAP Phase-26 entry | +| CF.1 | Pure `traceClaimDependencies` over the dossier graph (remove + negate) + derivation-carrying output + tests (incl. the no-numeric-estimate grep guard) | +| CF.2 | "Trace dependencies" affordance on a claim in the case view; plain structural-delta render | +| CF.3 (optional) | A short narrative restatement of the counts (no LLM required; if LLM, it only re-words the counts and is triple-gated) | +| — (refused) | A probabilistic Monte Carlo — separate feature, requires amending the constitution; not planned | + +## §7. Open questions for the maintainer + +- Does the trace belong **inline in the case view** (a per-claim + expander) or on a **dedicated surface** (a "dependency explorer")? +- Is the **negate** variant worth building, or is **remove** enough for + v1? (Negate is more evocative of the "if it were false" ask but adds + edge-flip semantics.) +- Confirm the hard refusal of the probabilistic version — or, if the + maintainer wants to reopen it, that is a distinct design + a + constitution amendment, not part of this doc. diff --git a/docs/HYPOTHESIS_MAP_DESIGN.md b/docs/HYPOTHESIS_MAP_DESIGN.md new file mode 100644 index 0000000..c870511 --- /dev/null +++ b/docs/HYPOTHESIS_MAP_DESIGN.md @@ -0,0 +1,190 @@ +# Hypothesis map — competing answers with structural evidence (design) + +> **Status:** design draft (2026-07-16), **not approved** — written +> overnight for maintainer review (Phase 26 prep). No code exists yet. +> This is the "propose hypotheses from multiple points of view that fit +> the evidence" ask. It governs a NEW capability; where it and the +> epistemic constitution ([`PHILOSOPHY.md`](PHILOSOPHY.md), +> [`TRUTH_ADJUDICATION_DESIGN.md`](TRUTH_ADJUDICATION_DESIGN.md) §1, +> [`CASE_DOSSIER_DESIGN.md`](CASE_DOSSIER_DESIGN.md) §2.2) disagree, the +> constitution governs. +> +> **Constraints (binding, from the constitution — restated so the +> design can't drift from them):** no fused case or hypothesis +> **score/probability/likelihood** — ever; disagreement renders **side +> by side, never merged** (P8; `EPISTACK_ENTRY.md` "the honest headline +> is the *distribution*"); every number is a **count that shows its +> derivation**, never an estimate (`TRUTH_ADJUDICATION §1`); the model +> **never picks a winner** (`CASE_SYNTHESIS_DESIGN.md`); single-user +> untouched; no new wire kind in v1. + +## §1. The problem + +Today a case's disagreement is expressed two ways, both flat: + +- **Synthesis `positions`** (Phase 20.4) — `{label, core_argument, + holders:[{article_hash}]}`: a hypothesis label, a prose argument, and + *which whole articles* hold it. Rendered side-by-side. +- **Cruxes** — `{question, sides, evidence_refs}`: the questions that + divide the positions, with grounded quotes. + +What is missing is a **structural map**: individual captured **claims** +(kind-30040) attached to a hypothesis as **supporting or undermining** +evidence, so a reader can see *which specific facts hold up which +answer*, follow each to its source, and judge the inference structure +themselves. Positions attach at the article level; there is no +claim→hypothesis edge, no per-claim role, and — deliberately — no vote +or weight. This design adds the edge, and nothing that sums it. + +`COMPETITION.md` asks exactly for this ("resolve the inference +structure"), and the COVID/Rootclaim corpus is the motivating case: the +right output is not *a* probability but a legible map of what each side +rests on. + +## §2. The model (structural, computed-on-read first) + +A **hypothesis** is a competing answer to the case's scope question. +Following the dossier precedent (`case-dossier.js` — "DERIVED, +COMPUTED ON READ, nothing persisted, no new wire kind"), v1 assembles +the map on read from data that already exists; a wire kind is a later, +separate decision (§7). + +``` +HypothesisMap { + question: string // the case scope question (author's framing) + hypotheses: Hypothesis[] // side by side; ORDER is not rank +} +Hypothesis { + id, label, statement // a competing answer, in the author's/model's words + edges: ClaimEdge[] // the structural attachments +} +ClaimEdge { + claim_ref // canonical kind-30040 ref (claim-ref.js) + role: 'supports' | 'undermines' + provenance: 'user' | 'llm:' // who drew the edge (never auto-trusted) + quote?, article_hash? // the grounded span, when the edge came from synthesis +} +``` + +Hard rules the schema encodes: + +- **No `weight`, `score`, `probability`, `confidence`, or `strength` + field on a hypothesis or an edge.** A key-grep test forbids them (the + `CASE_SYNTHESIS_DESIGN.md` §5 discipline, reused). +- A claim may support one hypothesis and undermine another — that IS + the disagreement; it is never resolved into a net. +- **No count that implies a winner.** "12 claims support A, 3 support + B" is a headline that reads as a tally — forbidden. Per-hypothesis + edge counts may render only as neutral section sizes ("Supporting + evidence (12)"), never compared across hypotheses as a scoreboard. + (This is the sharpest drift risk; call it out in review.) + +### Where the pieces map on + +- **`positions` → hypotheses.** Synthesis positions become the seed + hypotheses (label + statement); their article `holders` become a + starting set of `supports` edges to promote to claim-level. +- **Assessments (kind-30054 stance ±).** A stance is a per-claim + agree/disagree by an author — NOT a hypothesis edge. It may inform a + *suggested* edge (a disagreed claim that undermines a hypothesis) but + the edge is a distinct, human-drawn object; stance never auto-becomes + structure. +- **Propositions / verdicts (kind-30063).** A verdict is a truth state + on one claim, orthogonal to which hypothesis it serves. The map may + *show* a claim's verdict-state chip beside its edge (context), but a + verdict never weights or filters the edge — a `contested` claim can + still be load-bearing for a hypothesis, and that tension is the point. +- **Cruxes.** A crux is a question on which hypotheses' edges to the + *same* claim diverge (A supports it, B undermines it) — the map can + surface cruxes as the claims with opposing edges, tying the two + surfaces together. + +## §3. Assembly + the LLM-assist path + +Assembly is a pure function over `collectCaseDossierData` + +`digestDossier`'s claim index (both exist): seed hypotheses from +positions, carry any human-drawn edges, render. Deterministic, testable. + +Suggesting **claim→hypothesis edges** is the LLM step, and it must obey +the codified firewall (`CASE_SYNTHESIS_DESIGN.md` §4, verbatim pattern): + +1. **Gate** — `caseSynthesis` + `llmAssist` + API key (the existing + triple gate; this is a corpus-scale pass). +2. **Map/reduce shape** — reuse `corpus-prompts.js` / + `case-synthesis.js`. The reduce tool emits, per hypothesis, a list + of `{claim_ref, role, quote}` edges. **The tool schema has no + numeric slot** (grep-tested). +3. **Validate → ground → filter** — every edge's `claim_ref` must + resolve to a real captured claim (the `filterProposals` pattern); + every `quote` grounds verbatim against the source member + (`groundCaseBrief`); ungrounded edges drop, drop count disclosed. +4. **Human-accept** — edges land only when a human clicks Accept, + stamped `suggested_by: 'llm:'`. Nothing auto-applies, nothing + auto-publishes. The model proposes edges for BOTH sides and never + declares which hypothesis wins (a pre-flight instruction + a + post-check that no hypothesis was left with zero opposing scrutiny). + +## §4. Rendering (KS §8 / TC §3 discipline, generalized) + +- Hypotheses **side by side**, order is presentation not rank; a + visible "these are competing answers; X-Ray maps them, it does not + pick one" note. +- Each hypothesis: its statement, then **Supporting** and + **Undermining** claim lists, each claim a followable link to its + source article + grounded quote (reuse the T1.2 provenance render). +- A claim that appears under multiple hypotheses is badged as such + (a crux marker) — the disagreement made legible, never netted. +- No progress bars, no "strength" meters, no per-hypothesis totals + compared to each other. + +## §5. Non-goals (v1) + +No score/probability/likelihood on any hypothesis or edge; no +winner/ranking/ordering-by-strength; no auto-accept or auto-publish of +edges; no new wire kind (local-derived first); no merging of +disagreement into a net; no replacement of the existing positions/crux +prose (the map sits beside them); single-user posture unchanged. + +## §6. Collision with the philosophy — where this must stop + +The map is safe **only** as structure. The three ways it would cross a +red line, and the guard for each: + +1. **A fused hypothesis score.** → No numeric slot in the model or the + tool schema; grep test. (`TRUTH_ADJUDICATION §1` "never '73% true'"; + `CASE_DOSSIER §2.2` "no case-level score, ever".) +2. **A support-count scoreboard** ("A: 12, B: 3") implying a winner. → + Counts render only as neutral section sizes, never cross-compared; + a review checklist item and, ideally, a test asserting the render + emits no cross-hypothesis comparison string. (P8; Red Line #1 + "never average away disagreement into a single consensus number.") +3. **The model picking a side.** → Pre-flight refusal + the both-sides + requirement + human-accept-only. (`CASE_SYNTHESIS` "the model never + picks one.") + +`PHILOSOPHY.md` closes: "A proposed feature that requires crossing a +red line is not a feature; it is a different, worse system." A +structural argument map clears the bar; a scored one does not — and the +scored one is a **separate, constitution-amending decision the +maintainer has not taken.** + +## §7. Slice ladder (for review — not yet approved) + +| Slice | Content | +|---|---| +| H.0 | This doc, approved + a ROADMAP Phase-26 entry | +| H.1 | Pure model + assembler over `collectCaseDossierData` (seed from positions; carry human edges) + tests | +| H.2 | Render side-by-side on the case dashboard (reuse T1.2 provenance links); no-scoreboard guard test | +| H.3 | Manual "attach claim → hypothesis (supports/undermines)" affordance (human-drawn edges) | +| H.4 | LLM edge-suggestion via the map/reduce firewall (triple-gate, ground, human-accept, both-sides) | +| H.5 (deferred) | A wire kind for publishing a hypothesis map — only if the network needs it; a separate decision | + +## §8. Open questions for the maintainer + +- Should hypotheses ever get a **wire kind** (publish the map), or stay + local-derived like the dossier? (Recommend local-derived first.) +- Is a neutral per-section edge **count** acceptable at all, or is even + "Supporting evidence (12)" too scoreboard-like for the constitution? +- Should a claim's **verdict-state chip** show beside its edge (context) + or is that too close to weighting? (Recommend show-as-context, with + an explicit "verdict does not weight the edge" note.) diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index ac56ef9..24b3ddd 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -1804,6 +1804,30 @@ NIP-02); no mute list; no sidepanel KS.4 feed changes. --- +## Phase 26 — Corpus analysis, deepened 🚧 shipped (safe wins) + design-only (speculative) + +Driven from a maintainer review of the COVID-origins case dashboard. +The **safe wins shipped** (Phase-26 prep, T1.1–T1.3): download the +corpus brief (.md/.json), surface the provenance the brief already +carried (crux evidence quotes, load-bearing source links, position +holders), surface cross-article claim links, and clarify the +Evidence/Case-graph vocabulary. Two larger capabilities are +**design-only, pending maintainer review** — both bounded by the +epistemic firewall (no fused score/probability; structure, not +estimation): + +- 📝 **Hypothesis map** — competing answers to the case question with + captured claims attached as *supporting/undermining* structural + evidence, side by side, model never picks, no score + ([`HYPOTHESIS_MAP_DESIGN.md`](HYPOTHESIS_MAP_DESIGN.md)). +- 📝 **Structural counterfactual** — "what depends on this claim": + the maintainer's "Monte Carlo" ask reframed to counts-with-derivation + over the case graph, never probabilities (the constitution's answer + to Rootclaim-style scoring is a deliberate no) + ([`COUNTERFACTUAL_DESIGN.md`](COUNTERFACTUAL_DESIGN.md)). + +--- + ## Abandonment criteria From issue #20 — bears repeating. At any phase boundary, if the cost