Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
147 changes: 147 additions & 0 deletions docs/COUNTERFACTUAL_DESIGN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
# Structural counterfactual — "what depends on this claim" (design)

> **Status:** **approved** by the maintainer 2026-07-16 (drafted the
> same day for Phase-26 review; the §7 open questions are resolved —
> decisions recorded in §7). Implementation follows the hypothesis-map
> slices (CF.1–CF.2 after H.1–H.4), one PR per slice. This is the
> maintainer's "Monte Carlo / if a claim were true or false,
> what happens to the rest" ask, **reframed to clear the epistemic
> firewall** (their explicit decision: *structural, counts not
> probabilities*). Where this and the constitution
> ([`TRUTH_ADJUDICATION_DESIGN.md`](TRUTH_ADJUDICATION_DESIGN.md) §1,
> [`PHILOSOPHY.md`](PHILOSOPHY.md), [`CASE_DOSSIER_DESIGN.md`](CASE_DOSSIER_DESIGN.md)
> §2.2) disagree, the constitution governs.

## §1. The reframing (read this first)

The maintainer asked for a "Monte Carlo" simulator: *if claim X were
true or false, what would it mean for the other arguments and the
entities involved?* A simulator that answers with a **probability**
("X is 73% likely; the case is 60% lab-leak") is the **one shape the
project's constitution was written to exclude** — on this exact case:

> `EPISTACK_ENTRY.md` — "The COVID record's defining fact is that six
> independent Bayesian analyses of the same evidence span **23 orders of
> magnitude**. X-Ray deliberately **does not average, weight, or roll
> up** … the honest headline is the *distribution*."
> `TRUTH_ADJUDICATION §1` — "A proposition is true, false, contested, or
> unresolved — **not '73% true'** … **Quantities are measurements, never
> estimations. Every number shows its derivation from evidence, or it
> does not appear.**"

So this design builds the **structural** counterfactual: given a claim,
report **what in the case graph structurally changes if that claim is
removed or negated** — as **counts that show their derivation**, never
an estimated probability. That is a *measurement* (the constitution's
admissible category), not a *simulation of belief*. It answers "what
depends on this claim" honestly without ever asserting how likely
anything is.

## §2. What it computes

Input: one claim in a case. Substrate: the per-case graph that already
exists — `collectCaseDossierData` → `orbit.claims`, `propositions`,
canonicalized `links.{contradicts, attestations}` (+ the T1.3
`supports`/`updates`/`duplicates` edges), `digestDossier`'s claim
index, and (if the hypothesis map ships) its claim→hypothesis edges.

**Remove the claim** (or **negate** it — flip which side of a
contradiction/support it sits on) and report the *structural delta*, each
line a count with its derivation:

- **Contradiction knots** (`buildKnots` union-find over `contradicts`):
"removing this claim leaves knot K with one fewer node / **dissolves**
knot K (it was the only bridge)."
- **Support / attestation:** "M claims lose their only `supports` edge";
"proposition P loses baseline attestation — `origin_count` drops from
X to Y (the derivation: these N attestations traced to this origin)."
- **Hypothesis edges** (if the map ships): "hypothesis H loses S of its
supporting claims / U of its undermining claims" — as neutral counts,
never a recomputed strength.
- **Entities:** "entities E1, E2 lose their only claim in this case /
their only claim tying them to entity E3."
- **Timeline / coverage:** "this was the only source dated before
<event>; removing it empties that world-time band."

Every output is a diff over deterministic graph structure. **No output
is a probability, a likelihood, a confidence, or a fused score.** The
negate variant reports the same structural deltas for the flipped edge
direction — still counts, never "the case is now more likely X."

## §3. Shape (pure, computed-on-read, no wire kind, no LLM)

A pure module — `src/shared/case-counterfactual.js` (proposed) —
`traceClaimDependencies(dossierData, claimRef, { mode: 'remove' |
'negate' })` → a structured delta object of the counts above, each
carrying its `derivation` (the specific edges/claims/attestations that
produced it). Deterministic ⇒ unit-testable against fixture graphs (the
`case-dossier.test.mjs` style). **No LLM** — this is a graph walk, not a
generation; it needs no gate, no key, no grounding. Computed on read,
nothing persisted, no new wire kind (the dossier posture).

Render: a "Trace dependencies" affordance on a claim in the case view
(or its inspector) that expands the structural delta as a plain list
("Removing this claim would: dissolve 1 contradiction knot · leave 2
claims with no support · drop proposition P's attestation origins 3→2").
Optionally a short narrative rendering of the same counts — but the
counts and their derivations are the substance; prose only restates
them.

## §4. The red line this must not cross

The feature is admissible **only** as a structural measurement. It
crosses a red line the instant it:

- emits a **probability / likelihood / confidence** for the claim, the
case, or an outcome (`TRUTH_ADJUDICATION §1` "never '73% true'");
- produces a **fused case-strength number** or a recomputed hypothesis
score (`CASE_DOSSIER §2.2` "no case-level score, ever"; the Hypothesis
Map's no-scoreboard rule);
- **averages** competing analyses into one figure (P8; Red Line #1).

Guards, matching the `CASE_SYNTHESIS §5` discipline:

- the output object's **key set is grep-tested** to contain no
`probability` / `likelihood` / `confidence` / `score` / `weight`;
- every numeric field is a **count with a `derivation`** array (the
edges/claims it came from) — a number with no derivation is a bug,
not a feature;
- copy review: no output string may read as "more/less likely",
"stronger/weaker case", or "% chance".

If the maintainer later wants the **probabilistic** version (a real
Monte Carlo over analyst-supplied priors, Rootclaim-style), that is a
**separate feature and a separate, constitution-amending decision** —
it would require editing `PHILOSOPHY.md` / `TRUTH_ADJUDICATION §1`
first, and by their own framing (23 orders of magnitude from six honest
Bayesian analyses) the project's answer to date is a deliberate *no*.
This doc flags that fork; it does not take it.

## §5. Non-goals (v1)

No probabilities/likelihoods/confidences anywhere; no fused case or
hypothesis score; no averaging of analyses; no LLM (deterministic graph
walk); no persistence, no wire kind; no claim about *truth* — only about
*structure* ("what connects to what"), leaving the truth judgment to the
verdict layer and the reader.

## §6. Slice ladder (approved 2026-07-16)

| Slice | Content |
|---|---|
| CF.0 | This doc, approved + a ROADMAP Phase-26 entry |
| CF.1 | Pure `traceClaimDependencies` over the dossier graph (remove + negate) + derivation-carrying output + tests (incl. the no-numeric-estimate grep guard) |
| CF.2 | "Trace dependencies" affordance on a claim in the case view; plain structural-delta render |
| CF.3 (optional) | A short narrative restatement of the counts (no LLM required; if LLM, it only re-words the counts and is triple-gated) |
| — (refused) | A probabilistic Monte Carlo — separate feature, requires amending the constitution; not planned |

## §7. Open questions — RESOLVED (maintainer decisions, 2026-07-16)

- **Surface:** the trace renders **inline in the case view** as a
per-claim expander (the §3 render paragraph's shape). A dedicated
dependency-explorer surface is not planned for v1.
- **Negate variant:** **build both** — CF.1 ships `remove` and
`negate` as the §6 ladder already specifies.
- **Probabilistic version:** the hard refusal is **confirmed**. A
Rootclaim-style Monte Carlo remains a separate design requiring a
constitution amendment; it is not planned.
193 changes: 193 additions & 0 deletions docs/HYPOTHESIS_MAP_DESIGN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
# Hypothesis map — competing answers with structural evidence (design)

> **Status:** **approved** by the maintainer 2026-07-16 (drafted the
> same day for Phase-26 review; the §8 open questions are resolved —
> decisions recorded in §8). Implementation proceeds per the §7 slice
> ladder, one PR per slice. It governs a NEW capability; where it and
> the epistemic constitution ([`PHILOSOPHY.md`](PHILOSOPHY.md),
> [`TRUTH_ADJUDICATION_DESIGN.md`](TRUTH_ADJUDICATION_DESIGN.md) §1,
> [`CASE_DOSSIER_DESIGN.md`](CASE_DOSSIER_DESIGN.md) §2.2) disagree, the
> constitution governs.
>
> **Constraints (binding, from the constitution — restated so the
> design can't drift from them):** no fused case or hypothesis
> **score/probability/likelihood** — ever; disagreement renders **side
> by side, never merged** (P8; `EPISTACK_ENTRY.md` "the honest headline
> is the *distribution*"); every number is a **count that shows its
> derivation**, never an estimate (`TRUTH_ADJUDICATION §1`); the model
> **never picks a winner** (`CASE_SYNTHESIS_DESIGN.md`); single-user
> untouched; no new wire kind in v1.

## §1. The problem

Today a case's disagreement is expressed two ways, both flat:

- **Synthesis `positions`** (Phase 20.4) — `{label, core_argument,
holders:[{article_hash}]}`: a hypothesis label, a prose argument, and
*which whole articles* hold it. Rendered side-by-side.
- **Cruxes** — `{question, sides, evidence_refs}`: the questions that
divide the positions, with grounded quotes.

What is missing is a **structural map**: individual captured **claims**
(kind-30040) attached to a hypothesis as **supporting or undermining**
evidence, so a reader can see *which specific facts hold up which
answer*, follow each to its source, and judge the inference structure
themselves. Positions attach at the article level; there is no
claim→hypothesis edge, no per-claim role, and — deliberately — no vote
or weight. This design adds the edge, and nothing that sums it.

`COMPETITION.md` asks exactly for this ("resolve the inference
structure"), and the COVID/Rootclaim corpus is the motivating case: the
right output is not *a* probability but a legible map of what each side
rests on.

## §2. The model (structural, computed-on-read first)

A **hypothesis** is a competing answer to the case's scope question.
Following the dossier precedent (`case-dossier.js` — "DERIVED,
COMPUTED ON READ, nothing persisted, no new wire kind"), v1 assembles
the map on read from data that already exists; a wire kind is a later,
separate decision (§7).

```
HypothesisMap {
question: string // the case scope question (author's framing)
hypotheses: Hypothesis[] // side by side; ORDER is not rank
}
Hypothesis {
id, label, statement // a competing answer, in the author's/model's words
edges: ClaimEdge[] // the structural attachments
}
ClaimEdge {
claim_ref // canonical kind-30040 ref (claim-ref.js)
role: 'supports' | 'undermines'
provenance: 'user' | 'llm:<model>' // who drew the edge (never auto-trusted)
quote?, article_hash? // the grounded span, when the edge came from synthesis
}
```

Hard rules the schema encodes:

- **No `weight`, `score`, `probability`, `confidence`, or `strength`
field on a hypothesis or an edge.** A key-grep test forbids them (the
`CASE_SYNTHESIS_DESIGN.md` §5 discipline, reused).
- A claim may support one hypothesis and undermine another — that IS
the disagreement; it is never resolved into a net.
- **No count that implies a winner.** "12 claims support A, 3 support
B" is a headline that reads as a tally — forbidden. Per-hypothesis
edge counts may render only as neutral section sizes ("Supporting
evidence (12)"), never compared across hypotheses as a scoreboard.
(This is the sharpest drift risk; call it out in review.)

### Where the pieces map on

- **`positions` → hypotheses.** Synthesis positions become the seed
hypotheses (label + statement); their article `holders` become a
starting set of `supports` edges to promote to claim-level.
- **Assessments (kind-30054 stance ±).** A stance is a per-claim
agree/disagree by an author — NOT a hypothesis edge. It may inform a
*suggested* edge (a disagreed claim that undermines a hypothesis) but
the edge is a distinct, human-drawn object; stance never auto-becomes
structure.
- **Propositions / verdicts (kind-30063).** A verdict is a truth state
on one claim, orthogonal to which hypothesis it serves. The map may
*show* a claim's verdict-state chip beside its edge (context), but a
verdict never weights or filters the edge — a `contested` claim can
still be load-bearing for a hypothesis, and that tension is the point.
- **Cruxes.** A crux is a question on which hypotheses' edges to the
*same* claim diverge (A supports it, B undermines it) — the map can
surface cruxes as the claims with opposing edges, tying the two
surfaces together.

## §3. Assembly + the LLM-assist path

Assembly is a pure function over `collectCaseDossierData` +
`digestDossier`'s claim index (both exist): seed hypotheses from
positions, carry any human-drawn edges, render. Deterministic, testable.

Suggesting **claim→hypothesis edges** is the LLM step, and it must obey
the codified firewall (`CASE_SYNTHESIS_DESIGN.md` §4, verbatim pattern):

1. **Gate** — `caseSynthesis` + `llmAssist` + API key (the existing
triple gate; this is a corpus-scale pass).
2. **Map/reduce shape** — reuse `corpus-prompts.js` /
`case-synthesis.js`. The reduce tool emits, per hypothesis, a list
of `{claim_ref, role, quote}` edges. **The tool schema has no
numeric slot** (grep-tested).
3. **Validate → ground → filter** — every edge's `claim_ref` must
resolve to a real captured claim (the `filterProposals` pattern);
every `quote` grounds verbatim against the source member
(`groundCaseBrief`); ungrounded edges drop, drop count disclosed.
4. **Human-accept** — edges land only when a human clicks Accept,
stamped `suggested_by: 'llm:<model>'`. Nothing auto-applies, nothing
auto-publishes. The model proposes edges for BOTH sides and never
declares which hypothesis wins (a pre-flight instruction + a
post-check that no hypothesis was left with zero opposing scrutiny).

## §4. Rendering (KS §8 / TC §3 discipline, generalized)

- Hypotheses **side by side**, order is presentation not rank; a
visible "these are competing answers; X-Ray maps them, it does not
pick one" note.
- Each hypothesis: its statement, then **Supporting** and
**Undermining** claim lists, each claim a followable link to its
source article + grounded quote (reuse the T1.2 provenance render).
- A claim that appears under multiple hypotheses is badged as such
(a crux marker) — the disagreement made legible, never netted.
- No progress bars, no "strength" meters, no per-hypothesis totals
compared to each other.

## §5. Non-goals (v1)

No score/probability/likelihood on any hypothesis or edge; no
winner/ranking/ordering-by-strength; no auto-accept or auto-publish of
edges; no new wire kind (local-derived first); no merging of
disagreement into a net; no replacement of the existing positions/crux
prose (the map sits beside them); single-user posture unchanged.

## §6. Collision with the philosophy — where this must stop

The map is safe **only** as structure. The three ways it would cross a
red line, and the guard for each:

1. **A fused hypothesis score.** → No numeric slot in the model or the
tool schema; grep test. (`TRUTH_ADJUDICATION §1` "never '73% true'";
`CASE_DOSSIER §2.2` "no case-level score, ever".)
2. **A support-count scoreboard** ("A: 12, B: 3") implying a winner. →
Counts render only as neutral section sizes, never cross-compared;
a review checklist item and, ideally, a test asserting the render
emits no cross-hypothesis comparison string. (P8; Red Line #1
"never average away disagreement into a single consensus number.")
3. **The model picking a side.** → Pre-flight refusal + the both-sides
requirement + human-accept-only. (`CASE_SYNTHESIS` "the model never
picks one.")

`PHILOSOPHY.md` closes: "A proposed feature that requires crossing a
red line is not a feature; it is a different, worse system." A
structural argument map clears the bar; a scored one does not — and the
scored one is a **separate, constitution-amending decision the
maintainer has not taken.**

## §7. Slice ladder (approved 2026-07-16)

| Slice | Content |
|---|---|
| H.0 | This doc, approved + a ROADMAP Phase-26 entry |
| H.1 | Pure model + assembler over `collectCaseDossierData` (seed from positions; carry human edges) + tests |
| H.2 | Render side-by-side on the case dashboard (reuse T1.2 provenance links); no-scoreboard guard test |
| H.3 | Manual "attach claim → hypothesis (supports/undermines)" affordance (human-drawn edges) |
| H.4 | LLM edge-suggestion via the map/reduce firewall (triple-gate, ground, human-accept, both-sides) |
| H.5 (deferred) | A wire kind for publishing a hypothesis map — only if the network needs it; a separate decision |

## §8. Open questions — RESOLVED (maintainer decisions, 2026-07-16)

- **Wire kind:** stay **local-derived** like the dossier (the
recommendation). H.5 remains deferred; publishing the map is a
separate, future decision.
- **Neutral per-section edge counts:** **acceptable.** "Supporting
evidence (12)" may render as a plain section size. The guard holds:
counts are never cross-compared between hypotheses, and a test
asserts the render emits no cross-hypothesis comparison string.
- **Verdict-state chip:** **show as context** (the recommendation),
with an explicit "verdict does not weight the edge" note in the UI.
A contested claim staying load-bearing is the point.
28 changes: 28 additions & 0 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -1804,6 +1804,34 @@ NIP-02); no mute list; no sidepanel KS.4 feed changes.

---

## Phase 26 — Corpus analysis, deepened 🚧 in progress

Driven from a maintainer review of the COVID-origins case dashboard.
The **safe wins shipped** (Phase-26 prep, T1.1–T1.3): download the
corpus brief (.md/.json), surface the provenance the brief already
carried (crux evidence quotes, load-bearing source links, position
holders), surface cross-article claim links, and clarify the
Evidence/Case-graph vocabulary. The two larger capabilities were
**approved 2026-07-16** (maintainer decisions recorded in each doc's
open-questions section) — both bounded by the epistemic firewall (no
fused score/probability; structure, not estimation), both
computed-on-read with **no new wire kind**, one PR per slice:

- 🚧 **Hypothesis map** (slices H.1–H.4; H.5 wire kind deferred) —
competing answers to the case question with captured claims attached
as *supporting/undermining* structural evidence, side by side, model
never picks, no score; neutral per-section edge counts allowed but
never cross-compared; verdict chips shown as context only
([`HYPOTHESIS_MAP_DESIGN.md`](HYPOTHESIS_MAP_DESIGN.md)).
- 🚧 **Structural counterfactual** (slices CF.1–CF.2, after the map) —
"what depends on this claim": the maintainer's "Monte Carlo" ask
reframed to counts-with-derivation over the case graph, never
probabilities (the constitution's answer to Rootclaim-style scoring
is a deliberate no); inline per-claim expander, remove + negate modes
([`COUNTERFACTUAL_DESIGN.md`](COUNTERFACTUAL_DESIGN.md)).

---

## Abandonment criteria

From issue #20 — bears repeating. At any phase boundary, if the cost
Expand Down
Loading