Skip to content

feat(witness): re-verify the evidence read set before promotion (ADR-0010) - #97

Merged
ruvnet merged 1 commit into
mainfrom
sota/evidence-freshness
Sep 7, 2026
Merged

feat(witness): re-verify the evidence read set before promotion (ADR-0010)#97
ruvnet merged 1 commit into
mainfrom
sota/evidence-freshness

Conversation

@ruvnet

@ruvnet ruvnet commented Sep 7, 2026

Copy link
Copy Markdown
Owner

1. Hypothesis

Given a candidate whose evaluation receipt was produced against an earlier tree, when the paths that evaluation read are digest-anchored at evaluation time and re-digested against the promotion target, then drift in that read set should be detected even when git reports no conflict — subject to: (1) a moved base commit alone never yields STALE; (2) age alone is not the mechanism; (3) the witness primitive performs no I/O; (4) no receipt carries file content.

2. The failure this is derived from

PR #11 merged cleanly and still broke main. It was evaluated 2026-08-15 asserting withDefaults(selfConfig).autoMerge === true plus a golden prompt snapshot. Honest evaluation, passed. Three weeks later the repo deliberately set autoMerge: false; on landing, two tests went red (repaired in #95).

dream.config.json was never in #11's diff — git had nothing to conflict on. It was in #11's read set.

git protects the WRITE set. Nothing here protected the READ set.

This is not "receipts get old." A three-week-old candidate whose dependencies never moved is still valid; a one-day-old candidate whose dependencies moved is not. Age is the wrong axis.

3. Candidate

EvidenceFreshnessPolicy / EvidenceFreshnessReceipt in @dream-machine/witness, plus dream-machine freshness stamp|verify.

The policy freezes: policy id, base commit, evaluation timestamp, the declared read set (path + SHA-256 at eval time), requireDeclaredDependencies, optional maxAgeDays. At promotion the same paths are re-digested and compared.

FRESH / STALE / INDETERMINATE (unverifiable is never fresh) / INVALID. Every receipt carries authority: 'none' per ADR-0007/0009 — evidence that an evaluation still applies, never permission to merge.

Deliberate non-goals, because getting them wrong makes the gate useless:

  • a moved base commit is not staleness — main advances constantly; that gate fires always and gets switched off
  • age is a secondary bound, not the mechanism
  • the primitive does no I/O; the CLI supplies observed digests

4. Evaluation Receipt

Vitest, deterministic, $0, zero LLM calls. 616/616, typecheck clean.

Validated against real history, not fixtures: stamping dream.config.json from 8ce3857 (#11's actual evaluation base) and verifying against current main returns STALE, driftedPaths: ["dream.config.json"], exit 1.

5. Security Review

Adversarial review of this change found a real defect in this change: freshness verify originally digested every declared path before the library validated them, so an untrusted policy file naming ../../etc/hosts would be read — and its existence confirmed — before coming back INVALID. A read/existence oracle.

Fixed by validating and anchoring the policy before any filesystem access, with two regression tests (one asserts the only path read is the policy file itself). Also: digests only, never content (tested); bounded read set, path length, and age; no ReDoS-prone patterns; absolute paths, .. traversal, backslashes and null bytes rejected.

6. Reward Hack Check

No test weakened. One pre-existing assertion in my own earlier commit was strengthened (a rewritten policy now produces no receipt at all rather than an INVALID one emitted after touching the filesystem) — called out explicitly in the diff. Full suite reported both ways; the single local npm run check failure is the macOS-only /tmp/private/tmp symlink case in mission.test.mjs, not exercised by this repo's Linux CI.

7. What this does and does not do

It supplies the missing freshness input a bounded auto-landing path would need before acting on a three-week-old ACCEPT verdict. It does not close the promotion loop by itself, and a read set declared too narrowly will still miss drift — this raises the floor, it does not prove independence.

8. ADR

ADR-0010 (Proposed), indexed.

9. Merge Policy

Human review required. Evaluation is not promotion.

🤖 Generated with RuFlo

https://claude.ai/code/session_01Ff2xRKvYrqXJhefvcapfE1

…0010)

Every gate here freezes a policy before outcomes are visible, then treats the
receipt as timeless. PR #11 showed what that misses: evaluated 2026-08-15
against `dream.config.json` as it stood that day, honest evaluation, passed —
and three weeks later `autoMerge` had deliberately flipped, so landing it turned
main red on two tests.

git merged #11 cleanly. `dream.config.json` was never in its diff; it was in its
READ SET — a file the evaluation depended on but did not modify. git protects
the write set; nothing here protected the read set.

Adds `EvidenceFreshnessPolicy`/`Receipt` to @dream-machine/witness: the read set
is declared and digest-anchored at evaluation time, then re-digested against the
tree a candidate would land in. FRESH / STALE / INDETERMINATE / INVALID, every
receipt `authority: 'none'`.

Deliberately NOT age-based and NOT base-commit-based: main advances constantly,
so a gate keyed on either fires always and gets switched off. Drift in the
declared read set is the signal; age is an optional secondary bound.

`dream-machine freshness stamp|verify` does the I/O so the primitive stays pure
(exit 0 FRESH / 1 STALE / 2 indeterminate|invalid, usable as a CI gate).

Security: the CLI validates and anchors the policy BEFORE reading any
dependency. Without that ordering an untrusted policy file naming
`../../etc/hosts` would be read (and its existence confirmed) before the library
rejected it — a read/existence oracle. Caught in adversarial review of this
change, fixed, and regression-tested.

Validated against real history, not just fixtures: stamping `dream.config.json`
from 8ce3857 (PR #11's actual base) and verifying against current main yields
STALE, driftedPaths ["dream.config.json"], exit 1.

616/616 vitest, typecheck clean, sub-quadratic cost bound to 4096 deps.

Also removes a duplicated stale `tui` help line left behind by the #96 union merge.

Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01Ff2xRKvYrqXJhefvcapfE1
@ruvnet
ruvnet merged commit 3edd426 into main Sep 7, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant