Audit: self-referential reasoning — 20 more instances, published as a reference register of the recurring shapes

[WilfordGrimley]

Docs only. One new file, two index edits, no code changed: docs/reference/self-referential-reasoning.md.

Publishes the 2026-07-29 read-only audit that hunted deliberately for self-referential reasoning — places the pipeline measures itself against itself, or consumes its own output as if it were independent evidence — after seven such instances had each been found by accident.

The seven were not everything. 20 further instances: 9 CIRCULAR (the measurement cannot return a different answer), 11 CONTAMINATED (it can, but the sample is biased), plus 5 that look circular and provably are not, kept so nobody re-litigates them.

Where it went, and why

docs/reference/, not docs/reports/ and not docs/audits/.

The mechanical reason is decisive: both of those directories are in docs_lint.py's ARCHIVE_PREFIXES, so they are exempt from the orphan check — frozen buckets the index graph does not traverse. MANIFEST.md classifies both as historical: "never for what's true now."

Several of these findings are standing hazards a reader needs before writing a measurement, not history. A doc whose whole purpose is to be found before someone writes the next gate cannot live somewhere the index does not reach. reference/ is the bucket MANIFEST.md defines as "background, methodology, or a lookup index" — which is exactly what this is.

Not folded into pipeline-fidelity-gate.md (BINDING, scoped to one named gate; these findings span the feed, the soak gate, the importer, the lexicon, the tests and the lint) and not into theory.md (owner-approved, written for an external reader as the federation pitch's technical annex — 20 unsoundness findings do not belong in it).

Authority is reference, deliberately: the findings are diagnoses, and the rulings they need are open issues. Promoting it to BINDING is itself one of the open questions.

It leads with the SHAPE, not the instances

The register is long, but the part with a shelf life is the eight named shapes at the top — six from the commission, two that fell out of the findings — each stated as a smell you can check a diff against:

1. A "ground truth" fixture recorded from the thing it grades
2. A gate whose uncomputable criteria are dropped rather than failing it
3. An eligibility filter that removes rows from a population another calculator COUNTs
4. An agreement rate between two channels that share a decoder
5. A test whose fixture is built from the constant the implementation reads
6. A doc tether that checks a string APPEARS rather than that the claim is TRUE
7. A denominator that is the run's own output
8. A field that reads as a fact about the world and is a fact about our import completeness

It closes with five review questions in the order they are cheapest to answer. The findings register underneath is the evidence the shapes are real.

The sharpest findings

• The soak gate returns PASS for a run that never existed. Proven by execution, not by reading: the real evaluator run read-only against a run_id that has never existed returns all_passed = True, and the report prints VERDICT: PASS — safe to widen to the next step. Three of six criteria abstained and the abstentions were dropped. The criterion that abstains is the only one comparing output against an intended cohort — and it is uncomputable for 97.9% of runs (16 of 995 ledger rows carry the key it reads).
• A zero-weight vote still buys the outcome. 8,216 cards have the frozen 0.0-weight control cohort as their only machine identity; 8,207 are tier-1-eligible right now. The feed renders it as the suggestion to confirm, and the human's click returns as a full-weight USER vote. Against 125 human votes in the entire database.
• The control is not outside what it measures. It never touches the weight arithmetic, but its presence deletes rows from a population another calculator counts: 3,450 spurious singleton names in the name-frequency pool, 100% attributable to the deductive-backfill exclusion.
• The golden set calls itself "per-card ground truth"; every expectation was recorded from the extractor it grades, 9 of 13 pinned set codes are not real MTG set codes, and one commit widened a regex and edited the expectation to match in the same diff.
• The D1 "cross-verified against Scryfall" gate has rejected zero candidates in its lifetime — of 14,940 uniquely-named CanonicalCards, 14,940 have printings_count == 1, because that column is a count over our own rows (113,143 agree with a live COUNT(*), 0 disagree).

The five FINE findings are kept

They record what was checked and found sound — the confidence invariant (the persisted column is write-only; nothing compares confidences, which is why it is safe that the zero-weight cohort asserts the highest ones), the closed version-bump quorum bug, compute_covered_printing_pks, the UNRESOLVED eligibility filter, and evidence_transfer not double-counting.

F6 is the audit's own false alarm, corrected before reporting, kept verbatim in substance — and the corrected number turned out to be the stronger argument.

Relationship to #586

Deliberately separate, cross-linked at the one point they meet. #586 is a verdict on one named gate with a governing status page and a natural home in the dated record. This is a cross-cutting pattern catalogue that has to stay findable by someone who has never heard of issue #154.

They meet at shape 4. #586 found the gate's 83.2% "cross-method" headline is computed where both sides are the same decoder; F6 here measures the same non-independence from the other direction (disagreements strictly one-directional = strict subset). Two methods, same conclusion — which, unlike the agreement rate either is examining, is genuine corroboration.

Rulings filed

Each self-contained, written for a reader who has not seen the audit:

#	question
#592	Should an uncomputable soak-gate criterion block a PASS? (changes Stage E widen/halt semantics)
#593	May a 0.0-weight vote become the suggestion a human confirms into a full-weight vote?
#594	Does the zero-weight ruling extend to eligibility filters, or only to consensus weight?
#595	Relabel the golden set as a regression snapshot, or rebuild it with external labels?
#596	Fix the D1 gate to count from the Scryfall bulk file, or drop the conjunct and restate it honestly?
#597	.claude/hooks/test_guard_master.py — 26KB of tests, executed by nothing

No duplicate filed for the roster tethers — that is issue #577, and the four extensions the audit found (the skip-reason tether has the same shape; three bare literals in local_fallback.py bypass the derivation; two soft checks are mention-only; the tether's own tests cannot fail) are a comment there instead.

K5 and blocking run_name_frequency_elimination were deliberately not filed — reasons stated in the doc.

Verification

• python3 .github/scripts/docs_lint.py --strict → docs-lint: clean. (every backtick code path in the register resolves to a real file at origin/master)
• Indexed in docs/README.md ("Understanding the system") and docs/MANIFEST.md (authority reference)
• Findings are not softened. Several name specific code as unsound; that is the point. LIVE vs LATENT is preserved per finding, and every measurement carries its number.

Provenance

Audit ran read-only against production 85d88bfe while reading code at origin/master 6bc3e16; functions behind quoted figures were diffed between the two trees and confirmed identical. No source mutated, no test suite run, no dormant command invoked, no row written.

🤖 Generated with Claude Code

https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN