feat(m4): the vocabulary collateral strip — VOCAB-SPARING at 1.5B and 3B, AS-SCORED ONLY

[ksdisch]

Close-out stage M4, built to the frozen docs/M4-BRIEF.md (D19–D22, frozen by Kyle 2026-07-29, (a) across the board, with two ratified post-freeze amendments to the D20 wording package).

M3's own Honest limits stated the bound this closes: "the matrix measures collateral among 12 concepts, not across the vocabulary … Nothing here shows that deleting France spares the other 48 M1 concepts." M4 runs that experiment — keep the 12 characterized directions as the primes, widen the probes to all 60 M1 concepts.

Result

M4 VERDICT: VOCAB-SPARING at 1.5B AND 3B — AS-SCORED ONLY.

	0.5B	1.5B	3B
Gate arm (gated non-subset)	41	71	84
Survives all 12	11/41 = 0.268	51/71 = 0.718	63/84 = 0.750
Wilson 95%	[0.157, 0.419]	[0.605, 0.810]	[0.648, 0.830]
vs bar 0.5	fails (off-gate)	clears	clears
Residual-conservative	11/41	49/71, lower 0.575	62/84, lower 0.635
Concept-level collapse	4/23	24/41, lower 0.434	26/43, lower 0.456
Verdict	not shown	VOCAB-SPARING — AS-SCORED ONLY	VOCAB-SPARING — AS-SCORED ONLY

The qualifier fired, and it earned its existence. The concept-level collapse sits below the bar at both gate-bearing subjects while the item-level gate clears it — so per D20 those are the honest numbers to quote, and Amendment 1 (round 4, F20) is why they ride inside the verdict string rather than sitting in prose. Without it, the published label would have been a bare VOCAB-SPARING over a number the brief itself had already named as the honest one.

What's in the diff

• m4_strip.py — cut from m3_matrix.py (never from the certified m0_anchor.py), own byte-frozen GATE_WORDING verbatim from D20 including the AS-SCORED ONLY qualifier and both string templates. 2,340 cells/subject. The 255 M1-recorded and 468 M3-recorded cells (633 once the 90-cell overlap is counted once) are graded FIRST; any mismatch exits INVALID. D22's bars widened to all 60 scored words + the 12 direction words. oracle.py untouched — a fourth byte-shared consumer.
• m4_verdict.py — owns strip_verdict() (NOT A RESULT > DEGENERATE > UNDERPOWERED > the level bar; failing label the lineage null not shown) plus the cross-subject AND. The runner imports it, so one implementation of the verdict string exists.
• test_m4.py — 112 cases (full suite 384, green), pinning the six-round review's hard-won lessons: the residual selector selects exactly 2/2/0 gate-arm cells case-insensitively (case-exact selects 0; "fills the span" selects every cell), fail-in-place keeps the arm at 41/71/84, AS-SCORED ONLY attaches at bar level only, and every realized n and ceiling is cross-checked against the recorded M1 artifacts.
• docs/DECISIONS.md — D19–D22 appended, both amendments recorded (the second separately, since Amendment 1's ratification quote covers only Amendment 1).
• Docs spine: M4-BRIEF results + honest limits, ROADMAP row, LEARNING M4 section + 3 recall questions, PROJECT, HANDOFF, README.

Discipline record

GATE_WORDING frozen as code and gates dry-run before the first real cell; six wrong-arm inputs exit INVALID with named reasons; --limit 18 smoke reproduced its reused cells bit-for-bit and wrote nothing. Both re-certifications passed bit-for-bit on all three subjects (M1 255/255, M3 468/468, concept_mass exact). Every pre-registered n and ceiling landed exactly — a run that disagreed would have been an INVALID cross-check, not a power surprise.

Findings the gate never asked for

1. Finding 1 generalizes out of sample — collateral concentrates on fragile probes, not damaging primes. No prime's row falls below 63/71 (1.5B) or 77/84 (3B), while copper takes 6/12 and mosquito 8/12 at 1.5B, eagle 8/12 and platinum 9/12 at 3B; 32/53 and 33/55 gated columns take zero collateral. Bimodal — which is why the item- and concept-level statistics diverge.
2. Category-block collateral is real in the wider vocabulary and does not dissolve with scale (within 22/29 vs cross 769/823 at 1.5B; 35/53 vs 913/955 at 3B), reversing M3's read — whose within-category arm was 30 of 34 pairs countries. A re-scoping of M3's finding, not a retraction.
3. 0.5B is the first measured divergence between the subset-12's robustness and the wider roster's: 73% of gate-arm items damaged, vs 28% and 25%. Reported under the standing any-direction-damage frame, never as a gate claim.
4. The two statistics disagree by design, and 0.5B shows it starkly: the M3-comparable cluster-mean per-cell floor reads 32/41 → [0.633, 0.880] while the 12-fold conjunction on the same cells reads 0.268 — the sharpest illustration of why D20 refused to inherit M3's 0.5 constant.

Out of scope, flagged not fixed

.github/workflows/ci.yml runs uv run test_<name>.py, which executes each test file as a plain script; no test file has a __main__ guard, so CI has been passing vacuously without running a test. The suites do run under uv run pytest locally. One-line fix, belongs in its own PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KkTj11WoNyTF9SGXvrxxVy

[ksdisch]

Adversarial review — CLEAR TO MERGE (rounds: 3)

#	Severity (final)	Finding	Disposition
F1	should-fix	CI executed test files as scripts — the workflow ran zero assertions since fdcbfcc	Fixed in 11bab32, verified
F2	should-fix	M4 never pinned its own artifacts to its own frozen code (a convention M2 was burned into adding)	Fixed in 11bab32, verified
F5	should-fix	The CI fix turned a vacuously-green gate into a deterministically red one: two dry-run tests need the gitignored lens weights	Fixed in 40e7c26, verified
F7	nice-to-have	PUBLISHED_VERDICTS' comment claimed five docs quote those strings verbatim; none do	Fixed in 40e7c26, verified
F8	nice-to-have	HANDOFF's review record froze at round 1 — two follow-ups listed where three exist	Fixed in d18a79d (docs-only, cap spent, self-checked)
F3	nice-to-have	Frozen degeneracy wording promises per-pair-cell texture strip_package() never computes	Follow-up
F4	nice-to-have	main() re-parses the battery outside the guard that turns drift into a clean INVALID	Follow-up
F6	nice-to-have	The job named offline-suites now really fetches four HuggingFace tokenizer repos	Follow-up

Follow-ups (nice-to-have): F3 per-pair degeneracy texture (honouring the frozen clause would change the artifacts → a ~50-min re-run for a readout pre-declared non-verdict-bearing at n ≤ 3; the write-up owes a deviations row instead); F4 unguarded second battery parse (no reachable path); F6 offline-suites naming vs. real tokenizer fetches (390/396 pass with no network — a future red build there is network, not logic). All three are recorded in HANDOFF.md, so they survive outside the review mailbox.

Coverage: complete on all three rounds. Round 1 read the three behavioural files in full plus their neighbours, and audited the results JSONs by re-deriving every headline statistic from the raw recorded spans rather than reading the diff; rounds 2 and 3 each read their round's full diff. No round recorded an uncovered file-group.

What the review established about the science

The reviewer independently recomputed the D9(b) oracle and the D20 residual selector over all 2,340 cells × 3 subjects from the raw greedy_3 spans alone — 0 disagreements with the recorded produced/residual fields — and re-derived the gate, both conservative reads and the concept collapse from scratch: 69/41/11/11/23→4, 105/71/51/49/41→24, 116/84/63/62/43→26, exact. It confirmed the pre-registered surface arithmetic (255 = 180+36+39, 468 = 36×13, union 633 with a 90-cell overlap) and reproduced every descriptive number it sampled. No defect was found in the measurement path, the gate arithmetic, the conservative reads, the verdict precedence, or the cross-check machinery — all five findings were about what guards the result, and git diff 069e9da..40e7c26 is empty across all nine measurement-path files.

The two should-fixes that mattered form a pair worth stating plainly: this PR added a 1,269-line test suite that CI was not running, and fixing that immediately exposed that two tests depended on gitignored 73 MB–587 MB lens weights. The first fix was verified by running the corrected loop; the second by exporting tracked files only (git ls-files | tar, so lenses/ holds just PROVENANCE.md) and reproducing the reviewer's own failure, then confirming green on the real runner at 40e7c26. The reviewer also mutation-tested that the repaired test still has teeth — inserting a from_pretrained call before the dry-run exit still fails it — so stubbing torch.load did not hollow out the guarantee.

No finding was disputed, so the neutral judge was never dispatched.

Zero-context reviewer + neutral judge on disputes; anchored at d18a79d.