Skip to content

Commit f85ff54

Browse files
martex-devclaude
andcommitted
M16: the paper, enumerated rather than written
`nullius paper build` renders the record from the committed protocols and results. The failure it guards against is ordinary and nearly invisible: a project runs several protocols, one produces the good result, and the write-up quietly becomes about that one. It cannot select because it does not choose. Every registered protocol appears in registration order with its prediction and its outcome -- two upheld, two refuted, one registered and not yet run. The unrun one is labelled rather than omitted, because a plan with no result is part of the record. The two results later protocols retracted are still there, under the protocols that produced them. Nothing numeric is typed. Every figure is read from a results file whose stored summary re-scores from its own per-item rows; every prediction from a protocol whose hash is in the git history; bank difficulty computed from the locked truths. assemble(strict=True) refuses to build when a protocol fails to verify or a results file fails to re-score, which makes the paper a check on the record rather than a description of it. Two prose sections, declared as data rather than woven into the template so they can be counted and checked in one place: six flaws and five limitations. Each flaw names the milestone whose commit records it, and a test enforces that. That section is what a written-up-afterwards paper would not contain -- in that genre the flaws are fixed before anything is published; here five of the six were found by executing a preregistered plan rather than by reviewing one. A ninth CI job builds the paper on every push and uploads it. 456 tests pass. ruff and mypy clean on 94 source files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 867251d commit f85ff54

9 files changed

Lines changed: 1487 additions & 1 deletion

File tree

.github/workflows/ci.yml

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -208,6 +208,25 @@ jobs:
208208
name: report
209209
path: .nullius/ci/site
210210

211+
paper:
212+
name: the paper builds from committed sources
213+
runs-on: ubuntu-latest
214+
# Refuses to build when a protocol fails to verify or a results file fails
215+
# to re-score. That makes the paper a check on the record rather than a
216+
# description of it: if the two ever diverge, this job is what says so.
217+
steps:
218+
- uses: actions/checkout@v5
219+
- uses: astral-sh/setup-uv@v7
220+
with:
221+
enable-cache: true
222+
- run: uv sync --all-extras --dev
223+
- name: every protocol and every result, enumerated
224+
run: uv run nullius paper build --out paper/index.html
225+
- uses: actions/upload-artifact@v4
226+
with:
227+
name: paper
228+
path: paper/index.html
229+
211230
clean-clone:
212231
name: reproducible from a clean clone
213232
runs-on: ubuntu-latest

BUILD_PLAN.md

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
55
This is the executable plan derived from [`docs/`](docs/). The design documents say *what* to build and *why*; this says *in what order*, *with what acceptance test*, and *what changes because of the machine we're actually on*.
66

7-
**Status:** M0–M14 complete (mock-driven throughout; the first live run awaits an API key). M12's code-generation half is blocked on both a key and Docker. Nothing below is claimed as done until its acceptance criteria are green in CI.
7+
**Status:** M0–M16 complete; M15's v5 ladder is running (mock-driven throughout; the first live run awaits an API key). M12's code-generation half is blocked on both a key and Docker. Nothing below is claimed as done until its acceptance criteria are green in CI.
88

99
---
1010

@@ -425,6 +425,23 @@ So: **B4 − B3 is not a stable finding.** It was +0.033 spanning zero in v3 and
425425

426426
**The next milestone is replication of the ladder itself.** One draw per arm is not enough at this resolution, and the project has now measured how much it is not enough by. Every arm should run several times and the report should carry the distribution rather than a single number.
427427

428+
---
429+
430+
### M16 · The paper ✅
431+
`nullius paper build` renders `paper/index.html` from the committed protocols and results. A roadmap item (`docs/06`, template-rendered papers) and the natural place for the record to end up.
432+
433+
**Acceptance** — the document cannot report a flattering subset.
434+
435+
It does not select because it does not choose. Every registered protocol appears in registration order with its prediction and its outcome: **two upheld, two refuted, one registered and not yet run** — and the unrun one is labelled as such rather than omitted, because a plan with no result is part of the record too. Two results that later protocols retracted are still in the document, under the protocols that produced them.
436+
437+
**Nothing numeric is typed.** Every figure is read from a results file whose stored summary re-scores from its own per-item rows; every prediction is read from a protocol whose hash is in the git history; bank difficulty is computed from the locked truths. `assemble(strict=True)` refuses to build when a protocol fails to verify or a results file fails to re-score — a paper whose inputs no longer check out is worse than no paper, because it looks like evidence.
438+
439+
**Two prose sections, declared as data.** The six flaws and five limitations are the only hand-written content, held in `render.py` as constants so they can be counted and checked in one place. Each flaw names the milestone whose commit records it, and a test enforces that.
440+
441+
The flaw list is the section a written-up-afterwards paper would not have, because in that genre the flaws are fixed before anything is published. Here they are the record: five of them were found by *executing* a preregistered plan rather than by reviewing one.
442+
443+
A ninth CI job builds it on every push and uploads it as an artifact.
444+
428445
**A wiring bug worth recording.** The first v4 ladder ran eight arms under a nine-arm protocol: a `ruff format` pass had collapsed the `run_ladder(...)` call onto one line before an edit meant to add `arms=` to it, so the replacement matched nothing and the runner silently used its eight-arm default. It produced a complete-looking results file — seven of seven baseline comparisons, no halted items — and nothing objected except the adjudication, which happened to name the missing arm by id. `score_ladder` now refuses any run whose arms do not match the protocol's, in either direction. The eight completed arms were reused from their checkpoints, so the correction cost one arm's compute rather than nine.
429446

430447
---

0 commit comments

Comments
 (0)