Track: research · Level: core · Effort: ~6h · Phase: 7 · Depends on: FIX-3-2
Source: this plan, section 9
Why this matters
T20's credibility comes from having frozen the statistical contract before looking at the
numbers. The calibration study has more degrees of freedom than T20 did, so it needs the same
discipline or it will find a threshold that fits and prove nothing.
Steps
- Write down, before any calibration run: the decision rule, the threshold grid, the loss
quantile grid, the train/validation seed split, the primary endpoint, and the statistical
contract.
- State the falsifiable prediction, in the T20 style.
- State the kill criterion explicitly: if the calibrated rule does not beat
metric_argmax
on held-out seeds, diagnosis_single does not become the default and the result is published
as a negative result.
- Get maintainer sign-off before the first run.
Done when
The pre-registration document is merged and dated before any calibration output exists.
How to take this: comment "taking this" and wait to be assigned. Branch fix-7-1-preregistration from upstream/main, and put Closes #<this issue number> in your PR. Full workflow: the Cohort Handbook (pinned in Discord).
Plan: plans/FIX-PLAN-T20-2026-08-23.md, section 9.
Track: research · Level: core · Effort: ~6h · Phase: 7 · Depends on: FIX-3-2
Source: this plan, section 9
Why this matters
T20's credibility comes from having frozen the statistical contract before looking at the
numbers. The calibration study has more degrees of freedom than T20 did, so it needs the same
discipline or it will find a threshold that fits and prove nothing.
Steps
quantile grid, the train/validation seed split, the primary endpoint, and the statistical
contract.
metric_argmaxon held-out seeds,
diagnosis_singledoes not become the default and the result is publishedas a negative result.
Done when
The pre-registration document is merged and dated before any calibration output exists.
How to take this: comment "taking this" and wait to be assigned. Branch
fix-7-1-preregistrationfromupstream/main, and putCloses #<this issue number>in your PR. Full workflow: the Cohort Handbook (pinned in Discord).Plan:
plans/FIX-PLAN-T20-2026-08-23.md, section 9.