docs: plan for closing the 26 open evidence markers - #38
Merged
Conversation
The wiki carries 26 TODO(maintainer) markers. They are not one kind of problem and should not be worked as one: a third close with a benchmark run the CI workflow now makes cheap, a third are arithmetic that no run will fix, and the rest are decisions or are unrecoverable and have to say so. Ordered so effort goes where it buys the most evidence. Phase 1 is five markers that need one workflow run each. Phase 2 is numbers that contradict the rows printed beside them — worse than missing evidence, because a reader who checks them finds the error and every other figure loses credit. The global constraints are the five things this project learned by breaking them: grep for a page name after changing it and read every hit in context; never publish a single-fork measurement; bind every count to a commit; never compare figures from different machines; and make a checker fail before trusting it. Line numbers are bound to 662e4db and will drift. The plan says so and names the grep that is the authority instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzEdy5fN5JKGxJo8gwtNeu
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A worklist for the
TODO(maintainer)markers in the wiki, written so a fresh session can execute itwithout the context of the one that produced it.
Why it is phased rather than a list
The 26 markers are not one kind of problem:
-prof gcrunPhase 1 first because the evidence is merely missing and the tool now exists. Phase 2 next because
those numbers are wrong: 97 % and 63 % against published rows that give 98.0 % and 65.3 %, and a
ratio column that does not follow from its own operands. A reader who checks those finds the error,
and every other figure on the page loses credit with it.
The constraints are the interesting part
Five rules, each learned by breaking it in the round that produced this plan:
mechanically; the danger is the sentence without an anchor whose prose asserts what the page
holds. Eighteen survived one split, on nine pages, four of which were on no change list. This
failed five times in one session and no tool reports it.
confirm.
seven sentences across three files. A count in a heading is worse — prefer removing it.
number: 1.35× on a CI runner against 1.13× on the machine behind the published tables.
Note on the line numbers
Verified against the wiki at
662e4db— all fourteen spot-checked references hit their marker. Theywill drift, the plan says so, and it names
grep -rn 'TODO(maintainer)'as the authority instead.Following its own rule about counts bound to a commit.
🤖 Generated with Claude Code
https://claude.ai/code/session_01QzEdy5fN5JKGxJo8gwtNeu