Repository navigation
Tests: planted bugs as a script with a record, and clearer input labels - #4
Merged
Merged
Conversation
Follow-up to PR 3, as promised in its review thread. The two planted bugs whose counts the PR body gave by hand now run from tests/plant_bugs.py, with eight more, and tests/PLANTED-BUGS.md records the result: 10 of 10 caught, including the two original counts, 874 and 338. A bug that does not build, or a run that does not finish, is not counted as caught. The record's input kinds 'valid' and 'random' become 'random version' and 'random string', so the two read apart. The counts are unchanged: 3,021,721 inputs, 0 failures. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AovXVgJBHGiN44TxQM4dcd
bar.toml listed the spec authors as claude and fdn-0. The commits and pull requests come from the GitHub account zero-aska, so bar item 1's independence check would not have refused an approval from it. The same fix went into hostname#2, punycode#2 and bech32#2. plant_bugs.py also counts any run that gives up as missed, as in those repos. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AovXVgJBHGiN44TxQM4dcd
ahrazzle
approved these changes
Oct 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a follow-up to #3, which merged today. It does the two things I promised in the review thread there, and fixes the author list in
bar.toml. Nothing outsidetests/,README.mdandbar.tomlchanges.Planted bugs, now in the repo
The body of #3 gave counts for two planted bugs that I had run by hand, so nobody could trace them.
tests/plant_bugs.pynow runs them, with eight more. It writestests/PLANTED-BUGS.md, in the same shape as hostname and punycode.The script plants each bug as a one-line change to
SemVer/Basic.leanin a copy of the project, then runstests/run_tests.py --count 2000:All 10 are caught. The first two counts match the ones #3 stated, 874 and 338.
A run counts as a catch only if it finishes. A bug that does not build, a run that dies before it writes a record, and a run that gives up all count as missed. The rule about dying runs fixes a flaw that a review found in the same script in bech32: a run that died could reuse the old record. The fix also went into hostname and punycode. Each bug also breaks a proof. The tests build only the executable, so they still run.
Clearer labels in the record
As promised on #3, the input kind
validis nowrandom version, andrandomis nowrandom string. Before this, the body's "random versions" meant the record'svalid. The counts are unchanged: 3,021,721 inputs, 0 failures.The spec authors
bar.tomllisted the spec authors asclaudeandfdn-0. The commits and pull requests come from the GitHub accountzero-aska. So bar item 1's check would not have refused a spec approval from the account that wrote the spec.spec_authorsnow nameszero-askatoo. A self-review of bech32#2 found this, and the same fix went into hostname#2, punycode#2 and bech32#2.Checks
python3 tests/run_tests.py: 3,021,721 of 3,021,721 requests agree, 0 failures.python3 tests/plant_bugs.py: 10 of 10 caught.bar/check.py --offline --no-leanat the pinned mission commit: items 2, 4, 5 and 6 pass. Item 1 waits on a spec reviewer.bar.tomlnow liststests/plant_bugs.pyamong the test files.🤖 Generated with Claude Code
https://claude.ai/code/session_01AovXVgJBHGiN44TxQM4dcd