ci: fail-closed guard against left-over mutation mutants#129
Merged
Conversation
Incident 2026-07-23: a mutation probe's `if False:` at the SD-JWT key-binding
check in bundle.py survived into the working tree (plus a stray .mutbak file);
only a manual diff-before-commit caught it. try/finally restore cannot survive
a hard kill. Three mechanical layers close the class:
1. mutation_check.py v1.4 isolation: mutants are applied in a throwaway tempdir
copy of the tracked tree (git ls-files), the real working tree is never
written to, so even SIGKILL mid-probe cannot leave a mutant or backup litter
behind. The .mutbak mechanism is gone; restore is in-memory, inside the copy.
Belt and braces, the run compares `git status --porcelain` before/after and
fails closed on any left-over working tree change.
2. CI clean-assert: the mutation job ends with an always-run step that fails
closed if `git status --porcelain` is non-empty (the machine version of the
manual diff that caught the incident).
3. mutant_signature_guard.py: a narrow, explainable diff guard (pre-commit hook
via scripts/install_git_hooks.sh AND a new blocking CI job, so the hook
cannot be bypassed locally) over added lines on src/proofbundle/**:
A) trivial-truth branches (if/elif False|True, while False) at a check,
B) commented-out verification lines (two-stage: verify-call prefilter, then
the content must PARSE as a Python statement; the three false-positive
prose shapes found on real v3.6.0..HEAD history are pinned as negative
self-test cases),
C) `return True` opening a verify/validate/check function.
Legitimate exceptions need a visible `# mutant-guard: allow` comment in the
diff. --self-test proves each class is caught before a clean verdict is
trusted; the guard is quiet over the full v3.6.0..HEAD history.
Verified locally: 17 new unittest cases (staged + base + isolation contract),
self-test 11/11, SIGKILL mid-run leaves the tree byte-identical, the staged
incident replay (`if False:` at bundle.py:425) is blocked by the installed
pre-commit hook, ruff clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
On 2026-07-23 a mutation probe's
if False:at the SD-JWT key-binding check in bundle.py survived into the working tree (plus a stray.mutbakbackup); only a manual diff-before-commit caught it before commit.try/finallyrestore cannot survive a hard kill (SIGKILL skipsfinally), so the fix has to be structural, not procedural.What (three mechanical layers)
git ls-files→ tempdir); the real working tree is never written to, so even SIGKILL mid-probe cannot leave a mutant or backup litter behind. The.mutbakmechanism is gone (in-memory restore inside the copy). Differential-baseline semantics unchanged. The run additionally comparesgit status --porcelainbefore/after and fails closed on any left-over working tree change.git status --porcelainis non-empty, the machine version of the manual diff that caught the incident.src/proofbundle/**, wired as a pre-commit hook (scripts/install_git_hooks.sh) AND a new blocking CI job, so skipping the hook locally does not skip the check. Classes: (A) trivial-truth branches (if/elif False|True,while False) at a check, (B) commented-out verification lines (verify-call prefilter, then the comment content must PARSE as a Python statement, so prose stays quiet), (C)return Trueopening a verify/validate/check function. Legitimate exceptions need a visible# mutant-guard: allowcomment in the diff.Evidence
--self-test(runs first in the CI job): 11/11, five signature classes caught, six negative controls quiet, including the three real false-positive prose shapes found while scanning the fullv3.6.0..HEADhistory (pinned as fixtures after narrowing class B).v3.6.0..HEAD.if False:at bundle.py:425 is blocked by the installed pre-commit hook (exit 1, no commit created).git status --porcelainbefore/after byte-identical, zero.mutbak.No changelog entry:
ci:prefix is documented-trivial in check_version_and_changelog.py. No release, no tag, no PyPI.