Skip to content

ci: fail-closed guard against left-over mutation mutants#129

Merged
b7n0de merged 1 commit into
mainfrom
ci/mutation-mutant-fangnetz
Jul 23, 2026
Merged

ci: fail-closed guard against left-over mutation mutants#129
b7n0de merged 1 commit into
mainfrom
ci/mutation-mutant-fangnetz

Conversation

@b7n0de

@b7n0de b7n0de commented Jul 23, 2026

Copy link
Copy Markdown
Owner

Why

On 2026-07-23 a mutation probe's if False: at the SD-JWT key-binding check in bundle.py survived into the working tree (plus a stray .mutbak backup); only a manual diff-before-commit caught it before commit. try/finally restore cannot survive a hard kill (SIGKILL skips finally), so the fix has to be structural, not procedural.

What (three mechanical layers)

  1. mutation_check.py v1.4, tempdir isolation: mutants are applied in a throwaway copy of the tracked tree (git ls-files → tempdir); the real working tree is never written to, so even SIGKILL mid-probe cannot leave a mutant or backup litter behind. The .mutbak mechanism is gone (in-memory restore inside the copy). Differential-baseline semantics unchanged. The run additionally compares git status --porcelain before/after and fails closed on any left-over working tree change.
  2. CI clean-assert: the mutation job ends with an always-run step that fails if git status --porcelain is non-empty, the machine version of the manual diff that caught the incident.
  3. mutant_signature_guard.py: a narrow, explainable diff guard over added lines on src/proofbundle/**, wired as a pre-commit hook (scripts/install_git_hooks.sh) AND a new blocking CI job, so skipping the hook locally does not skip the check. Classes: (A) trivial-truth branches (if/elif False|True, while False) at a check, (B) commented-out verification lines (verify-call prefilter, then the comment content must PARSE as a Python statement, so prose stays quiet), (C) return True opening a verify/validate/check function. Legitimate exceptions need a visible # mutant-guard: allow comment in the diff.

Evidence

  • --self-test (runs first in the CI job): 11/11, five signature classes caught, six negative controls quiet, including the three real false-positive prose shapes found while scanning the full v3.6.0..HEAD history (pinned as fixtures after narrowing class B).
  • Full-history negative control: guard is quiet over v3.6.0..HEAD.
  • Incident replay: staging the exact if False: at bundle.py:425 is blocked by the installed pre-commit hook (exit 1, no commit created).
  • SIGKILL mid-probe on the real repo: git status --porcelain before/after byte-identical, zero .mutbak.
  • Isolated end-to-end on the real repo: baseline red 0, operator "bundle: KB check unwired" KILLED, the documented-equivalent survivor SURVIVED, both as expected.
  • 17 new unittest cases (staged + base modes, isolation contract, leftover fail-closed), ruff clean.

No changelog entry: ci: prefix is documented-trivial in check_version_and_changelog.py. No release, no tag, no PyPI.

Incident 2026-07-23: a mutation probe's `if False:` at the SD-JWT key-binding
check in bundle.py survived into the working tree (plus a stray .mutbak file);
only a manual diff-before-commit caught it. try/finally restore cannot survive
a hard kill. Three mechanical layers close the class:

1. mutation_check.py v1.4 isolation: mutants are applied in a throwaway tempdir
   copy of the tracked tree (git ls-files), the real working tree is never
   written to, so even SIGKILL mid-probe cannot leave a mutant or backup litter
   behind. The .mutbak mechanism is gone; restore is in-memory, inside the copy.
   Belt and braces, the run compares `git status --porcelain` before/after and
   fails closed on any left-over working tree change.

2. CI clean-assert: the mutation job ends with an always-run step that fails
   closed if `git status --porcelain` is non-empty (the machine version of the
   manual diff that caught the incident).

3. mutant_signature_guard.py: a narrow, explainable diff guard (pre-commit hook
   via scripts/install_git_hooks.sh AND a new blocking CI job, so the hook
   cannot be bypassed locally) over added lines on src/proofbundle/**:
   A) trivial-truth branches (if/elif False|True, while False) at a check,
   B) commented-out verification lines (two-stage: verify-call prefilter, then
      the content must PARSE as a Python statement; the three false-positive
      prose shapes found on real v3.6.0..HEAD history are pinned as negative
      self-test cases),
   C) `return True` opening a verify/validate/check function.
   Legitimate exceptions need a visible `# mutant-guard: allow` comment in the
   diff. --self-test proves each class is caught before a clean verdict is
   trusted; the guard is quiet over the full v3.6.0..HEAD history.

Verified locally: 17 new unittest cases (staged + base + isolation contract),
self-test 11/11, SIGKILL mid-run leaves the tree byte-identical, the staged
incident replay (`if False:` at bundle.py:425) is blocked by the installed
pre-commit hook, ruff clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@b7n0de
b7n0de merged commit 3929c27 into main Jul 23, 2026
22 checks passed
@b7n0de
b7n0de deleted the ci/mutation-mutant-fangnetz branch July 23, 2026 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant