Repository navigation
test: pin the match threshold from both sides - #23
Merged
Merged
Conversation
`test_unmatched_line_is_honest` withholds the segment (window=1), so it
exercises the forward window, not the score. Mutation testing found what
that leaves open: raising the CJK floor 0.25 -> 0.60 killed 1 of 57 tests,
and deleting the gate that implements the honest-gap contract killed 1 —
neither of them the test named for that contract.
Two tests now hold the decision boundary from opposite sides:
- a candidate in reach but below the floor stays unmatched, so the gate
itself is guarded rather than the window standing in for it
- a real mishearing of a real line ("九月十五朝霧開戦" heard as "9月15 朝霧
開戦 来るだ...") scores 0.256 and must still match, which is the evidence
the floor sits at 0.25 and cannot be raised
Each fails under its corresponding mutation; that is the whole point of
adding them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The tests looked comprehensive. Mutation testing says they were not covering the one contract this project is built on.
What was open
test_unmatched_line_is_honestpasseswindow=1, so it withholds the segment entirely — it exercises the forward window, not the score. Deleting the threshold check that implements the honest-gap contract left it green.Two mutations against
main, 57 tests:CJK_THRESHOLD0.25 → 0.60 (2.4x)test_latin_default_rejects_chance_similarity)best_score > thresholdgatetest_interpolate_fills_gaps)Both survivors were indirect. Nothing was holding the decision boundary on purpose.
What this adds
Two tests, pinning that boundary from opposite sides.
test_a_reachable_segment_below_the_threshold_stays_unmatched— puts a candidate squarely in reach and lets the score alone reject it, so the gate is guarded rather than the window standing in for it.test_the_cjk_floor_still_accepts_a_badly_heard_true_match— real ASR output for a real line, and the reason the floor sits at 0.25:九月十五朝霧開戦 / 黒田細川矢刃交わし合戦was heard as9月15 朝霧開戦 来るだ 遅かえば 可視化せんand scores 0.256. The model got the moment right and still clears the floor by a hair. Raise the floor and this true match is lost — which is the evidence the README's 0.25 rests on, now enforced.Verification
Each new test fails under its corresponding mutation — that is the point of adding them, and it was checked rather than assumed:
No source changes. Clear
__pycache__between mutation runs — an unchecked hash-based.pycwill keep a reverted constant alive and makegit statuslook clean while the suite still fails.🤖 Generated with Claude Code