Surface test counts in CI; bump Node 20 actions; ignore __pycache__#72
Merged
Conversation
ctest reports pass/fail for the binary as a whole, and --output-on-failure hides Catch2's summary on success, so a green build recorded nothing about how many cases actually ran. A case that starts skipping stays present in --list-tests (so the #65 guard misses it) and needs no fixture (so the #63 guard misses it too), yet exercises nothing. That is not hypothetical: #70 was a completely broken FFV1 path hidden behind exactly this, for however long it had been there. Every codec these tests need is installed in CI, so a skip there is a real gap rather than graceful degradation, and the check fails the build. It uses the default console reporter deliberately -- --reporter compact omits the skip count entirely, so the grep would never fire. Verified both directions against real output: the line from a run with skips matches, the current clean run does not. Also bumps actions/checkout@v4 -> v5 and actions/setup-python@v5 -> v6 across all three workflows. Both target Node 20, which GitHub is currently force-running on Node 24 behind a shim it calls temporary; when that goes they break, for reasons unrelated to any change here. And adds __pycache__/ to .gitignore, which has been dirtying git status since the Python bindings landed. Fixes #66, fixes #68 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #66, fixes #68.
Skipped tests were invisible (#66)
ctestreports pass/fail for the binary as a whole, and--output-on-failurehides Catch2's summary on success — so a green build recorded nothing about how many cases actually ran.A case that starts skipping stays present in
--list-tests(so the #65 guard misses it) and needs no fixture (so the #63 guard misses it too), yet exercises nothing.This is not hypothetical. #70 was a completely broken FFV1 write path —
avcodec_open2failing on every call — hidden behind exactly this for however long it had been there. Two tests calledSKIP()andctestsaid "100% tests passed".Every codec these tests need is installed in CI, so a skip there is a real gap rather than graceful degradation. The check now prints the counts and fails the build on any skip.
Two details worth review:
--reporter compact. I tried compact first; it omits the skip count entirely, so the grep would never have fired — a check that silently matches nothing, which is the whole failure mode being fixed.test cases: 41 | 39 passed | 2 skipped) matches the pattern, and the current clean run does not false-positive.Ordering note: this had to land after #71, since it would have failed CI while the FFV1 bug was live. That is the correct behaviour, but it needed the bug fixed first.
Node 20 actions (#68)
actions/checkout@v4→v5andactions/setup-python@v5→v6across all three workflows. Both target Node 20, which GitHub is force-running on Node 24 behind a shim it explicitly calls temporary. When that goes,ci.yml,tla.ymlandrelease.ymlbreak at once for reasons unrelated to any change here.actions/setup-java@v4and the artifact actions did not appear in the deprecation warnings, so they are left alone.Housekeeping
__pycache__/and*.py[cod]added to.gitignore— they have been dirtyinggit statussince the Python bindings landed.🤖 Generated with Claude Code