test: guard schema-probe allowlist against format drift - #312
Open
millsmillsymills wants to merge 1 commit into
Open
test: guard schema-probe allowlist against format drift#312millsmillsymills wants to merge 1 commit into
millsmillsymills wants to merge 1 commit into
Conversation
The scrub step in .github/workflows/schema-probe.yml validates probe.log against two regexes that encode the literal output of compute_schema_drift, with no reference between the two files. A change to the drift message format desyncs the allowlist and the scrub then withholds every genuine drift report. The test lifts the scrub's shell fragments verbatim from the workflow and runs them, asserting that every drift line SCHEMA_EXPECTATIONS can produce is accepted and that lookalikes of both drift formats are rejected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
tests/unit/test_schema_probe_allowlist.py, a guard test for the implicit coupling betweencompute_schema_drift(src/unraid_mcp/clients/unraid.py) and the fail-closed allowlist in thescrubstep of.github/workflows/schema-probe.yml. Nothing in either file references the other, so a change to the drift message format desyncs the allowlist and the scrub then withholds every genuine drift report.The test lifts four shell fragments out of the workflow verbatim — the
field_list,allow_missingandallow_typeassignments plus theunexpected=line carrying thegrep -vEinvocation — orders them by position in the file, and runs them underset -euo pipefailwith the candidate lines on stdin. The patterns are therefore checked by a real shell rather than by a Python copy of them, and a change to the alternation or to the assignment order fails the test rather than being transcribed away. The CI run onubuntu-latestis the authoritative one; a local run uses whatevergrepis onPATH.Coverage:
SCHEMA_EXPECTATIONScan produce passes the scrub. Three probe results span the output space: no type present, every type present but empty, and every type missing its alphabetically first field.[]list repr, reachable only through the type-missing branch) passes.allow_missingrejects everymissing fieldslookalike on its own, so a corpus of those alone would leaveallow_typewith acceptance coverage only._scrub_rejectsasserts the shell wrote nothing to stderr, because a malformed pattern makes grep exit 2 with empty stdout, which the workflow's|| trueswallows.No source or workflow changes.
Closes #293
🤖 Generated with Claude Code