Repository navigation
test: make the job reader fail on a header it does not recognise - #42
Merged
Merged
Conversation
jobs() in test_workflows.py reads the workflows as text and dropped the lines of any job whose header it did not recognise, so a first job written ` php: # matrix` or ` "php":` was never checked by the per-job tests and nothing said so. The header now accepts a quoted key and a trailing comment, and a new test counts the `runs-on:` lines of each file against the jobs the reader found, so a header it still does not know fails the test instead of hiding the job. A fixture test covers the header shapes it must tell apart, two it does not know (mismatched quotes, an anchor), a top-level key after the jobs, and the count tripping on them. Seen failing with the old reader on tests.yml: a commented or a quoted first-job header left two jobs against three `runs-on:` lines, and an anchored header did the same; with the new reader the first two pass and the anchored one fails the count. docs/testing.md counts the two new tests. Closes #36
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #36
What changed
tests/python/test_workflows.pyjobs(): a job header is now its key two spaces in, quoted or not, with nothing after the colon but a comment (php: # matrixand"php":are jobs). Its docstring says why the count below exists.test_the_job_reader_sees_every_job: per workflow, the number of jobs read equals the number ofruns-on:lines in the file, so a header the reader still does not know fails the test instead of hiding the job from the per-job checks (checkout before compose, no local action, the runner label).test_the_job_reader_on_the_headers_it_must_tell_apart: an inline fixture with a commented header, a double- and a single-quoted key, a comment between jobs, a mismatched-quote header and an anchored header the reader does not know (the count is what catches them), and a top-level key after the jobs whoseruns-on:line is counted on purpose, so the guard fails closed.docs/testing.md: the Python worker count is 156 (154 onmain).Tests that ran
tests.yml, each edit alone and reverted: a commented first-job header (php: # matrix) and a quoted one ("php":) both left the reader with['mcp', 'deterministic']against threeruns-on:lines; an anchored header (deterministic: &gate) left['php', 'mcp']against three. With the new reader the commented and quoted headers pass, and the anchored one failstest_the_job_reader_sees_every_job(2 != 3) as well as the per-job runner test.python -m unittest tests.python.test_workflows: 12 tests OK on the host.docker compose -f docker/compose.yaml run --rm --no-deps python-tests: 156 tests OK on this commit (a7ec490). The PHP, integration and MCP layers run in CI on this PR; only test code and docs changed.Independent review
delivery-reviewer on the first draft of this commit: nothing blocking. Folded into the one commit before pushing: the reader's docstring now says what an unrecognised header does to a first job and to a later one; the
runs-on:count lives in one helper,runs_on_lines(), used by the guard and by the fixture; the fixture asserts what matters (the jobs read, the count, and that the two disagree) instead of pinning where a missed job's lines end up; a mismatched-quote header pins the backreference.Left as they are, all on
mainbefore this branch and failing closed:jobs()raises on ajobs: # commentline instead of failing with a message; the pin test's spelling count misses a flow-style step whoseusesis not the first key. Filed as one issue (linked below) since both are the text readers' remaining blind spots.Follow-up issue for the readers' remaining blind spots: #43.