Skip to content

test: make the job reader fail on a header it does not recognise - #42

Merged
cristiangirlea merged 1 commit into
mainfrom
test/workflow-job-parser
Oct 15, 2026
Merged

cristiangirlea merged 1 commit into
mainfrom
test/workflow-job-parser

Conversation

@cristiangirlea

@cristiangirlea cristiangirlea commented Oct 15, 2026 •

Copy link
Copy Markdown
Owner

Closes #36

What changed

  • tests/python/test_workflows.py
    • jobs(): a job header is now its key two spaces in, quoted or not, with nothing after the colon but a comment ( php: # matrix and "php": are jobs). Its docstring says why the count below exists.
    • test_the_job_reader_sees_every_job: per workflow, the number of jobs read equals the number of runs-on: lines in the file, so a header the reader still does not know fails the test instead of hiding the job from the per-job checks (checkout before compose, no local action, the runner label).
    • test_the_job_reader_on_the_headers_it_must_tell_apart: an inline fixture with a commented header, a double- and a single-quoted key, a comment between jobs, a mismatched-quote header and an anchored header the reader does not know (the count is what catches them), and a top-level key after the jobs whose runs-on: line is counted on purpose, so the guard fails closed.
  • docs/testing.md: the Python worker count is 156 (154 on main).

Tests that ran

  • Red check: the new count against the old reader (the test file as of 8cd6cb8) on tests.yml, each edit alone and reverted: a commented first-job header ( php: # matrix) and a quoted one ( "php":) both left the reader with ['mcp', 'deterministic'] against three runs-on: lines; an anchored header ( deterministic: &gate) left ['php', 'mcp'] against three. With the new reader the commented and quoted headers pass, and the anchored one fails test_the_job_reader_sees_every_job (2 != 3) as well as the per-job runner test.
  • python -m unittest tests.python.test_workflows: 12 tests OK on the host.
  • docker compose -f docker/compose.yaml run --rm --no-deps python-tests: 156 tests OK on this commit (a7ec490). The PHP, integration and MCP layers run in CI on this PR; only test code and docs changed.

Independent review

delivery-reviewer on the first draft of this commit: nothing blocking. Folded into the one commit before pushing: the reader's docstring now says what an unrecognised header does to a first job and to a later one; the runs-on: count lives in one helper, runs_on_lines(), used by the guard and by the fixture; the fixture asserts what matters (the jobs read, the count, and that the two disagree) instead of pinning where a missed job's lines end up; a mismatched-quote header pins the backreference.

Left as they are, all on main before this branch and failing closed: jobs() raises on a jobs: # comment line instead of failing with a message; the pin test's spelling count misses a flow-style step whose uses is not the first key. Filed as one issue (linked below) since both are the text readers' remaining blind spots.

Follow-up issue for the readers' remaining blind spots: #43.

jobs() in test_workflows.py reads the workflows as text and dropped the
lines of any job whose header it did not recognise, so a first job written
`  php:  # matrix` or `  "php":` was never checked by the per-job tests and
nothing said so. The header now accepts a quoted key and a trailing comment,
and a new test counts the `runs-on:` lines of each file against the jobs the
reader found, so a header it still does not know fails the test instead of
hiding the job. A fixture test covers the header shapes it must tell apart,
two it does not know (mismatched quotes, an anchor), a top-level key after
the jobs, and the count tripping on them.

Seen failing with the old reader on tests.yml: a commented or a quoted
first-job header left two jobs against three `runs-on:` lines, and an
anchored header did the same; with the new reader the first two pass and the
anchored one fails the count. docs/testing.md counts the two new tests.

Closes #36
@cristiangirlea
cristiangirlea merged commit 1110485 into main Oct 15, 2026
6 checks passed
@cristiangirlea
cristiangirlea deleted the test/workflow-job-parser branch October 15, 2026 11:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: make test_workflows' job parser fail on a header it does not recognise

1 participant