Skip to content

docs(backlog): file the three worktree_gate residuals PR 777 disclosed (BACKLOG #1427) #2527

docs(backlog): file the three worktree_gate residuals PR 777 disclosed (BACKLOG #1427)

docs(backlog): file the three worktree_gate residuals PR 777 disclosed (BACKLOG #1427) #2527

name: quality-advisory
# Advisory-only quality-measurement gates from the Code Quality and Anti-Slop Standards rubric
# (docs/Code_Quality_Standards.md, signals 6-11). Every MEASUREMENT job here runs with `--exit-zero`
# (or continue-on-error) so it can never fail on what it finds -- it surfaces findings for triage only.
# This is the rubric's "advisory-first, ratchet to blocking once the baseline is trusted" rule: raw
# complexity, coverage %, and clone counts are weak / gameable signals (rubric section 4.1) and must
# never be a hard gate. This workflow is NOT in branch protection; adding a job here cannot freeze the repo.
#
# ONE DELIBERATE EXCEPTION: the `liveness` job at the bottom CAN go red, and is built to. It never rules
# on findings -- it rules on whether each gate proved it MEASURED anything at all. A red mark there still
# blocks nothing (this workflow holds no required contexts); it is simply the loudest honest way to say
# a gate has stopped working. See that job's own comment for the three incidents that motivated it.
#
# Signals from the rubric (renumbered by tier -- docs/Code_Quality_Standards.md v0.7), each with how
# it reaches a reviewer:
# * signal 11 - complexity triage (ruff C901) -> BUILT (complexity job); PR-caused DELTA as
# annotations + step summary, full list in the log
# * signal 9 - clone / duplication (jscpd) -> BUILT (clone job); step summary only, see below
# * signal 8 - diff-coverage visibility -> BUILT (coverage job; PR-only); INLINE `::notice`
# annotations on the Files changed tab + summary
# * signal 7 - mutation testing (highest leverage) -> BUILT and WORKING on mutmut 3 (2.5.1 crashed on
# Python 3.14 before producing a mutant, and `|| true`
# made that look green for months). Measured: 461
# mutants in 3s -- 87 killed, 19 survived. Runs on
# PRs too; survivor table in the step summary.
# signal 10 (lint breadth) is a coordinated one-shot sweep, not an advisory job; DORA is a context caveat, not a gate.
#
# ONE JOB HERE IS NOT A RUBRIC SIGNAL, AND IT IS HERE ON PURPOSE (BACKLOG #1431). `dangling-citations`
# reports backlog `#N` references that name no filed item in either ledger. It is a LEDGER-hygiene check,
# not a code-quality measurement, and its natural sibling -- backlog-hygiene.yml -- is the wrong home:
# that workflow's only job IS a required context, so a step added there could block every merge. This
# file is the repository's one place where a check can report without being able to gate, which is the
# property that job needs. Promoting it to blocking is an owner decision and is deliberately untaken;
# see the item for the reasoning and for what the nine current findings actually are.
#
# WHY THIS USES WORKFLOW-COMMAND ANNOTATIONS AND *NOT* CODE SCANNING / SARIF. This was measured, not
# assumed, and the measurements are recorded here so a future session does not "helpfully" add a SARIF
# upload back:
# 1. All 122 C901 findings on this tree are SINGLE-LINE regions anchored on the function-name token of
# the `def`. GitHub renders an alert inline only when its lines are in the diff, so a PR adding
# branching INSIDE a function surfaces nothing, while a signature reflow fires on pre-existing debt.
# 2. jscpd emits ONE location per clone pair, chosen by scan order, so ~8 files can never be the anchor
# and roughly half of newly-introduced clones would anchor on the untouched twin.
# 3. Uploading on pull_request only leaves NO default-branch baseline, so every PR forever would report
# all ~161 findings as new -- not a one-off first-PR blip.
# Workflow commands need no token and no permission grant, and behave identically on fork PRs. Every job
# below therefore holds `contents: read` and nothing else -- least privilege, worth pinning on its own
# merits given that two of these jobs execute third-party code fetched at run time.
#
# What actually keeps this advisory is TWO things, and permissions are neither of them: (a) these job
# contexts are not in the required-checks set on `main`, and (b) every analysis step is
# `continue-on-error: true` plus `--exit-zero` / `--fail-under=0` / `|| true`, so the job reports success
# regardless of findings. Branch-protection membership is the only thing that determines gating.
# tests/test_quality_advisory_invariants.py pins (b), the absent write scopes, and the absence of SARIF.
#
# HONEST LIMIT of the annotation mechanism: a workflow-command annotation renders INLINE on the Files
# changed tab only when its line is in the diff. That always holds for diff-coverage (every line it
# flags is a line the PR changed) but usually does NOT hold for the complexity delta, whose findings
# anchor on a `def` line the PR often did not touch -- those land in the Checks tab and the step summary
# instead. The summary table is the primary surface for complexity, not a nice-to-have.
on:
pull_request:
workflow_dispatch:
schedule:
# Nightly sweep. Every job here is cheap (mutation is ~3s of actual mutating; complexity and clone
# are seconds), so the cron is a safety net that catches drift on `main` rather than a cost dodge.
# Coverage is PR-only -- it needs a base ref to diff against -- so it no-ops here.
- cron: "23 4 * * *"
# Deny-by-default at the workflow level; each job grants only what it needs.
permissions: {}
jobs:
complexity:
# Signal 11 - cyclomatic-complexity triage. Surfaces functions over the mccabe threshold (>10) so large
# units are visible in review. Raw complexity is a WEAK defect predictor (rubric section 2), so this is
# ADVISORY ONLY: --exit-zero reports but never fails, and this job is not a required status check.
name: complexity triage (advisory)
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
receipt: ${{ steps.receipt.outputs.receipt }}
steps:
- name: Check out the source
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
# The delta step needs the base branch's history for `git merge-base`.
fetch-depth: 0
- name: Install ruff (version derived from the lock)
# DERIVED, not duplicated. This job used to hardcode 0.15.19 while all four locks pinned 0.15.22,
# and that drift is load-bearing: c901_delta.py parses ruff's human-readable C901 message because
# ruff exposes the complexity number nowhere else, so a skew can change the wording under the
# parser. The obvious fix -- hardcode the right version and assert it matches the lock in a test --
# would leave a REQUIRED check (the pytest suite) hostage to a routine Dependabot ruff bump:
# the lock moves, the workflow string does not, and a blocking context goes red for a purely
# advisory concern. Reading the lock at run time removes the drift AND the coupling.
run: |
RUFF_PIN="$(sed -n 's/^ruff==\([^ ;]*\).*/\1/p' constraints.lock | head -1)"
if [ -z "$RUFF_PIN" ]; then
echo "::notice title=Ruff pin not found::falling back to the unpinned ruff from PyPI"
pipx install ruff
else
echo "ruff pin from constraints.lock: $RUFF_PIN"
pipx install "ruff==$RUFF_PIN"
fi
- name: Ruff C901 complexity triage (advisory - never gates)
continue-on-error: true
run: |
echo "== functions over mccabe complexity 10 (advisory triage; NEVER a merge gate) =="
ruff check --select C901 --output-format=concise --exit-zero messagefoundry
echo ""
echo "== summary =="
ruff check --select C901 --statistics --exit-zero messagefoundry
- name: Complexity delta vs the merge base (advisory - never gates)
# PR-only: there is no base ref to diff against on the cron or on workflow_dispatch, where the
# whole-repo console triage above remains the full picture.
if: github.event_name == 'pull_request'
continue-on-error: true
env:
BASE_REF: ${{ github.base_ref }}
run: |
echo "== complexity this PR CHANGED (advisory; pre-existing findings deliberately omitted) =="
git fetch --no-tags origin "$BASE_REF" || true
# Steps run under `bash -e`, so an unresolvable merge base would abort this step with no
# explanation at all. Say why instead: an advisory signal that goes quiet should at least
# leave a reason in the log, since nobody watches a green advisory job for absence.
if ! MERGE_BASE="$(git merge-base HEAD "origin/$BASE_REF" 2>/dev/null)"; then
echo "::notice title=Complexity delta skipped::could not resolve a merge base against origin/$BASE_REF"
# Record the bail so the liveness receipt can SEE it. Without this the step exits 0, the
# receipt still reports "measured" off the whole-repo triage, and the PR-caused delta --
# the entire point of this step on a PR -- goes quiet behind a green check. That is the
# failure mode this workflow's liveness job exists to catch, sitting inside it.
# QUOTE the reason: the receipt SOURCES this file, and an unquoted value containing
# spaces parses as a command, silently leaving the variable unset -- which reported
# "no reason recorded" and threw away the one useful diagnostic. Caught by executing
# the receipt shell, not by reading it.
echo "C901_DELTA=skipped" > c901-delta.env
echo "C901_DELTA_REASON='could not resolve a merge base against origin/$BASE_REF'" >> c901-delta.env
exit 0
fi
echo "merge base: $MERGE_BASE"
ruff check --select C901 --output-format=json --exit-zero messagefoundry > c901-head.json
# A second checkout of the merge base, so the "before" numbers come from real ruff output
# rather than a reconstruction. Its absolute paths differ from HEAD's by the extra directory;
# c901_delta.py collapses both to repo-relative keys (regression-tested -- if it did not,
# every function would report as new).
git worktree add --detach base-tree "$MERGE_BASE"
( cd base-tree && ruff check --select C901 --output-format=json --exit-zero messagefoundry ) \
> c901-base.json
python3 scripts/quality/c901_delta.py \
--base c901-base.json \
--head c901-head.json \
--repo-root . \
--summary-file "$GITHUB_STEP_SUMMARY"
# Written LAST, so it exists only if every step above actually completed. Its absence is
# therefore proof the delta died somewhere, which the receipt reports as a dead gate.
BASE_N="$(python3 -c 'import json,sys; print(len(json.load(open(sys.argv[1]))))' c901-base.json)"
HEAD_N="$(python3 -c 'import json,sys; print(len(json.load(open(sys.argv[1]))))' c901-head.json)"
{
echo "C901_DELTA=ok"
echo "C901_DELTA_MERGE_BASE=$MERGE_BASE"
echo "C901_DELTA_BASE_N=$BASE_N"
echo "C901_DELTA_HEAD_N=$HEAD_N"
} > c901-delta.env
- name: Record gate liveness
id: receipt
if: always()
# Routed through env, not interpolated into the shell body (zizmor: template injection).
env:
IS_PR: ${{ github.event_name == 'pull_request' }}
EVENT_NAME: ${{ github.event_name }}
continue-on-error: true
run: |
# Units count what was EXAMINED, never what was found: a repo with zero functions over the
# threshold is good news, and a liveness check that fires on good news gets muted.
# `--show-files` lists exactly the files ruff would check, so it is non-zero iff ruff ran.
FILES="$(ruff check --select C901 --show-files messagefoundry 2>/dev/null | wc -l)"
FINDINGS="$(ruff check --select C901 --output-format=json --exit-zero messagefoundry \
| python3 -c 'import json,sys; print(len(json.load(sys.stdin)))')"
# THE DELTA IS THE POINT ON A PR, and until now this receipt could not see it. `--show-files`
# proves ruff enumerated files, which is true whether or not the delta step ran -- so a delta
# that bailed on an unresolvable merge base left a green "measured" receipt and vanished.
# That is this workflow's own failure mode reproduced inside its liveness control.
#
# On a non-PR event the delta legitimately does not run and the whole-repo triage IS the
# measurement. On a PR its silence is a dead gate, not an absence of news.
if [ "$IS_PR" != "true" ]; then
python3 scripts/quality/liveness.py record \
--signal complexity --status measured \
--units "$FILES" --unit-name "files scanned" \
--evidence "ruff C901 scanned $FILES files; $FINDINGS over the threshold (whole-repo triage; the PR delta does not apply on $EVENT_NAME)" \
--extra "{\"findings\":$FINDINGS}"
exit 0
fi
if [ ! -s c901-delta.env ]; then
python3 scripts/quality/liveness.py record --signal complexity --status failed \
--reason "the PR complexity delta produced no outcome marker; it did not complete (see the job log)"
exit 0
fi
. ./c901-delta.env
if [ "$C901_DELTA" != "ok" ]; then
python3 scripts/quality/liveness.py record --signal complexity --status failed \
--reason "the PR complexity delta did not run: ${C901_DELTA_REASON:-no reason recorded}"
exit 0
fi
python3 scripts/quality/liveness.py record \
--signal complexity --status measured \
--units "$FILES" --unit-name "files scanned" \
--evidence "ruff C901 scanned $FILES files ($FINDINGS over the threshold); delta compared ${C901_DELTA_BASE_N} base vs ${C901_DELTA_HEAD_N} head findings against ${C901_DELTA_MERGE_BASE}" \
--extra "{\"findings\":$FINDINGS,\"delta_base_n\":${C901_DELTA_BASE_N},\"delta_head_n\":${C901_DELTA_HEAD_N}}"
clone:
# Signal 9 - duplication / clone detection. Flags copy-pasted blocks (the "copy-instead-of-abstract"
# AI-slop signature, rubric section 3). ADVISORY: reports duplicates, never fails. The justified
# ~21k-LOC store-backend parity (store/sqlserver.py + store/postgres.py = deliberate T-SQL vs Postgres
# dialect duplication, whitelisted in the rubric) is ignored so it does not drown the real signal.
name: clone detection (advisory)
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
receipt: ${{ steps.receipt.outputs.receipt }}
steps:
- name: Check out the source
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: jscpd clone report (advisory - never gates)
continue-on-error: true
run: |
echo "== copy-pasted blocks in messagefoundry/ (advisory; justified store-backend parity ignored) =="
# Summary only, DELIBERATELY. jscpd ships a SARIF reporter and it works, but each result
# carries exactly ONE location -- whichever file jscpd scanned first -- so the case this
# signal exists for (a PR copy-pasting an existing block into a new place) anchors on the
# untouched twin about half the time. That is a coin flip, not a diff-scoped signal.
# Keep the 4.x pin: npm `latest` is a 5.x Rust rewrite with a different CLI.
npx --yes jscpd@4.0.5 messagefoundry \
--min-tokens 60 \
--ignore "**/store/sqlserver.py,**/store/postgres.py" \
--reporters console,markdown \
--output ./jscpd-report \
|| true
if [ -f jscpd-report/jscpd-report.md ]; then
{
echo "## Duplicate blocks (jscpd)"
echo ""
# $GITHUB_STEP_SUMMARY is capped at 1 MiB per step and an oversized write is DROPPED
# ENTIRELY, so truncate rather than silently lose the whole summary.
head -c 900000 jscpd-report/jscpd-report.md
echo ""
echo "<sub>Advisory only. Clone locations are in the job log — see the note in the"
echo "workflow for why these are not annotated on the diff.</sub>"
} >> "$GITHUB_STEP_SUMMARY"
fi
- name: Record gate liveness
id: receipt
if: always()
continue-on-error: true
run: |
# Files ANALYSED, not clones found -- zero clones is a healthy repo, not a dead gate.
REPORT=jscpd-report/jscpd-report.md
FILES="$(grep -oE 'duplicated lines in [0-9]+' "$REPORT" 2>/dev/null | grep -oE '[0-9]+$' || true)"
CLONES="$(grep -oE 'Found [0-9]+ exact clones' "$REPORT" 2>/dev/null | grep -oE '[0-9]+' || true)"
python3 scripts/quality/liveness.py record \
--signal clone --status measured \
--units "${FILES:-0}" --unit-name "files analysed" \
--evidence "jscpd analysed ${FILES:-0} files; found ${CLONES:-0} clones" \
--extra "{\"clones\":${CLONES:-0}}"
coverage:
# Signal 8 - diff-coverage visibility. Runs the suite under coverage, then reports coverage of the
# lines CHANGED by this PR (diff-cover) -- never a whole-repo % gate, which the rubric (section 4.1)
# calls a gameable signal. ADVISORY: --fail-under=0 + `|| true` so it never fails a build, and this
# job is not a required status check. PR-only (needs a base ref to diff against). It re-runs the
# ubuntu suite (~a few min) per PR; add a paths filter if that cost bites. (This used to say
# "move it to the mirror" -- there is no mirror since the cutover; development happens directly
# on the public repo.)
name: diff-coverage (advisory)
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
# SIZED FROM A MEASUREMENT, NOT FROM A GUESS (BACKLOG #1409). This cap was 20 from the
# 2026-07-26 publish snapshot until 2026-09-01, and it was never edited. The SUITE grew into
# it: the job took 9.7 minutes on 2026-07-26, 14.8 on 2026-08-06, and 19m36s on its LAST GREEN
# run (job 97249962177, 2026-08-23 15:18 CDT) -- 24 seconds of margin.
#
# 2026-08-23 IS THE LAST-GREEN BOUNDARY, NOT THE FIRST-KILL ONE, and the difference matters for
# anyone reconstructing this. The first cap kill was 2026-08-08 (job 93153178205, cancelled at
# 20m16s), a fortnight earlier. The job then sat ON the boundary, sometimes finishing and
# sometimes not, until 2026-08-23 was simply the last time it made it. A cap is not crossed
# cleanly; it is crossed intermittently first, and that window is the one where the signal is
# already unreliable while still looking occasionally healthy.
#
# After 2026-08-23 the kill was consistent: `The operation was canceled` and no verdict. 714
# pull_request runs of this workflow were created between that green job and the fix; in a
# 30-run sample, 28 of 28 COMPLETED coverage jobs were cancelled and none succeeded. (An
# earlier draft said "over 700 of them across 8 days" as though every run were a confirmed
# kill. 714 is the RUN population; the all-kills half is a sample and is now written as one.)
# That is the failure this file's `liveness` job exists to catch, and it
# could not see it (BACKLOG #1410, fixed in scripts/quality/liveness.py in the same change).
#
# WHERE THE TIME WENT, measured across four jobs (99554977847, 99578445996, 99580323074,
# 99685011404): setup -- checkout, setup-python, setup-uv, the Qt apt install and the editable
# install -- totals 23 to 36 SECONDS, 2 to 3 percent of the budget. The pytest step took
# 19m37s to 19m49s and reached exactly [ 91%] every time. So splitting the install out of this
# job buys nothing; the measurement is the whole cost, and the fix is the `-m 'not tooling'`
# selection on the pytest line below, not a bigger box.
#
# THE CAP IS NOW SIZED ON A REAL GREEN RUN, and it replaces every estimate that stood here.
# Job 99720329464 on PR 724 (2026-09-01 02:52:25Z to 03:06:58Z) finished SUCCESSFULLY in
# 14m33s: `11510 passed, 839 skipped, 2731 deselected in 823.75s (0:13:43)`, and step 8 emitted
# a diff-cover verdict -- the first this gate had produced in eight days. Against 30 minutes
# that is 15m27s of headroom, about 2.1x.
#
# 2.1x IS DELIBERATE, NOT LEFTOVER SLACK. The convention elsewhere is ~1.5x, which here would
# be 22 minutes. The defect this comment documents WAS a cap with thin margin that the suite
# silently grew into, so the margin is the mitigation and shrinking it to convention would
# rebuild the trap. Re-derive if the measured run approaches 20 minutes, not before.
#
# THREE ESTIMATES THAT STOOD HERE WERE WRONG, and they are named rather than deleted because
# this block's whole authority is that its numbers were measured.
# "no coverage run has ever finished" -- FALSE, and it contradicted this same comment two
# paragraphs up. Job 97249962177, named above as the last green run, reached [100%] with
# `13046 passed, 1380 skipped in 1130.96s (0:18:50)`. A completed run existed all along.
# "the honest measured floor is 21m35s for the UNNARROWED suite" -- WRONG, wrong in the
# UNSAFE direction, and not a measurement at all. 21m35s is 19m40s/0.91, which assumes the
# last 9 percent costs what the first 91 did. pytest's percentage counts TESTS, not time,
# and the tail is 15 to 26 times more expensive per point than the average, so that
# division is invalid. Two independent extrapolations, by different routes, both land well
# above it: applying the observed tail rate (0.35-0.45 s/test, from the four killed jobs'
# own logs) to the unreached tests gives 26 to 28 minutes, and scaling from the last green
# run's curve (91 percent at 602.9s of a 1110.3s span, so the final 9 percent took 45
# percent of the run) gives about 33. Call it 26 to 33 MINUTES, extrapolated -- and note
# that the whole range sits at or above this 30-minute cap. So a bigger box alone would NOT
# have fixed this. The narrowing is what brings it to a measured 14m33s, which is the
# opposite of what 21m35s implied.
# "ESTIMATED 18 to 20 minutes for the narrowed serial run" -- superseded by the 14m33s above.
#
# NEXT LEVER IF IT DRIFTS AGAIN: pytest-xdist. ci.yml's engine leg runs `-n 4 --dist loadfile`
# and this job does not. Adding it here needs `[tool.coverage.run] parallel = true` plus a
# combine before --cov-report=xml, so it is a separate change with its own verification --
# deliberately NOT bundled here, or a shifted coverage number would have two possible causes.
timeout-minutes: 30
permissions:
contents: read
outputs:
receipt: ${{ steps.receipt.outputs.receipt }}
steps:
- name: Check out the source
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
fetch-depth: 0
- name: Set up Python 3.14
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.14"
- name: Set up uv
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
- name: Qt headless system deps
# BOUNDED AND RETRIED. Unguarded, this step turns an external apt-mirror hiccup into a
# blocked merge queue: measured 2026-08-18, three hangs across two attempts on three
# different jobs, one of them 27+ minutes on `test (ubuntu-latest, py3.14)` -- a REQUIRED
# context, so the queue stops. Two levers, because they answer different failures. The
# per-command `timeout` kills a HANG and lets the retry run; `timeout-minutes` is the
# backstop that stops this step ever eating a job budget again if the loop is edited wrong.
# The failure text names the cause, so the next reader is not sent hunting in their diff --
# a check whose label points at the wrong subject is what BACKLOG #1254 is about.
timeout-minutes: 8
run: |
for attempt in 1 2 3; do
if sudo timeout 120 apt-get update && sudo timeout 180 apt-get install -y libegl1 libgl1 libxkbcommon0 libdbus-1-3; then
exit 0
fi
echo "::warning::apt attempt ${attempt}/3 failed or timed out; retrying"
sleep $((attempt * 5))
done
echo "::error::apt-get failed 3 times. This is the UBUNTU RUNNER MIRROR, not the change under test."
exit 1
- name: Install project + coverage tools
run: |
uv pip install --system --constraint constraints.lock -e ".[dev,harness,fhir,dicom,x12,xml,webauthn]" -e packaging/messagefoundry-webconsole
# HASH-PINNED from the CI toolchain lock (ADR 0034 §3). These tools now DO live in
# pyproject.toml ([dependency-groups].ci-quality, where each pin's rationale sits) and DEP-1
# re-exports this lock by design -- superseding the note that used to sit here claiming they
# were deliberately kept out of pyproject to avoid tripping that gate.
#
# `python -m pip`, not `uv pip install --system`, for ONE honest reason: every other hashed
# install in this repo is a pip install, and keeping one spelling means one thing for the
# guards in tests/test_ci_venv_pinning.py to match. Same setup-python interpreter either way.
#
# NOT because uv cannot: `uv pip install --require-hashes` exists (env UV_REQUIRE_HASHES), and
# setup-uv is already in this job -- the earlier claim that "--require-hashes is pip's
# contract" was simply wrong. NOR for Scorecard's benefit: its parser keys on
# pip/pip3/`python -m pip`, so a `uv pip` line generates NO finding at all, and converting one
# can only ADD parseable surface -- it cannot close an alert. This line is posture-neutral to
# Scorecard and is here for uniformity.
#
# Safe against the editable install above: the group lock and constraints.lock agree on all 10
# shared packages (pytest and pytest-timeout among them), so this cannot re-point what the
# coverage run executes under.
python -m pip install --require-hashes -r ci/locks/ci-quality.lock
- name: Tests under coverage (advisory - never fails)
continue-on-error: true
env:
QT_QPA_PLATFORM: offscreen
# `-m 'not tooling'` DESELECTS THE TOOLING TIER, AND IT COSTS THIS GATE NOTHING (#1409).
# The mark is applied at runtime by tests/conftest.py from tests/tooling_manifest.txt (132
# files), not by decorators, so grepping for `pytest.mark.tooling` finds 2 hits and badly
# under-reads its size. ci.yml carries the same flag on its engine leg and runs the tier as
# its own parallel job; this job was the only place in CI running it unfiltered AND serial.
#
# WHY DESELECTING IT REMOVES NO SIGNAL: the report is scoped to `--cov=messagefoundry`, and
# ZERO of the 132 manifest files import that package. Measured with the repo's OWN
# instrument, the _ENGINE_IMPORT regex in tests/test_tooling_partition.py, against a
# positive control of 507 files under tests/ that DO import it -- so the zero is a real
# absence and not a broken pattern. A test that never imports the measured
# package cannot contribute a covered line through its OWN test bodies. The tier is the
# run's hard tail, and the honest form of that is a RANGE, not one job's pair of numbers:
# across the four killed jobs the suite reaches 89 percent at 14.1 to 15.6 minutes, and the
# next two percentage points then cost 3.5 to 4.7 minutes. An earlier draft said "89
# percent in 15.2 minutes, then the next two points cost 3.4 minutes", which is one job's
# figures presented as the suite's -- and even for that job the second number is 3.47.
# Note the percentages count TESTS, not time, which is exactly why the tail is so much
# more expensive per point than the average.
#
# ONE QUALIFICATION, because "costs this gate NOTHING" is stronger than the grep supports.
# The zero counts DIRECT imports. Reach is not the same thing: tests/test_dast_claims.py is
# in the manifest and imports scripts.security.dast_auth_sweep, which pulls messagefoundry
# modules transitively, and every import-time line of those counts as covered under --cov.
# So deselecting the tier can drop some IMPORT-TIME coverage even though no tooling test
# exercises the package directly. The measured effect is small and the trade is still
# plainly worth it -- 14m33s against a job that produced no number at all -- but the honest
# claim is "costs this gate almost nothing", not "nothing".
#
# THAT CONTROL WENT 510 -> 508 -> 507, AND THE LAST STEP IS THE INTERESTING ONE.
# 510 is the same regex with the trailing word boundary dropped, so it also matched
# messagefoundry_webconsole -- a DIFFERENT package. Adding \b gives 508. But 508 is still
# one too many: tests/test_lifespan_startup_unwinds.py matches at lines 48-50, and those
# lines sit INSIDE a 1256-character string literal beginning at line 42, which holds
# child-process source written to a temp file. It is not an import this process executes.
# An AST census (ast.Import / ast.ImportFrom over all 729 .py files under tests/, 0
# unparseable) returns 507, and the regex-minus-AST difference is exactly that one file,
# with no files the regex missed in the other direction.
#
# WHY THIS IS RECORDED AND NOT JUST FIXED: 508 was reported by two separate verification
# passes that agreed with each other, and both were wrong the same way, because both ran
# the SAME line-anchored regex. Agreement between runs of one instrument is one
# measurement, not two. Only a different METHOD -- parsing instead of matching -- could see
# a string literal. The ZERO this controls holds under every spelling tried, so no
# conclusion moved; the control's own size did, twice.
#
# THE WEB CONSOLE TIER STAYS, and that is the one place this job must NOT copy ci.yml.
# Both ci.yml legs carry `--ignore-glob='*messagefoundry-webconsole*'` (lines 824 and 1134)
# because that package has its own job over there. Here it would DELETE REAL COVERAGE: 16 of
# the tier's 17 PYTHON files import `messagefoundry` -- the 15 test modules plus conftest.py,
# i.e. every file except a WebAuthn test double. They stand up a live `Engine` via
# `Engine.create()` (searching for the constructor `Engine(` returns ZERO, so that needle is
# recorded here beside its zero), and 13 of them build `create_app` and `AuthService`. They
# exercise AT LEAST api/, auth/, pipeline/, config/, store/ and transports/signing -- "at
# least" because a closed enumeration is a liability (SDS-3.6), and __main__ is imported too.
# Collecting the tier alone loads 174 messagefoundry.* modules, every import-time line of
# which --cov records as covered, which is what makes this the ONLY place in CI where the
# tier reaches a coverage number. Measured 130 to 140 seconds in its own ubuntu ci.yml leg
# (177 to 237 on Windows), which is a cheap price for coverage this gate reports.
#
# THIS PARAGRAPH PREVIOUSLY SAID "16 of those TEST FILES" AND "114 to 134 seconds"; neither
# reproduced, and both are named rather than silently edited. The tier holds 15 test modules,
# so a reader who asks the tree for 16 test files does not find them. The web console leg
# measures 130-140s on ubuntu across the last 10 green ci.yml runs on main, and 114 is below
# every sample taken. The conclusion "a cheap price" survives at 140s.
run: pytest -q -m 'not tooling' --cov=messagefoundry --cov-report=xml --cov-report=term-missing:skip-covered || true
- name: Diff-coverage vs the PR base (advisory - never fails)
continue-on-error: true
env:
BASE_REF: ${{ github.base_ref }}
run: |
echo "== coverage of the lines this PR changed (advisory; not a whole-repo % gate) =="
# NO --depth HERE. Running a shallow fetch against the COMPLETE clone that `fetch-depth: 0`
# just produced writes .git/shallow and grafts away everything behind origin/<base>'s tip.
# diff-cover diffs with the three-dot range `origin/<base>...HEAD`, which needs a merge base --
# so once the base branch advances past the PR's merge ref (another PR merging, or a re-run of
# a stale check) the diff dies with "no merge base". Reproduced. It then fails in the worst
# possible way: the markdown reporter runs BEFORE the annotations reporter and truncates
# diff-cover.md on open, so `|| true` swallows the error, zero annotations are emitted, and the
# summary shows a bare heading that reads as "nothing uncovered".
git fetch --no-tags origin "$BASE_REF" || true
if [ ! -s coverage.xml ]; then
echo "::notice title=Diff coverage skipped::coverage.xml was not produced (the test step did not complete)"
exit 0
fi
# `github-annotations:notice` writes `::notice file=...,line=N,endLine=M` workflow commands to
# STDOUT, which the runner turns into INLINE annotations on the Files changed tab -- no token,
# no permission grant, identical behaviour on fork PRs. Adjacent uncovered lines are coalesced
# into ranges, so a long uncovered block costs one annotation rather than one per line. Every
# line flagged is a line this PR changed, so there is no pre-existing-debt noise by construction.
# The console report still prints, so the job log is unchanged.
diff-cover coverage.xml --compare-branch="origin/$BASE_REF" --fail-under=0 \
--format "github-annotations:notice,markdown:diff-cover.md" 2>&1 | tee diff-cover-console.txt || true
# `-s` not `-f`: diff-cover creates and truncates this file BEFORE it can fail, so a crashed
# run leaves a 0-byte report. Testing for existence alone would append an empty section that
# looks like a clean result -- the exact silent-success shape this workflow keeps producing.
if [ -s diff-cover.md ]; then
{
echo "## Diff coverage"
echo ""
# The markdown report embeds a source snippet per uncovered line, so it can get large.
# $GITHUB_STEP_SUMMARY is capped at 1 MiB per step and an oversized write is DROPPED
# ENTIRELY -- truncate rather than silently lose the whole summary.
head -c 900000 diff-cover.md
} >> "$GITHUB_STEP_SUMMARY"
else
echo "::notice title=Diff coverage unavailable::diff-cover produced no report (see the job log)"
fi
- name: Record gate liveness
id: receipt
if: always()
# BASE_REF is needed here too -- without it the evidence string read "against origin/".
env:
BASE_REF: ${{ github.base_ref }}
continue-on-error: true
run: |
# THE DISTINCTION THIS WHOLE CHECK RESTS ON, and the place it is easiest to get wrong.
#
# "No lines with coverage information in this diff" is a real, correct outcome -- it is what
# PRs #18 and #19 produced, because neither touched measured source. But diff-cover prints
# that SAME line when coverage.xml itself measured nothing (its console template branches on
# whether any source has measured lines, and never consults the diff size). So the message
# alone cannot tell "nothing to measure" from "the measurement is dead" -- exactly the
# ambiguity this control exists to resolve, one layer up. Reproduced with a valid-but-empty
# coverage.xml against a diff that really did change six lines.
#
# So: prove coverage.xml measured something before accepting the not-applicable branch, and
# treat a MISSING coverage.xml as a dead gate (failed), never as inapplicable -- the same way
# the mutation job treats a missing mutmut-counts.env.
if [ ! -s coverage.xml ]; then
python3 scripts/quality/liveness.py record --signal coverage --status failed \
--reason "coverage.xml was not produced; the measurement never happened"
exit 0
fi
MEASURED_FILES="$(grep -c '<class ' coverage.xml || true)"
if [ "${MEASURED_FILES:-0}" -eq 0 ] 2>/dev/null; then
python3 scripts/quality/liveness.py record --signal coverage --status failed \
--reason "coverage.xml contains no measured files; diff-cover had nothing to compare against"
exit 0
fi
if grep -q "No lines with coverage information" diff-cover-console.txt 2>/dev/null; then
python3 scripts/quality/liveness.py record --signal coverage --status not-applicable \
--reason "no lines with coverage information in this diff; coverage.xml measured ${MEASURED_FILES} files, so the tooling is live and this PR simply changed no measured source"
else
LINES="$(grep -oE 'Total:[[:space:]]+[0-9]+' diff-cover-console.txt 2>/dev/null \
| grep -oE '[0-9]+' | head -1 || true)"
python3 scripts/quality/liveness.py record --signal coverage --status measured \
--units "${LINES:-0}" --unit-name "changed lines analysed" \
--evidence "diff-cover reported on ${LINES:-0} changed lines against origin/${BASE_REF}; coverage.xml measured ${MEASURED_FILES} files"
fi
mutation:
# Signal 7 - mutation testing (advisory; the HIGHEST-leverage gate -- it adversarially checks the
# tests actually assert something, catching the "tests that never fail" AI-slop mode, rubric section 3).
#
# NOT expensive, contrary to the note that used to sit here. MEASURED on Python 3.14 (linux, the
# bounded scope below): 461 mutants in 3 SECONDS -- 87 killed, 19 survived, 355 not covered by the
# scoped test file. mutmut 3 only runs the tests that actually cover each mutant, so the old
# "runs the suite once per mutant" cost model does not apply. That is why this now runs on PRs too:
# a real signal for seconds of runtime is worth having in review, which is where survivors get fixed.
name: mutation (advisory)
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
outputs:
receipt: ${{ steps.receipt.outputs.receipt }}
steps:
- name: Check out the source
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Set up Python 3.14
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.14"
- name: Set up uv
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
- name: Qt headless system deps
# BOUNDED AND RETRIED. Unguarded, this step turns an external apt-mirror hiccup into a
# blocked merge queue: measured 2026-08-18, three hangs across two attempts on three
# different jobs, one of them 27+ minutes on `test (ubuntu-latest, py3.14)` -- a REQUIRED
# context, so the queue stops. Two levers, because they answer different failures. The
# per-command `timeout` kills a HANG and lets the retry run; `timeout-minutes` is the
# backstop that stops this step ever eating a job budget again if the loop is edited wrong.
# The failure text names the cause, so the next reader is not sent hunting in their diff --
# a check whose label points at the wrong subject is what BACKLOG #1254 is about.
timeout-minutes: 8
run: |
for attempt in 1 2 3; do
if sudo timeout 120 apt-get update && sudo timeout 180 apt-get install -y libegl1 libgl1 libxkbcommon0 libdbus-1-3; then
exit 0
fi
echo "::warning::apt attempt ${attempt}/3 failed or timed out; retrying"
sleep $((attempt * 5))
done
echo "::error::apt-get failed 3 times. This is the UBUNTU RUNNER MIRROR, not the change under test."
exit 1
- name: Install project + mutmut
run: |
uv pip install --system --constraint constraints.lock -e ".[dev,harness,fhir,dicom,x12,xml,webauthn]" -e packaging/messagefoundry-webconsole
# mutmut 3 + pytest-timeout, HASH-PINNED from the CI toolchain lock (ADR 0034 §3). Both live in
# pyproject.toml's [dependency-groups].ci-quality, which records why each is load-bearing:
# `mutmut<3` resolved to 2.5.1, which CRASHES on Python 3.14 in its pony-ORM cache
# (`cannot pickle 'itertools.count'`) before generating a single mutant -- verified from run
# 30248096425; and pytest-timeout is REQUIRED because mutmut 3 always passes `--timeout`/
# `--timeout-method` to pytest, so without the plugin every invocation dies with an
# unrecognised-argument error surfacing only as BadTestExecutionCommandsException.
# `python -m pip` for UNIFORMITY with every other hashed install in this repo, not because uv
# cannot do it (`uv pip install --require-hashes` exists) and not for Scorecard (a `uv pip`
# line generates no finding at all) -- see the coverage job's install step for the full note.
python -m pip install --require-hashes -r ci/locks/ci-quality.lock
- name: Mutation-test a bounded scope (advisory - never fails)
continue-on-error: true
env:
QT_QPA_PLATFORM: offscreen
run: |
echo "== mutation testing messagefoundry/parsing/binary.py (advisory; bounded scope) =="
# Ephemeral mutmut config (the repo ships none). THREE things here are load-bearing, each
# found by a run that silently produced nothing:
# * source_paths is the PACKAGE, not the one file. mutmut 3 copies source_paths into
# `mutants/` and runs pytest THERE; with a single file copied, tests/conftest.py died on
# `ModuleNotFoundError: messagefoundry.config` and every mutant came back "not checked".
# * only_mutate keeps the bounded scope -- the whole package is copied, one module mutated.
# * pytest_add_cli_args_test_selection replaces mutmut 2's `runner=`; the old `runner` and
# `paths_to_mutate` keys are gone/deprecated in 3.x.
printf '[mutmut]\nsource_paths=messagefoundry\nonly_mutate=messagefoundry/parsing/binary.py\npytest_add_cli_args_test_selection=tests/test_binary_carriage.py\n' > setup.cfg
if mutmut run > mutmut-run.txt 2>&1; then
MUTMUT_OK=1
else
MUTMUT_OK=0
fi
tail -c 20000 mutmut-run.txt
if [ "$MUTMUT_OK" = "1" ]; then
mutmut results > mutmut-results.txt 2>&1 || true
# `mutmut results` lists ONLY the mutants worth looking at -- survived / no tests /
# timeout / suspicious. Killed mutants are NEVER listed, so `grep -c ': killed'` is
# always 0 and a naive table reports "Killed 0" on a perfectly healthy run. Derive it:
# the run's final progress line carries the total (`461/461`), and every non-killed
# mutant is exactly one line in the results file. Verified against run 30308667584:
# 461 total - 374 listed = 87 killed, matching mutmut's own counter.
TOTAL="$(tr '\r' '\n' < mutmut-run.txt | grep -oE '[0-9]+/[0-9]+' | tail -1 | cut -d/ -f2)"
LISTED="$(grep -cE '^[[:space:]]+\S+: ' mutmut-results.txt || true)"
SURVIVED="$(grep -c ': survived' mutmut-results.txt || true)"
NOTESTS="$(grep -c ': no tests' mutmut-results.txt || true)"
# Counted INDEPENDENTLY, never as `LISTED - SURVIVED - NOTESTS`. Deriving it as a
# remainder would make the liveness reconciliation tautological -- the parts would add up
# by construction and the check could never fire.
#
# The alternation must cover EVERY non-killed status mutmut can print, or the sum breaks
# on a healthy run and the liveness job reddens for no reason. A check whose only
# reachable failure is a false positive gets muted within a month, which would leave us
# worse off than having no check.
OTHER="$(grep -cE ': (timeout|suspicious|skipped|segfault|not checked|caught by type check|check was interrupted by user)$' mutmut-results.txt || true)"
# mutmut's OWN killed counter, off its final progress line. KILLED above is derived a
# completely different way (total minus listed); the liveness check reconciles the two.
# Two independent derivations agreeing is what actually protects the number -- the sum
# check alone cannot, because TOTAL cancels out of it algebraically.
KILLED_REPORTED="$(tr '\r' '\n' < mutmut-run.txt | grep -oE '🎉 [0-9]+' | tail -1 \
| grep -oE '[0-9]+' || true)"
if [ -n "$TOTAL" ] && [ "$TOTAL" -ge "$LISTED" ] 2>/dev/null; then
KILLED=$((TOTAL - LISTED))
else
# Never invent a number: if the total could not be read, say so rather than print 0.
KILLED="?"
fi
echo "== total=$TOTAL killed=$KILLED survived=$SURVIVED no-tests=$NOTESTS =="
grep ': survived' mutmut-results.txt || true
{
echo "## Mutation testing — \`messagefoundry/parsing/binary.py\`"
echo ""
echo "| Killed | Survived | Not covered |"
echo "| --- | --- | --- |"
echo "| $KILLED | **$SURVIVED** | $NOTESTS |"
echo ""
if [ "$SURVIVED" != "0" ]; then
echo "Survivors — injected bugs the tests did **not** catch:"
echo ""
echo '```'
grep ': survived' mutmut-results.txt | head -c 100000
echo '```'
fi
echo ""
echo "<sub>Advisory only. A survivor is a hint to strengthen an assertion, not a defect —"
echo "and mutation score is a poor single number (rubric §2), so this never gates.</sub>"
} >> "$GITHUB_STEP_SUMMARY"
# Stash the breakdown for the liveness receipt in the next step.
{
echo "MUT_TOTAL=$TOTAL"
echo "MUT_KILLED=$KILLED"
echo "MUT_SURVIVED=$SURVIVED"
echo "MUT_NOTESTS=$NOTESTS"
echo "MUT_OTHER=$OTHER"
echo "MUT_LISTED=$LISTED"
echo "MUT_KILLED_REPORTED=${KILLED_REPORTED:-}"
} > mutmut-counts.env
else
echo "::warning title=Mutation run failed::mutmut exited non-zero — see the job log"
{
echo "## Mutation testing — FAILED TO RUN"
echo ""
echo "\`mutmut run\` exited non-zero, so this signal produced nothing. Tail of the run:"
echo ""
echo '```'
tail -c 4000 mutmut-run.txt
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
fi
- name: Record gate liveness
id: receipt
if: always()
continue-on-error: true
run: |
if [ ! -s mutmut-counts.env ]; then
# mutmut crashed. This is the 2.5.1 case verbatim -- and it must NOT be silent.
python3 scripts/quality/liveness.py record --signal mutation --status failed \
--reason "mutmut produced no counts; the run did not complete (see the job log)"
exit 0
fi
. ./mutmut-counts.env
# `?` means the mutant total was unreadable, so KILLED is underivable. Interpolating it into
# the --extra JSON would emit `"killed":?` -- malformed, and it would surface as "receipt is
# not valid JSON" rather than the real diagnosis.
case "$MUT_KILLED" in
''|*[!0-9]*)
python3 scripts/quality/liveness.py record --signal mutation --status failed \
--reason "could not read mutmut's mutant total from the progress line; the killed count is underivable"
exit 0
;;
esac
# An EMPTY results file yields LISTED=0, hence killed=TOTAL, survived=0 -- a flawless score
# that reconciles perfectly. A 100% kill rate with nothing listed is not a triumph, it is a
# tool that produced no output. Refuse to report it as success.
if [ "${MUT_LISTED:-0}" -eq 0 ] && [ "${MUT_TOTAL:-0}" -gt 0 ] 2>/dev/null; then
python3 scripts/quality/liveness.py record --signal mutation --status failed \
--reason "mutmut listed no mutants at all while reporting $MUT_TOTAL processed; a perfect score here means the results file was empty, not that every mutant died"
exit 0
fi
EXTRA="{\"killed\":$MUT_KILLED,\"survived\":$MUT_SURVIVED,\"no_tests\":$MUT_NOTESTS,\"other\":$MUT_OTHER"
case "$MUT_KILLED_REPORTED" in
''|*[!0-9]*) ;;
*) EXTRA="$EXTRA,\"killed_reported\":$MUT_KILLED_REPORTED" ;;
esac
EXTRA="$EXTRA}"
python3 scripts/quality/liveness.py record \
--signal mutation --status measured \
--units "$MUT_TOTAL" --unit-name "mutants processed" \
--evidence "mutmut processed $MUT_TOTAL mutants; $MUT_LISTED listed as non-killed" \
--extra "$EXTRA"
- name: Upload the mutmut results (advisory)
if: always()
continue-on-error: true
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: mutmut-results
path: |
mutmut-results.txt
mutmut-run.txt
.mutmut-cache
# .mutmut-cache is a DOTFILE and upload-artifact skips hidden files by DEFAULT. Without this
# the step logs "No files were found", uploads nothing, and still reports success -- a silent
# no-op that looks green.
include-hidden-files: true
# Deliberately `warn`, not `ignore`: if mutmut ever renames its cache, that must be visible in
# the log rather than silently swallowed.
if-no-files-found: warn
dangling-citations:
# BACKLOG #1431 -- a working detector that ran nowhere.
#
# `scripts/docs/dangling_citation_check.py` reports a `#N` backlog citation that names no item in
# EITHER ledger. Until this job, the SCRIPT was invoked by no workflow and no pre-commit hook:
# measured at 46ea10a78 with controls in the same pass, the script path occurs 0 times across
# .github/ plus .pre-commit-config.yaml, while backlog_citation_check.py (a real `run:`) occurs
# once and an invented path occurs 0 times. Only its TEST was wired, via ci.yml's DOC_GUARDS.
#
# WHY IT RUNS HERE AND CANNOT GATE. Four independent things keep it advisory, and
# tests/test_dangling_citation_advisory.py pins the three a repository test can see:
# 1. this workflow holds no required context (.github/required-contexts.txt records that it must
# never be promoted) -- branch-protection membership is the only thing that decides gating;
# 2. `--advisory` makes a live-shape finding exit 0 rather than 1, which is this tool's default;
# 3. `continue-on-error: true` means even the checker's own empty-population REFUSAL cannot fail
# the job -- it shows as a warning on a green job, which is the visibility this wants;
# 4. this job is deliberately absent from `liveness`'s `needs`, so it cannot redden the one job
# in this file that is built to go red.
#
# THE EXIT CODE IS NOT SWALLOWED WITH `|| true`, and that is the point of (3). With `--advisory`
# the only way this tool exits non-zero is a GENUINE malfunction -- most plausibly its refusal to
# report on an empty population, which is what running from the wrong directory looks like. A
# clean scan and a scan that read nothing must not render alike, so that case stays visible as a
# red step inside a green job rather than being turned into a silent success.
#
# NO PATHS FILTER ON THIS WORKFLOW, which is what makes the job worth having: a citation is
# introduced by editing PROSE, and the pytest tier that holds this tool's unit test is skipped on
# a documentation-only pull request. This job runs on all of them.
name: dangling backlog citations (advisory)
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Check out the source
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Report backlog citations that name no filed item (advisory - never gates)
continue-on-error: true
run: |
echo "== backlog citations under docs/ that name no item in either ledger =="
echo "== advisory: this reports, it never gates a merge =="
TOOL_STATUS=0
python3 scripts/docs/dangling_citation_check.py --advisory \
> dangling-citations.txt 2>&1 || TOOL_STATUS=$?
cat dangling-citations.txt
{
echo "## Dangling backlog citations"
echo ""
echo "Advisory only. A reported token is an UPPER BOUND ON CITATIONS, not a count of"
echo "defects: a number at or below the allocator high-water mark can never be issued,"
echo "and a PR, issue or foreign-repo reference is not a backlog citation at all. Both"
echo "kinds are printed so a human judges them. See BACKLOG item 1431 for the triage."
echo ""
echo '```'
# $GITHUB_STEP_SUMMARY is capped at 1 MiB per step and an oversized write is DROPPED
# ENTIRELY, so truncate rather than silently lose the whole summary.
head -c 200000 dangling-citations.txt
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
if [ "$TOOL_STATUS" -ne 0 ]; then
echo "::warning title=Dangling-citation scan did not complete::the checker exited ${TOOL_STATUS} in advisory mode, which means a malfunction rather than a finding. See the job log."
fi
# Re-raise the checker's status. `continue-on-error` above keeps the JOB green; this keeps a
# broken scan distinguishable from a clean one.
exit "$TOOL_STATUS"
liveness:
# THE META-GATE. Every other job here is built so it can never fail; this one is built so it CAN.
#
# Three separate signals in this workflow spent months reporting success while measuring nothing:
# diff-coverage (a shallow fetch destroyed its merge base, and the empty report looked clean),
# mutation (mutmut 2.5.1 crashed before producing a mutant, and `|| true` made that green in 37s),
# and the killed count (a grep for a line mutmut never prints, so a healthy run said "Killed 0").
# The rubric's anti-metric rule guards against trusting a NUMBER too much. Nothing guarded against
# trusting a GREEN CHECK THAT NEVER RAN. This job is that control.
#
# It reads each job's liveness receipt and demands proof of EXECUTION -- a count of things
# examined -- or an explicit, reasoned declaration that there was nothing to measure. It does not
# demand findings: a clean repo legitimately has zero clones, and a gate that fires on good news
# gets muted, which would leave us worse off than before.
#
# DELIBERATELY has no `continue-on-error` and no `|| true`. A dead gate should be loud, and a red
# mark here blocks nothing -- this job is not, and must never become, a required status check.
name: gate liveness (advisory)
if: always()
needs: [complexity, clone, coverage, mutation]
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Check out the source
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Verify every gate proved it measured something
# `needs` carries each job's result AND its receipt output. Routed through `env:` rather than
# interpolated into the shell body -- the receipts are job-authored strings, and expanding
# them into a command line is the template-injection shape zizmor exists to catch.
env:
NEEDS_JSON: ${{ toJSON(needs) }}
run: python3 scripts/quality/liveness.py verify --needs-json "$NEEDS_JSON"