backlog: file items 1307/1308, land 1309, and retire 1311 as a pointer (rebuilt off 298a2bf9) #1353

Workflow file for this run

.github/workflows/quality-advisory.yml at 6071cdb

	name: quality-advisory

	# Advisory-only quality-measurement gates from the Code Quality and Anti-Slop Standards rubric
	# (docs/Code_Quality_Standards.md, signals 6-11). Every MEASUREMENT job here runs with `--exit-zero`
	# (or continue-on-error) so it can never fail on what it finds -- it surfaces findings for triage only.
	# This is the rubric's "advisory-first, ratchet to blocking once the baseline is trusted" rule: raw
	# complexity, coverage %, and clone counts are weak / gameable signals (rubric section 4.1) and must
	# never be a hard gate. This workflow is NOT in branch protection; adding a job here cannot freeze the repo.
	#
	# ONE DELIBERATE EXCEPTION: the `liveness` job at the bottom CAN go red, and is built to. It never rules
	# on findings -- it rules on whether each gate proved it MEASURED anything at all. A red mark there still
	# blocks nothing (this workflow holds no required contexts); it is simply the loudest honest way to say
	# a gate has stopped working. See that job's own comment for the three incidents that motivated it.
	#
	# Signals from the rubric (renumbered by tier -- docs/Code_Quality_Standards.md v0.7), each with how
	# it reaches a reviewer:
	# * signal 11 - complexity triage (ruff C901) -> BUILT (complexity job); PR-caused DELTA as
	# annotations + step summary, full list in the log
	# * signal 9 - clone / duplication (jscpd) -> BUILT (clone job); step summary only, see below
	# * signal 8 - diff-coverage visibility -> BUILT (coverage job; PR-only); INLINE `::notice`
	# annotations on the Files changed tab + summary
	# * signal 7 - mutation testing (highest leverage) -> BUILT and WORKING on mutmut 3 (2.5.1 crashed on
	# Python 3.14 before producing a mutant, and `\|\| true`
	# made that look green for months). Measured: 461
	# mutants in 3s -- 87 killed, 19 survived. Runs on
	# PRs too; survivor table in the step summary.
	# signal 10 (lint breadth) is a coordinated one-shot sweep, not an advisory job; DORA is a context caveat, not a gate.
	#
	# WHY THIS USES WORKFLOW-COMMAND ANNOTATIONS AND NOT CODE SCANNING / SARIF. This was measured, not
	# assumed, and the measurements are recorded here so a future session does not "helpfully" add a SARIF
	# upload back:
	# 1. All 122 C901 findings on this tree are SINGLE-LINE regions anchored on the function-name token of
	# the `def`. GitHub renders an alert inline only when its lines are in the diff, so a PR adding
	# branching INSIDE a function surfaces nothing, while a signature reflow fires on pre-existing debt.
	# 2. jscpd emits ONE location per clone pair, chosen by scan order, so ~8 files can never be the anchor
	# and roughly half of newly-introduced clones would anchor on the untouched twin.
	# 3. Uploading on pull_request only leaves NO default-branch baseline, so every PR forever would report
	# all ~161 findings as new -- not a one-off first-PR blip.
	# Workflow commands need no token and no permission grant, and behave identically on fork PRs. Every job
	# below therefore holds `contents: read` and nothing else -- least privilege, worth pinning on its own
	# merits given that two of these jobs execute third-party code fetched at run time.
	#
	# What actually keeps this advisory is TWO things, and permissions are neither of them: (a) these job
	# contexts are not in the required-checks set on `main`, and (b) every analysis step is
	# `continue-on-error: true` plus `--exit-zero` / `--fail-under=0` / `\|\| true`, so the job reports success
	# regardless of findings. Branch-protection membership is the only thing that determines gating.
	# tests/test_quality_advisory_invariants.py pins (b), the absent write scopes, and the absence of SARIF.
	#
	# HONEST LIMIT of the annotation mechanism: a workflow-command annotation renders INLINE on the Files
	# changed tab only when its line is in the diff. That always holds for diff-coverage (every line it
	# flags is a line the PR changed) but usually does NOT hold for the complexity delta, whose findings
	# anchor on a `def` line the PR often did not touch -- those land in the Checks tab and the step summary
	# instead. The summary table is the primary surface for complexity, not a nice-to-have.

	on:
	pull_request:
	workflow_dispatch:
	schedule:
	# Nightly sweep. Every job here is cheap (mutation is ~3s of actual mutating; complexity and clone
	# are seconds), so the cron is a safety net that catches drift on `main` rather than a cost dodge.
	# Coverage is PR-only -- it needs a base ref to diff against -- so it no-ops here.
	- cron: "23 4 * * *"

	# Deny-by-default at the workflow level; each job grants only what it needs.
	permissions: {}

	jobs:
	complexity:
	# Signal 11 - cyclomatic-complexity triage. Surfaces functions over the mccabe threshold (>10) so large
	# units are visible in review. Raw complexity is a WEAK defect predictor (rubric section 2), so this is
	# ADVISORY ONLY: --exit-zero reports but never fails, and this job is not a required status check.
	name: complexity triage (advisory)
	runs-on: ubuntu-latest
	permissions:
	contents: read
	outputs:
	receipt: ${{ steps.receipt.outputs.receipt }}
	steps:
	- name: Check out the source
	uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
	with:
	persist-credentials: false
	# The delta step needs the base branch's history for `git merge-base`.
	fetch-depth: 0
	- name: Install ruff (version derived from the lock)
	# DERIVED, not duplicated. This job used to hardcode 0.15.19 while all four locks pinned 0.15.22,
	# and that drift is load-bearing: c901_delta.py parses ruff's human-readable C901 message because
	# ruff exposes the complexity number nowhere else, so a skew can change the wording under the
	# parser. The obvious fix -- hardcode the right version and assert it matches the lock in a test --
	# would leave a REQUIRED check (the pytest suite) hostage to a routine Dependabot ruff bump:
	# the lock moves, the workflow string does not, and a blocking context goes red for a purely
	# advisory concern. Reading the lock at run time removes the drift AND the coupling.
	run: \|
	RUFF_PIN="$(sed -n 's/^ruff==$[^ ;]$./\1/p' constraints.lock \| head -1)"
	if [ -z "$RUFF_PIN" ]; then
	echo "::notice title=Ruff pin not found::falling back to the unpinned ruff from PyPI"
	pipx install ruff
	else
	echo "ruff pin from constraints.lock: $RUFF_PIN"
	pipx install "ruff==$RUFF_PIN"
	fi
	- name: Ruff C901 complexity triage (advisory - never gates)
	continue-on-error: true
	run: \|
	echo "== functions over mccabe complexity 10 (advisory triage; NEVER a merge gate) =="
	ruff check --select C901 --output-format=concise --exit-zero messagefoundry
	echo ""
	echo "== summary =="
	ruff check --select C901 --statistics --exit-zero messagefoundry
	- name: Complexity delta vs the merge base (advisory - never gates)
	# PR-only: there is no base ref to diff against on the cron or on workflow_dispatch, where the
	# whole-repo console triage above remains the full picture.
	if: github.event_name == 'pull_request'
	continue-on-error: true
	env:
	BASE_REF: ${{ github.base_ref }}
	run: \|
	echo "== complexity this PR CHANGED (advisory; pre-existing findings deliberately omitted) =="
	git fetch --no-tags origin "$BASE_REF" \|\| true
	# Steps run under `bash -e`, so an unresolvable merge base would abort this step with no
	# explanation at all. Say why instead: an advisory signal that goes quiet should at least
	# leave a reason in the log, since nobody watches a green advisory job for absence.
	if ! MERGE_BASE="$(git merge-base HEAD "origin/$BASE_REF" 2>/dev/null)"; then
	echo "::notice title=Complexity delta skipped::could not resolve a merge base against origin/$BASE_REF"
	# Record the bail so the liveness receipt can SEE it. Without this the step exits 0, the
	# receipt still reports "measured" off the whole-repo triage, and the PR-caused delta --
	# the entire point of this step on a PR -- goes quiet behind a green check. That is the
	# failure mode this workflow's liveness job exists to catch, sitting inside it.
	# QUOTE the reason: the receipt SOURCES this file, and an unquoted value containing
	# spaces parses as a command, silently leaving the variable unset -- which reported
	# "no reason recorded" and threw away the one useful diagnostic. Caught by executing
	# the receipt shell, not by reading it.
	echo "C901_DELTA=skipped" > c901-delta.env
	echo "C901_DELTA_REASON='could not resolve a merge base against origin/$BASE_REF'" >> c901-delta.env
	exit 0
	fi
	echo "merge base: $MERGE_BASE"
	ruff check --select C901 --output-format=json --exit-zero messagefoundry > c901-head.json
	# A second checkout of the merge base, so the "before" numbers come from real ruff output
	# rather than a reconstruction. Its absolute paths differ from HEAD's by the extra directory;
	# c901_delta.py collapses both to repo-relative keys (regression-tested -- if it did not,
	# every function would report as new).
	git worktree add --detach base-tree "$MERGE_BASE"
	( cd base-tree && ruff check --select C901 --output-format=json --exit-zero messagefoundry ) \
	> c901-base.json
	python3 scripts/quality/c901_delta.py \
	--base c901-base.json \
	--head c901-head.json \
	--repo-root . \
	--summary-file "$GITHUB_STEP_SUMMARY"
	# Written LAST, so it exists only if every step above actually completed. Its absence is
	# therefore proof the delta died somewhere, which the receipt reports as a dead gate.
	BASE_N="$(python3 -c 'import json,sys; print(len(json.load(open(sys.argv[1]))))' c901-base.json)"
	HEAD_N="$(python3 -c 'import json,sys; print(len(json.load(open(sys.argv[1]))))' c901-head.json)"
	{
	echo "C901_DELTA=ok"
	echo "C901_DELTA_MERGE_BASE=$MERGE_BASE"
	echo "C901_DELTA_BASE_N=$BASE_N"
	echo "C901_DELTA_HEAD_N=$HEAD_N"
	} > c901-delta.env
	- name: Record gate liveness
	id: receipt
	if: always()
	# Routed through env, not interpolated into the shell body (zizmor: template injection).
	env:
	IS_PR: ${{ github.event_name == 'pull_request' }}
	EVENT_NAME: ${{ github.event_name }}
	continue-on-error: true
	run: \|
	# Units count what was EXAMINED, never what was found: a repo with zero functions over the
	# threshold is good news, and a liveness check that fires on good news gets muted.
	# `--show-files` lists exactly the files ruff would check, so it is non-zero iff ruff ran.
	FILES="$(ruff check --select C901 --show-files messagefoundry 2>/dev/null \| wc -l)"
	FINDINGS="$(ruff check --select C901 --output-format=json --exit-zero messagefoundry \
	\| python3 -c 'import json,sys; print(len(json.load(sys.stdin)))')"
	# THE DELTA IS THE POINT ON A PR, and until now this receipt could not see it. `--show-files`
	# proves ruff enumerated files, which is true whether or not the delta step ran -- so a delta
	# that bailed on an unresolvable merge base left a green "measured" receipt and vanished.
	# That is this workflow's own failure mode reproduced inside its liveness control.
	#
	# On a non-PR event the delta legitimately does not run and the whole-repo triage IS the
	# measurement. On a PR its silence is a dead gate, not an absence of news.
	if [ "$IS_PR" != "true" ]; then
	python3 scripts/quality/liveness.py record \
	--signal complexity --status measured \
	--units "$FILES" --unit-name "files scanned" \
	--evidence "ruff C901 scanned $FILES files; $FINDINGS over the threshold (whole-repo triage; the PR delta does not apply on $EVENT_NAME)" \
	--extra "{\"findings\":$FINDINGS}"
	exit 0
	fi
	if [ ! -s c901-delta.env ]; then
	python3 scripts/quality/liveness.py record --signal complexity --status failed \
	--reason "the PR complexity delta produced no outcome marker; it did not complete (see the job log)"
	exit 0
	fi
	. ./c901-delta.env
	if [ "$C901_DELTA" != "ok" ]; then
	python3 scripts/quality/liveness.py record --signal complexity --status failed \
	--reason "the PR complexity delta did not run: ${C901_DELTA_REASON:-no reason recorded}"
	exit 0
	fi
	python3 scripts/quality/liveness.py record \
	--signal complexity --status measured \
	--units "$FILES" --unit-name "files scanned" \
	--evidence "ruff C901 scanned $FILES files ($FINDINGS over the threshold); delta compared ${C901_DELTA_BASE_N} base vs ${C901_DELTA_HEAD_N} head findings against ${C901_DELTA_MERGE_BASE}" \
	--extra "{\"findings\":$FINDINGS,\"delta_base_n\":${C901_DELTA_BASE_N},\"delta_head_n\":${C901_DELTA_HEAD_N}}"

	clone:
	# Signal 9 - duplication / clone detection. Flags copy-pasted blocks (the "copy-instead-of-abstract"
	# AI-slop signature, rubric section 3). ADVISORY: reports duplicates, never fails. The justified
	# ~21k-LOC store-backend parity (store/sqlserver.py + store/postgres.py = deliberate T-SQL vs Postgres
	# dialect duplication, whitelisted in the rubric) is ignored so it does not drown the real signal.
	name: clone detection (advisory)
	runs-on: ubuntu-latest
	permissions:
	contents: read
	outputs:
	receipt: ${{ steps.receipt.outputs.receipt }}
	steps:
	- name: Check out the source
	uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
	with:
	persist-credentials: false
	- name: jscpd clone report (advisory - never gates)
	continue-on-error: true
	run: \|
	echo "== copy-pasted blocks in messagefoundry/ (advisory; justified store-backend parity ignored) =="
	# Summary only, DELIBERATELY. jscpd ships a SARIF reporter and it works, but each result
	# carries exactly ONE location -- whichever file jscpd scanned first -- so the case this
	# signal exists for (a PR copy-pasting an existing block into a new place) anchors on the
	# untouched twin about half the time. That is a coin flip, not a diff-scoped signal.
	# Keep the 4.x pin: npm `latest` is a 5.x Rust rewrite with a different CLI.
	npx --yes jscpd@4.0.5 messagefoundry \
	--min-tokens 60 \
	--ignore "/store/sqlserver.py,/store/postgres.py" \
	--reporters console,markdown \
	--output ./jscpd-report \
	\|\| true
	if [ -f jscpd-report/jscpd-report.md ]; then
	{
	echo "## Duplicate blocks (jscpd)"
	echo ""
	# $GITHUB_STEP_SUMMARY is capped at 1 MiB per step and an oversized write is DROPPED
	# ENTIRELY, so truncate rather than silently lose the whole summary.
	head -c 900000 jscpd-report/jscpd-report.md
	echo ""
	echo "<sub>Advisory only. Clone locations are in the job log — see the note in the"
	echo "workflow for why these are not annotated on the diff.</sub>"
	} >> "$GITHUB_STEP_SUMMARY"
	fi
	- name: Record gate liveness
	id: receipt
	if: always()
	continue-on-error: true
	run: \|
	# Files ANALYSED, not clones found -- zero clones is a healthy repo, not a dead gate.
	REPORT=jscpd-report/jscpd-report.md
	FILES="$(grep -oE 'duplicated lines in [0-9]+' "$REPORT" 2>/dev/null \| grep -oE '[0-9]+$' \|\| true)"
	CLONES="$(grep -oE 'Found [0-9]+ exact clones' "$REPORT" 2>/dev/null \| grep -oE '[0-9]+' \|\| true)"
	python3 scripts/quality/liveness.py record \
	--signal clone --status measured \
	--units "${FILES:-0}" --unit-name "files analysed" \
	--evidence "jscpd analysed ${FILES:-0} files; found ${CLONES:-0} clones" \
	--extra "{\"clones\":${CLONES:-0}}"

	coverage:
	# Signal 8 - diff-coverage visibility. Runs the suite under coverage, then reports coverage of the
	# lines CHANGED by this PR (diff-cover) -- never a whole-repo % gate, which the rubric (section 4.1)
	# calls a gameable signal. ADVISORY: --fail-under=0 + `\|\| true` so it never fails a build, and this
	# job is not a required status check. PR-only (needs a base ref to diff against). It re-runs the
	# ubuntu suite (~a few min) per PR; add a paths filter if that cost bites. (This used to say
	# "move it to the mirror" -- there is no mirror since the cutover; development happens directly
	# on the public repo.)
	name: diff-coverage (advisory)
	if: github.event_name == 'pull_request'
	runs-on: ubuntu-latest
	timeout-minutes: 20
	permissions:
	contents: read
	outputs:
	receipt: ${{ steps.receipt.outputs.receipt }}
	steps:
	- name: Check out the source
	uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
	with:
	persist-credentials: false
	fetch-depth: 0
	- name: Set up Python 3.14
	uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
	with:
	python-version: "3.14"
	- name: Set up uv
	uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
	- name: Qt headless system deps
	# BOUNDED AND RETRIED. Unguarded, this step turns an external apt-mirror hiccup into a
	# blocked merge queue: measured 2026-08-18, three hangs across two attempts on three
	# different jobs, one of them 27+ minutes on `test (ubuntu-latest, py3.14)` -- a REQUIRED
	# context, so the queue stops. Two levers, because they answer different failures. The
	# per-command `timeout` kills a HANG and lets the retry run; `timeout-minutes` is the
	# backstop that stops this step ever eating a job budget again if the loop is edited wrong.
	# The failure text names the cause, so the next reader is not sent hunting in their diff --
	# a check whose label points at the wrong subject is what BACKLOG #1254 is about.
	timeout-minutes: 8
	run: \|
	for attempt in 1 2 3; do
	if sudo timeout 120 apt-get update && sudo timeout 180 apt-get install -y libegl1 libgl1 libxkbcommon0 libdbus-1-3; then
	exit 0
	fi
	echo "::warning::apt attempt ${attempt}/3 failed or timed out; retrying"
	sleep $((attempt * 5))
	done
	echo "::error::apt-get failed 3 times. This is the UBUNTU RUNNER MIRROR, not the change under test."
	exit 1
	- name: Install project + coverage tools
	run: \|
	uv pip install --system --constraint constraints.lock -e ".[dev,harness,fhir,dicom,x12,xml,webauthn]" -e packaging/messagefoundry-webconsole
	# HASH-PINNED from the CI toolchain lock (ADR 0034 §3). These tools now DO live in
	# pyproject.toml ([dependency-groups].ci-quality, where each pin's rationale sits) and DEP-1
	# re-exports this lock by design -- superseding the note that used to sit here claiming they
	# were deliberately kept out of pyproject to avoid tripping that gate.
	#
	# `python -m pip`, not `uv pip install --system`, for ONE honest reason: every other hashed
	# install in this repo is a pip install, and keeping one spelling means one thing for the
	# guards in tests/test_ci_venv_pinning.py to match. Same setup-python interpreter either way.
	#
	# NOT because uv cannot: `uv pip install --require-hashes` exists (env UV_REQUIRE_HASHES), and
	# setup-uv is already in this job -- the earlier claim that "--require-hashes is pip's
	# contract" was simply wrong. NOR for Scorecard's benefit: its parser keys on
	# pip/pip3/`python -m pip`, so a `uv pip` line generates NO finding at all, and converting one
	# can only ADD parseable surface -- it cannot close an alert. This line is posture-neutral to
	# Scorecard and is here for uniformity.
	#
	# Safe against the editable install above: the group lock and constraints.lock agree on all 10
	# shared packages (pytest and pytest-timeout among them), so this cannot re-point what the
	# coverage run executes under.
	python -m pip install --require-hashes -r ci/locks/ci-quality.lock
	- name: Tests under coverage (advisory - never fails)
	continue-on-error: true
	env:
	QT_QPA_PLATFORM: offscreen
	run: pytest -q --cov=messagefoundry --cov-report=xml --cov-report=term-missing:skip-covered \|\| true
	- name: Diff-coverage vs the PR base (advisory - never fails)
	continue-on-error: true
	env:
	BASE_REF: ${{ github.base_ref }}
	run: \|
	echo "== coverage of the lines this PR changed (advisory; not a whole-repo % gate) =="
	# NO --depth HERE. Running a shallow fetch against the COMPLETE clone that `fetch-depth: 0`
	# just produced writes .git/shallow and grafts away everything behind origin/<base>'s tip.
	# diff-cover diffs with the three-dot range `origin/<base>...HEAD`, which needs a merge base --
	# so once the base branch advances past the PR's merge ref (another PR merging, or a re-run of
	# a stale check) the diff dies with "no merge base". Reproduced. It then fails in the worst
	# possible way: the markdown reporter runs BEFORE the annotations reporter and truncates
	# diff-cover.md on open, so `\|\| true` swallows the error, zero annotations are emitted, and the
	# summary shows a bare heading that reads as "nothing uncovered".
	git fetch --no-tags origin "$BASE_REF" \|\| true
	if [ ! -s coverage.xml ]; then
	echo "::notice title=Diff coverage skipped::coverage.xml was not produced (the test step did not complete)"
	exit 0
	fi
	# `github-annotations:notice` writes `::notice file=...,line=N,endLine=M` workflow commands to
	# STDOUT, which the runner turns into INLINE annotations on the Files changed tab -- no token,
	# no permission grant, identical behaviour on fork PRs. Adjacent uncovered lines are coalesced
	# into ranges, so a long uncovered block costs one annotation rather than one per line. Every
	# line flagged is a line this PR changed, so there is no pre-existing-debt noise by construction.
	# The console report still prints, so the job log is unchanged.
	diff-cover coverage.xml --compare-branch="origin/$BASE_REF" --fail-under=0 \
	--format "github-annotations:notice,markdown:diff-cover.md" 2>&1 \| tee diff-cover-console.txt \|\| true
	# `-s` not `-f`: diff-cover creates and truncates this file BEFORE it can fail, so a crashed
	# run leaves a 0-byte report. Testing for existence alone would append an empty section that
	# looks like a clean result -- the exact silent-success shape this workflow keeps producing.
	if [ -s diff-cover.md ]; then
	{
	echo "## Diff coverage"
	echo ""
	# The markdown report embeds a source snippet per uncovered line, so it can get large.
	# $GITHUB_STEP_SUMMARY is capped at 1 MiB per step and an oversized write is DROPPED
	# ENTIRELY -- truncate rather than silently lose the whole summary.
	head -c 900000 diff-cover.md
	} >> "$GITHUB_STEP_SUMMARY"
	else
	echo "::notice title=Diff coverage unavailable::diff-cover produced no report (see the job log)"
	fi
	- name: Record gate liveness
	id: receipt
	if: always()
	# BASE_REF is needed here too -- without it the evidence string read "against origin/".
	env:
	BASE_REF: ${{ github.base_ref }}
	continue-on-error: true
	run: \|
	# THE DISTINCTION THIS WHOLE CHECK RESTS ON, and the place it is easiest to get wrong.
	#
	# "No lines with coverage information in this diff" is a real, correct outcome -- it is what
	# PRs #18 and #19 produced, because neither touched measured source. But diff-cover prints
	# that SAME line when coverage.xml itself measured nothing (its console template branches on
	# whether any source has measured lines, and never consults the diff size). So the message
	# alone cannot tell "nothing to measure" from "the measurement is dead" -- exactly the
	# ambiguity this control exists to resolve, one layer up. Reproduced with a valid-but-empty
	# coverage.xml against a diff that really did change six lines.
	#
	# So: prove coverage.xml measured something before accepting the not-applicable branch, and
	# treat a MISSING coverage.xml as a dead gate (failed), never as inapplicable -- the same way
	# the mutation job treats a missing mutmut-counts.env.
	if [ ! -s coverage.xml ]; then
	python3 scripts/quality/liveness.py record --signal coverage --status failed \
	--reason "coverage.xml was not produced; the measurement never happened"
	exit 0
	fi
	MEASURED_FILES="$(grep -c '<class ' coverage.xml \|\| true)"
	if [ "${MEASURED_FILES:-0}" -eq 0 ] 2>/dev/null; then
	python3 scripts/quality/liveness.py record --signal coverage --status failed \
	--reason "coverage.xml contains no measured files; diff-cover had nothing to compare against"
	exit 0
	fi
	if grep -q "No lines with coverage information" diff-cover-console.txt 2>/dev/null; then
	python3 scripts/quality/liveness.py record --signal coverage --status not-applicable \
	--reason "no lines with coverage information in this diff; coverage.xml measured ${MEASURED_FILES} files, so the tooling is live and this PR simply changed no measured source"
	else
	LINES="$(grep -oE 'Total:[[:space:]]+[0-9]+' diff-cover-console.txt 2>/dev/null \
	\| grep -oE '[0-9]+' \| head -1 \|\| true)"
	python3 scripts/quality/liveness.py record --signal coverage --status measured \
	--units "${LINES:-0}" --unit-name "changed lines analysed" \
	--evidence "diff-cover reported on ${LINES:-0} changed lines against origin/${BASE_REF}; coverage.xml measured ${MEASURED_FILES} files"
	fi

	mutation:
	# Signal 7 - mutation testing (advisory; the HIGHEST-leverage gate -- it adversarially checks the
	# tests actually assert something, catching the "tests that never fail" AI-slop mode, rubric section 3).
	#
	# NOT expensive, contrary to the note that used to sit here. MEASURED on Python 3.14 (linux, the
	# bounded scope below): 461 mutants in 3 SECONDS -- 87 killed, 19 survived, 355 not covered by the
	# scoped test file. mutmut 3 only runs the tests that actually cover each mutant, so the old
	# "runs the suite once per mutant" cost model does not apply. That is why this now runs on PRs too:
	# a real signal for seconds of runtime is worth having in review, which is where survivors get fixed.
	name: mutation (advisory)
	runs-on: ubuntu-latest
	timeout-minutes: 30
	permissions:
	contents: read
	outputs:
	receipt: ${{ steps.receipt.outputs.receipt }}
	steps:
	- name: Check out the source
	uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
	with:
	persist-credentials: false
	- name: Set up Python 3.14
	uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
	with:
	python-version: "3.14"
	- name: Set up uv
	uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
	- name: Qt headless system deps
	# BOUNDED AND RETRIED. Unguarded, this step turns an external apt-mirror hiccup into a
	# blocked merge queue: measured 2026-08-18, three hangs across two attempts on three
	# different jobs, one of them 27+ minutes on `test (ubuntu-latest, py3.14)` -- a REQUIRED
	# context, so the queue stops. Two levers, because they answer different failures. The
	# per-command `timeout` kills a HANG and lets the retry run; `timeout-minutes` is the
	# backstop that stops this step ever eating a job budget again if the loop is edited wrong.
	# The failure text names the cause, so the next reader is not sent hunting in their diff --
	# a check whose label points at the wrong subject is what BACKLOG #1254 is about.
	timeout-minutes: 8
	run: \|
	for attempt in 1 2 3; do
	if sudo timeout 120 apt-get update && sudo timeout 180 apt-get install -y libegl1 libgl1 libxkbcommon0 libdbus-1-3; then
	exit 0
	fi
	echo "::warning::apt attempt ${attempt}/3 failed or timed out; retrying"
	sleep $((attempt * 5))
	done
	echo "::error::apt-get failed 3 times. This is the UBUNTU RUNNER MIRROR, not the change under test."
	exit 1
	- name: Install project + mutmut
	run: \|
	uv pip install --system --constraint constraints.lock -e ".[dev,harness,fhir,dicom,x12,xml,webauthn]" -e packaging/messagefoundry-webconsole
	# mutmut 3 + pytest-timeout, HASH-PINNED from the CI toolchain lock (ADR 0034 §3). Both live in
	# pyproject.toml's [dependency-groups].ci-quality, which records why each is load-bearing:
	# `mutmut<3` resolved to 2.5.1, which CRASHES on Python 3.14 in its pony-ORM cache
	# (`cannot pickle 'itertools.count'`) before generating a single mutant -- verified from run
	# 30248096425; and pytest-timeout is REQUIRED because mutmut 3 always passes `--timeout`/
	# `--timeout-method` to pytest, so without the plugin every invocation dies with an
	# unrecognised-argument error surfacing only as BadTestExecutionCommandsException.
	# `python -m pip` for UNIFORMITY with every other hashed install in this repo, not because uv
	# cannot do it (`uv pip install --require-hashes` exists) and not for Scorecard (a `uv pip`
	# line generates no finding at all) -- see the coverage job's install step for the full note.
	python -m pip install --require-hashes -r ci/locks/ci-quality.lock
	- name: Mutation-test a bounded scope (advisory - never fails)
	continue-on-error: true
	env:
	QT_QPA_PLATFORM: offscreen
	run: \|
	echo "== mutation testing messagefoundry/parsing/binary.py (advisory; bounded scope) =="
	# Ephemeral mutmut config (the repo ships none). THREE things here are load-bearing, each
	# found by a run that silently produced nothing:
	# * source_paths is the PACKAGE, not the one file. mutmut 3 copies source_paths into
	# `mutants/` and runs pytest THERE; with a single file copied, tests/conftest.py died on
	# `ModuleNotFoundError: messagefoundry.config` and every mutant came back "not checked".
	# * only_mutate keeps the bounded scope -- the whole package is copied, one module mutated.
	# * pytest_add_cli_args_test_selection replaces mutmut 2's `runner=`; the old `runner` and
	# `paths_to_mutate` keys are gone/deprecated in 3.x.
	printf '[mutmut]\nsource_paths=messagefoundry\nonly_mutate=messagefoundry/parsing/binary.py\npytest_add_cli_args_test_selection=tests/test_binary_carriage.py\n' > setup.cfg
	if mutmut run > mutmut-run.txt 2>&1; then
	MUTMUT_OK=1
	else
	MUTMUT_OK=0
	fi
	tail -c 20000 mutmut-run.txt
	if [ "$MUTMUT_OK" = "1" ]; then
	mutmut results > mutmut-results.txt 2>&1 \|\| true
	# `mutmut results` lists ONLY the mutants worth looking at -- survived / no tests /
	# timeout / suspicious. Killed mutants are NEVER listed, so `grep -c ': killed'` is
	# always 0 and a naive table reports "Killed 0" on a perfectly healthy run. Derive it:
	# the run's final progress line carries the total (`461/461`), and every non-killed
	# mutant is exactly one line in the results file. Verified against run 30308667584:
	# 461 total - 374 listed = 87 killed, matching mutmut's own counter.
	TOTAL="$(tr '\r' '\n' < mutmut-run.txt \| grep -oE '[0-9]+/[0-9]+' \| tail -1 \| cut -d/ -f2)"
	LISTED="$(grep -cE '^[[:space:]]+\S+: ' mutmut-results.txt \|\| true)"
	SURVIVED="$(grep -c ': survived' mutmut-results.txt \|\| true)"
	NOTESTS="$(grep -c ': no tests' mutmut-results.txt \|\| true)"
	# Counted INDEPENDENTLY, never as `LISTED - SURVIVED - NOTESTS`. Deriving it as a
	# remainder would make the liveness reconciliation tautological -- the parts would add up
	# by construction and the check could never fire.
	#
	# The alternation must cover EVERY non-killed status mutmut can print, or the sum breaks
	# on a healthy run and the liveness job reddens for no reason. A check whose only
	# reachable failure is a false positive gets muted within a month, which would leave us
	# worse off than having no check.
	OTHER="$(grep -cE ': (timeout\|suspicious\|skipped\|segfault\|not checked\|caught by type check\|check was interrupted by user)$' mutmut-results.txt \|\| true)"
	# mutmut's OWN killed counter, off its final progress line. KILLED above is derived a
	# completely different way (total minus listed); the liveness check reconciles the two.
	# Two independent derivations agreeing is what actually protects the number -- the sum
	# check alone cannot, because TOTAL cancels out of it algebraically.
	KILLED_REPORTED="$(tr '\r' '\n' < mutmut-run.txt \| grep -oE '🎉 [0-9]+' \| tail -1 \
	\| grep -oE '[0-9]+' \|\| true)"
	if [ -n "$TOTAL" ] && [ "$TOTAL" -ge "$LISTED" ] 2>/dev/null; then
	KILLED=$((TOTAL - LISTED))
	else
	# Never invent a number: if the total could not be read, say so rather than print 0.
	KILLED="?"
	fi
	echo "== total=$TOTAL killed=$KILLED survived=$SURVIVED no-tests=$NOTESTS =="
	grep ': survived' mutmut-results.txt \|\| true
	{
	echo "## Mutation testing — \`messagefoundry/parsing/binary.py\`"
	echo ""
	echo "\| Killed \| Survived \| Not covered \|"
	echo "\| --- \| --- \| --- \|"
	echo "\| $KILLED \| $SURVIVED \| $NOTESTS \|"
	echo ""
	if [ "$SURVIVED" != "0" ]; then
	echo "Survivors — injected bugs the tests did not catch:"
	echo ""
	echo '```'
	grep ': survived' mutmut-results.txt \| head -c 100000
	echo '```'
	fi
	echo ""
	echo "<sub>Advisory only. A survivor is a hint to strengthen an assertion, not a defect —"
	echo "and mutation score is a poor single number (rubric §2), so this never gates.</sub>"
	} >> "$GITHUB_STEP_SUMMARY"
	# Stash the breakdown for the liveness receipt in the next step.
	{
	echo "MUT_TOTAL=$TOTAL"
	echo "MUT_KILLED=$KILLED"
	echo "MUT_SURVIVED=$SURVIVED"
	echo "MUT_NOTESTS=$NOTESTS"
	echo "MUT_OTHER=$OTHER"
	echo "MUT_LISTED=$LISTED"
	echo "MUT_KILLED_REPORTED=${KILLED_REPORTED:-}"
	} > mutmut-counts.env
	else
	echo "::warning title=Mutation run failed::mutmut exited non-zero — see the job log"
	{
	echo "## Mutation testing — FAILED TO RUN"
	echo ""
	echo "\`mutmut run\` exited non-zero, so this signal produced nothing. Tail of the run:"
	echo ""
	echo '```'
	tail -c 4000 mutmut-run.txt
	echo '```'
	} >> "$GITHUB_STEP_SUMMARY"
	fi
	- name: Record gate liveness
	id: receipt
	if: always()
	continue-on-error: true
	run: \|
	if [ ! -s mutmut-counts.env ]; then
	# mutmut crashed. This is the 2.5.1 case verbatim -- and it must NOT be silent.
	python3 scripts/quality/liveness.py record --signal mutation --status failed \
	--reason "mutmut produced no counts; the run did not complete (see the job log)"
	exit 0
	fi
	. ./mutmut-counts.env
	# `?` means the mutant total was unreadable, so KILLED is underivable. Interpolating it into
	# the --extra JSON would emit `"killed":?` -- malformed, and it would surface as "receipt is
	# not valid JSON" rather than the real diagnosis.
	case "$MUT_KILLED" in
	''\|[!0-9])
	python3 scripts/quality/liveness.py record --signal mutation --status failed \
	--reason "could not read mutmut's mutant total from the progress line; the killed count is underivable"
	exit 0
	;;
	esac
	# An EMPTY results file yields LISTED=0, hence killed=TOTAL, survived=0 -- a flawless score
	# that reconciles perfectly. A 100% kill rate with nothing listed is not a triumph, it is a
	# tool that produced no output. Refuse to report it as success.
	if [ "${MUT_LISTED:-0}" -eq 0 ] && [ "${MUT_TOTAL:-0}" -gt 0 ] 2>/dev/null; then
	python3 scripts/quality/liveness.py record --signal mutation --status failed \
	--reason "mutmut listed no mutants at all while reporting $MUT_TOTAL processed; a perfect score here means the results file was empty, not that every mutant died"
	exit 0
	fi
	EXTRA="{\"killed\":$MUT_KILLED,\"survived\":$MUT_SURVIVED,\"no_tests\":$MUT_NOTESTS,\"other\":$MUT_OTHER"
	case "$MUT_KILLED_REPORTED" in
	''\|[!0-9]) ;;
	*) EXTRA="$EXTRA,\"killed_reported\":$MUT_KILLED_REPORTED" ;;
	esac
	EXTRA="$EXTRA}"
	python3 scripts/quality/liveness.py record \
	--signal mutation --status measured \
	--units "$MUT_TOTAL" --unit-name "mutants processed" \
	--evidence "mutmut processed $MUT_TOTAL mutants; $MUT_LISTED listed as non-killed" \
	--extra "$EXTRA"
	- name: Upload the mutmut results (advisory)
	if: always()
	continue-on-error: true
	uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
	with:
	name: mutmut-results
	path: \|
	mutmut-results.txt
	mutmut-run.txt
	.mutmut-cache
	# .mutmut-cache is a DOTFILE and upload-artifact skips hidden files by DEFAULT. Without this
	# the step logs "No files were found", uploads nothing, and still reports success -- a silent
	# no-op that looks green.
	include-hidden-files: true
	# Deliberately `warn`, not `ignore`: if mutmut ever renames its cache, that must be visible in
	# the log rather than silently swallowed.
	if-no-files-found: warn

	liveness:
	# THE META-GATE. Every other job here is built so it can never fail; this one is built so it CAN.
	#
	# Three separate signals in this workflow spent months reporting success while measuring nothing:
	# diff-coverage (a shallow fetch destroyed its merge base, and the empty report looked clean),
	# mutation (mutmut 2.5.1 crashed before producing a mutant, and `\|\| true` made that green in 37s),
	# and the killed count (a grep for a line mutmut never prints, so a healthy run said "Killed 0").
	# The rubric's anti-metric rule guards against trusting a NUMBER too much. Nothing guarded against
	# trusting a GREEN CHECK THAT NEVER RAN. This job is that control.
	#
	# It reads each job's liveness receipt and demands proof of EXECUTION -- a count of things
	# examined -- or an explicit, reasoned declaration that there was nothing to measure. It does not
	# demand findings: a clean repo legitimately has zero clones, and a gate that fires on good news
	# gets muted, which would leave us worse off than before.
	#
	# DELIBERATELY has no `continue-on-error` and no `\|\| true`. A dead gate should be loud, and a red
	# mark here blocks nothing -- this job is not, and must never become, a required status check.
	name: gate liveness (advisory)
	if: always()
	needs: [complexity, clone, coverage, mutation]
	runs-on: ubuntu-latest
	permissions:
	contents: read
	steps:
	- name: Check out the source
	uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
	with:
	persist-credentials: false
	- name: Verify every gate proved it measured something
	# `needs` carries each job's result AND its receipt output. Routed through `env:` rather than
	# interpolated into the shell body -- the receipts are job-authored strings, and expanding
	# them into a command line is the template-injection shape zizmor exists to catch.
	env:
	NEEDS_JSON: ${{ toJSON(needs) }}
	run: python3 scripts/quality/liveness.py verify --needs-json "$NEEDS_JSON"

Navigation Menu

Uh oh!

backlog: file items 1307/1308, land 1309, and retire 1311 as a pointer (rebuilt off 298a2bf9) #1353

Workflow file

backlog: file items 1307/1308, land 1309, and retire 1311 as a pointer (rebuilt off 298a2bf9) #1353

Uh oh!

Workflow file for this run