test: nothing checked the bytes site/ actually publishes

[ChelseaKR]

The defect

site/ is a committed artifact standing in for a computation. It is built from CAL FIRE's acquired files, which are not in git and never in CI, so no gate regenerated it and compared. tests/test_published_site.py says so itself: "it does not prove site/ is what the current pipeline would produce from CAL FIRE's files."

That is true of the numbers. It is not true of the rest of the page, and the rest of the page is most of it. Change the palette, the skip link, the nav, the canonical, the footer prose, the three-state key, a quoted CAL FIRE caveat or a line of a provenance table, and the pages pages.yml uploads keep saying the old thing while every check stays green. The measurement pages would be the last place anyone looked, because they look right.

Had it drifted

No. Measured 2026-08-29, before writing anything: make site against the acquired FRAP and DINS files in data/raw/, into a scratch directory, reproduced the committed site/ byte for byte (diff -r reported nothing). Nothing was keeping it that way.

The gate

tests/test_published_site_is_current.py, which make verify runs because verify runs pytest.

• Chrome, byte for byte. Everything before <main id="content"> and everything from </main>, compared against a build from the committed fixtures into tmp_path. That covers the doctype, head, title, description, canonical and Open Graph block, the whole stylesheet with both palettes inline, the skip link, the nav, and the footer with the disclaimer and the licence note. A fixture build and a real build differ inside <main> and nowhere else, which is why the split is where it is: 8,096 bytes of head and 511 of foot per page, identical today.
• Blocks written from committed constants. render.legend(), render.caveat_block(source) and render.provenance_block(source, is_fixture=False), each required verbatim in the pages that publish it. The provenance block cannot come from a fixture build, which publishes nulls where the acquisition facts go, so it is called directly. test_published_site.py already holds this for the JSON; the pages are what a reader sees.
• The shape perimeter.artifacts imposes on both published JSON payloads: top-level keys, the reported fields in order with each field's key set, and the fixed prose and thresholds.

The rebuild is written into tmp_path and never into site/.

What is excluded, and why

Every count. Those measure CAL FIRE's files; the fixtures hold ten records each; no offline check can settle them. make site on a machine holding data/raw/, with no diff, remains the only thing that can, and this is said out loud in the module docstring rather than implied.

The JSON comparison is a shape comparison rather than a whole-document one for a related reason: several of these payloads' dictionary keys are measurements too, because a marker table is keyed by the marker strings the file actually holds. A fixture cannot agree on those and must not be made to.

ADR 0004 evidence

Form 1, a test that runs the gate against failing input. TestTheGateCanFail runs chrome_problems, block_problems and spine_problems against a changed stylesheet, a changed canonical, a changed footer, a page with no content marker, a page with two, a removed block, a block altered by one character, a dropped payload key and a renamed field. Both directions are covered, so neither check is passing by always finding a problem.

Form 2, measured against the committed bytes on disk, not in-memory copies:

site/index.html:  "  --accent: "        ->  "  --accent-colour: "
site/dins.html:   "<dt>SHA-256</dt>"    ->  "<dt>SHA256</dt>"

grep -c -- "--accent-colour" site/index.html -> 1; grep -c "dt>SHA256<" site/dins.html -> 1; git diff --stat site/ showed both files changed. Red run, pytest exit 1:

site/index.html: the published head is not what the renderer now writes.
8103 bytes committed, 8096 bytes rebuilt.
FAILED ...::test_the_published_page_carries_the_current_provenance_table[dins.html]

Restored from backup, git diff --stat site/ empty, green run: 22 passed, exit 0.

make verify on the final tree: exit 0, 670 passed, coverage 100.00% against the 90% floor, determinism gate byte-identical over 5 files.

🤖 Generated with Claude Code