A retrieval that goes stale fails the build - #31
Merged
Conversation
ChelseaKR
force-pushed
the
feat/data-cards-and-a-staleness-gate
branch
from
August 29, 2026 00:31
3f5ad08 to
35cae44
Compare
Every figure these pages publish is a count of a file downloaded on one day. The pages say the date, PROVENANCE.md says the version and the hash, and none of that is the same thing as noticing. DINS carries no version string and is appended to as inspections complete, so it differs from the acquired file the day after the acquisition and increasingly every season after. Nothing here raised its hand, and nothing could: no code in this repository had ever asked what today is. DG-01 asks for a data card per ingest source with seven named rows. PROVENANCE.md carried five of them. The two it had never carried are the ones the README's Data Governance row recorded as missing: the tier, L1 for both, and the refresh cadence with its staleness SLA. docs/data/frap-perimeters.md and docs/data/dins.md are those cards, and tests/test_data_cards.py enumerates them against the declared source list rather than against what is on disk, so adding a source without a card fails rather than passing. DG-04 makes the staleness alarm an AUTO-GATE, and this is it. 400 days for the FRAP layer, which is one annual release cycle plus about five weeks so a retrieval does not go stale on the eve of the version that would replace it. 180 days for DINS, which is about one fire season. On the retrieval on record the gate fires 2027-02-03 for DINS and 2027-09-11 for FRAP. Firing is it working, and the failure message says what to do: re-acquire and rebuild site/, or move the SLA in the card with a reason. What it must not become is a number quietly raised to get a green build, which is the move CONTRIBUTING.md already forbids for a marker set. The clock is in the gate and not in the artifacts, on purpose. A page computing its own staleness would read the wall clock at build time and two builds a day apart would differ, which is in direct conflict with the byte-identical output make determinism exists to protect, and determinism is the claim that would quietly lose. So the artifacts publish retrieved and staleness_sla_days and compute nothing from them; a consumer holding both does the subtraction against its own clock and gets the same answer this build would. One function reads date.today(), one caller passes it, and both sides of the SLA boundary are tested against fixed dates. The SLA is this project's declaration and every card says so. Neither CAL FIRE nor FRAP promises a publication schedule and neither is asked to. Measured on this tree. Against the pre-change source and with docs/data/ absent, all 25 tests in tests/test_data_cards.py fail, on AttributeError for the fields sources.py did not carry and on the cards not existing. tests/test_provenance.py passes identically in both states, 14 tests. After, make verify exits 0 with 606 tests and 100% branch coverage. site/ is rebuilt from the acquired files: four lines added to each artifact, the policy fields, and no measured number moves. See ADR-0009. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nine open PRs and four open issues had accumulated past what one person can work through in a sitting, so each PR is now written up to be decidable on its own: what it does, what was verified rather than taken from the body, the defects found, and a single recommendation. The findings worth surfacing: no PR in the queue is failing CI; codeql never ran on the four PRs that target another branch rather than main, so their CodeQL result is absent rather than green; the two three-PR stacks collide in CHANGELOG.md, site/dins.html and render.py whichever order they land in; merging a stack base with --delete-branch auto-closes its dependent, which has already happened once here; and nothing in the queue should be closed. This is the review, not its execution. Nothing was merged, closed, retargeted or rebased to produce it, and no recommendation in it has been acted on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… SLA day
ADR-0009 said the staleness gate "fires for DINS on 2027-02-03 and for FRAP on
2027-09-11". Those are the last days it passes. The comparison in
tests/test_data_cards.py is `age > source.staleness_sla_days`, so on the SLA day
itself age equals the SLA and the source is not stale; the first failing day is
the one after. Retrieval on record is 2026-08-07, so DINS first fires 2027-02-04
and FRAP 2027-09-12.
The code was never wrong and is not touched here. Its own boundary test,
test_the_staleness_gate_fires_the_day_after_the_sla_and_not_the_day_of, asserts
both sides of exactly this line, and docs/PR-TRIAGE.md on the same branch reads
it correctly ("from 2027-02-04 every CI run goes red"). The ADR was the one
document in the set that stated it a day early, and an ADR is published output.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ChelseaKR
force-pushed
the
feat/data-cards-and-a-staleness-gate
branch
from
August 29, 2026 17:05
35cae44 to
ae0e2d6
Compare
ChelseaKR
added a commit
that referenced
this pull request
Aug 29, 2026
This file said "Two of the five contexts in `main.json` do not exist on `main` yet" and made landing them step 1 of the apply order. It said the same thing a second time in the `bypass_actors` bullet. Both are now false. The sentence was written on 2026-08-15 in 18827df, at 17:14. `zizmor`, one of the two contexts it meant, landed on `main` in #16 (839557e) the same day, and `sast` had been a job in ci.yml since ea06580 on 2026-08-07. So it was true when written on its own branch and false within the day, and nothing here noticed for two weeks, which is the shape #30 fixed in the Standards Conformance table and ADR-0009 turned out to have in its fire dates. Re-measured 2026-08-29 rather than reasoned about. `verify`, `secret-scan`, `sast` and `zizmor` are jobs in .github/workflows/ci.yml on `main`; `codeql (actions · python · javascript)` is the `name:` of the `analyze` job in codeql.yml, which triggers on `pull_request: branches: [main]`. Read off the two most recent pull request head commits, 62c406e (#30) and ae0e2d6 (#31), each reported all five contexts. The other dated claims in the file were re-read at the same time and are unchanged: no ruleset is applied, `"protected": false`, and the branch protection endpoint still 404s. The dependabot queue that step 2 said to drain is drained, measured as 0 open. The two completed steps are kept as history rather than deleted, so the order stays legible, and the remaining steps are renumbered. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ChelseaKR
added a commit
that referenced
this pull request
Aug 29, 2026
#32) * fix: the committed ruleset would lock the owner out on its first apply No ruleset is applied on this repository. `gh api repos/ChelseaKR/perimeter/rulesets` returned `[]` on 2026-08-15 and again on 2026-08-28, so nothing is broken yet; what was committed was a first application that would have broken it. `.github/rulesets/main.json` carried `"bypass_actors": []`, and the ruleset README argued for it by name, calling the empty list "the stricter reading" of CICD-15. It is not the stricter reading. `bypass_actors` now holds exactly the repository owner's standing bypass, `RepositoryRole` 5 with `bypass_mode: always`, deliberately and permanently: an agent once applied a ruleset with no bypass and locked the owner out of their own repository, and restoring access took a sweep across eighteen repositories. An empty list there is not a stricter gate, it is the lockout. That matters more here than anywhere else in the portfolio, because this profile has never met a live ruleset. It asks for `required_signatures`, `required_linear_history`, a strict up-to-date policy and five required contexts, two of which do not exist on `main` yet. That is a lot of ways for a first application to wedge, and the empty list would have removed the only way back in that does not go through GitHub support. `bypass_mode: always` rather than CICD-15's `pull_request` for the same reason: a bypass that only works inside a pull request is no use when the pull request is the thing that is wedged. The README bullet is replaced with the reversal and the reasoning rather than quietly rewritten, the apply procedure gains a step that checks the bypass actually came through, and "what is true today" now states plainly that no ruleset is applied and that this file has therefore never been corrected by a live one. Nothing here touches a live setting; there is none to touch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: check the bypass before posting, not only after The ordered list is what a reader follows, so the check belongs in it as a step rather than only in the confirmation after the fence. Records that POST adds a ruleset rather than replacing one: rules from every applicable ruleset combine and bypass actors are per-ruleset, so posting twice without deleting leaves an empty-bypass ruleset over `main` that blocks the owner whatever the first one allows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(rulesets): all five required contexts exist, measured 2026-08-29 This file said "Two of the five contexts in `main.json` do not exist on `main` yet" and made landing them step 1 of the apply order. It said the same thing a second time in the `bypass_actors` bullet. Both are now false. The sentence was written on 2026-08-15 in 18827df, at 17:14. `zizmor`, one of the two contexts it meant, landed on `main` in #16 (839557e) the same day, and `sast` had been a job in ci.yml since ea06580 on 2026-08-07. So it was true when written on its own branch and false within the day, and nothing here noticed for two weeks, which is the shape #30 fixed in the Standards Conformance table and ADR-0009 turned out to have in its fire dates. Re-measured 2026-08-29 rather than reasoned about. `verify`, `secret-scan`, `sast` and `zizmor` are jobs in .github/workflows/ci.yml on `main`; `codeql (actions · python · javascript)` is the `name:` of the `analyze` job in codeql.yml, which triggers on `pull_request: branches: [main]`. Read off the two most recent pull request head commits, 62c406e (#30) and ae0e2d6 (#31), each reported all five contexts. The other dated claims in the file were re-read at the same time and are unchanged: no ruleset is applied, `"protected": false`, and the branch protection endpoint still 404s. The dependabot queue that step 2 said to drain is drained, measured as 0 open. The two completed steps are kept as history rather than deleted, so the order stays legible, and the remaining steps are renumbered. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: an empty bypass list is a test failure, not a comment The previous commit in this branch corrected `"bypass_actors": []` in .github/rulesets/main.json. Correcting the file is not the fix, because nothing in this repository reads that file. Measured before writing this: no test and no source module referenced `ruleset` or `bypass_actors` anywhere, so the value could regress to the empty list in one edit and every gate would stay green. That is what made the committed file dangerous rather than merely wrong. It is never applied, so no live ruleset has ever contradicted it, and nothing read it, so nothing here could contradict it either. The apply command in the README posts the file as it stands, which means following this repository's own documented procedure was enough to lock the owner out of it. tests/test_ruleset.py makes the empty list red. `lockout_risk` is a pure function of a parsed document, run against the five shapes that lose the bypass (empty list, absent key, wrong type, a different actor, and the owner with `bypass_mode: pull_request`) as well as against the committed file, with a positive control so it is not passing by refusing everything. `load_ruleset` fails on a missing or unparseable file rather than returning an empty document. A check that passes when its subject is absent is the defect class this repository keeps finding, and it would be a poor joke to introduce one here. This does not apply the ruleset and does not ask anyone to. Applying it changes a live repository setting and remains the owner's call. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(changelog): the guard, the re-measurement, and what is still not applied Records the three things the ruleset entry did not yet cover: that nothing read main.json before this branch so correcting it once would not have held, that the apply order's claim about missing required contexts was re-measured and is false, and that landing this changes nothing on the server. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the DG-01 and DG-04 gaps the Data Governance row recorded. Adds ADR-0009. Stacked on #30; base is
docs/the-manifest-entry-exists.What was missing
Every figure these pages publish is a count of a file downloaded on one day. The pages say the date,
PROVENANCE.mdsays the version and the hash, and none of that is the same thing as noticing. DINS carries no version string and is appended to as inspections complete, so it differs from the acquired file the day after the acquisition and increasingly every season after.Nothing here raised its hand, and nothing could: no code in this repository had ever asked what today is.
The README's Data Governance row had recorded the gap: "no refresh cadence, staleness SLA or stated tier".
What this adds
DG-01, a data card per source.
docs/data/frap-perimeters.mdanddocs/data/dins.md, carrying the seven rows the standard's section 1 names.PROVENANCE.mdalready carried five of them; the two it did not are the tier (L1 for both) and the refresh cadence with its SLA.tests/test_data_cards.pyenumerates the cards against the declared source list rather than against what is on disk, so adding a source without a card fails rather than passing a directory listing.DG-04, the staleness alarm, as an AUTO-GATE.
This gate is built to fire, and firing is it working. The message says what to do: re-acquire and rebuild
site/, or move the SLA in the card with a reason. What it must not become is a number quietly raised to get a green build, which is the moveCONTRIBUTING.mdalready forbids for a marker set.The SLA is this project's declaration, and every card says so in those words, held by a test. Neither CAL FIRE nor FRAP promises a publication schedule and neither is asked to.
Where the clock is allowed to be
The obvious implementation is the one the standard's wording suggests: flag it in the report. That is exactly what must not happen here.
README.mdpromises the same inputs produce byte-identical output,tools/determinism.shis the gate behind that promise, andtest_no_wall_clock_leaks_into_an_artifactalready asserts it. A page computing its own staleness would read the wall clock at build time, two builds a day apart would differ, and of the two claims in conflict the one that would quietly lose is determinism, because its failure is invisible.So the artifacts publish
retrievedandstaleness_sla_daysand compute nothing from them. A consumer holding both does the subtraction against its own clock and gets the same answer this build would, whenever it asks. One function readsdate.today(), one caller passes it, and both sides of the SLA boundary are tested against fixed dates. ADR-0009 records that trade and names what it does not solve: ADR-0003's worry that apublishedmarker basis can go stale when a publisher revises a domain is a different check, and a file-level SLA only bounds the window.Proof each new check can fail
Against the pre-change source with
docs/data/absent, all 25 tests in the new file fail:The staleness gate is a pure function of a date, so it is separately run against dates it must reject:
test_the_staleness_gate_fires_once_the_sla_is_pastruns it at 2099-01-01 and requires every source flaggedtest_the_staleness_gate_fires_the_day_after_the_sla_and_not_the_day_ofpins both sides of the boundary per source, so an off-by-one is a failure rather than a gate that fires early or latePositive control, passing identically before and after:
tests/test_provenance.py, 14 tests, which reads the samesources.pyand is unaffected.The gate stacked underneath also earned its keep again: adding the policy fields without rebuilding
site/failedtest_the_published_artifacts_carry_the_reviewed_provenanceon both artifacts.Gate output
Exit code read from
$?.site/is rebuilt from the acquired files: four lines added to each artifact, all of them the policy fields. No measured number moves.🤖 Generated with Claude Code