Skip to content

A retrieval that goes stale fails the build - #31

Merged
ChelseaKR merged 3 commits into
mainfrom
feat/data-cards-and-a-staleness-gate
Aug 29, 2026
Merged

A retrieval that goes stale fails the build#31
ChelseaKR merged 3 commits into
mainfrom
feat/data-cards-and-a-staleness-gate

Conversation

@ChelseaKR

Copy link
Copy Markdown
Owner

Closes the DG-01 and DG-04 gaps the Data Governance row recorded. Adds ADR-0009. Stacked on #30; base is docs/the-manifest-entry-exists.

What was missing

Every figure these pages publish is a count of a file downloaded on one day. The pages say the date, PROVENANCE.md says the version and the hash, and none of that is the same thing as noticing. DINS carries no version string and is appended to as inspections complete, so it differs from the acquired file the day after the acquisition and increasingly every season after.

Nothing here raised its hand, and nothing could: no code in this repository had ever asked what today is.

The README's Data Governance row had recorded the gap: "no refresh cadence, staleness SLA or stated tier".

What this adds

DG-01, a data card per source. docs/data/frap-perimeters.md and docs/data/dins.md, carrying the seven rows the standard's section 1 names. PROVENANCE.md already carried five of them; the two it did not are the tier (L1 for both) and the refresh cadence with its SLA. tests/test_data_cards.py enumerates the cards against the declared source list rather than against what is on disk, so adding a source without a card fails rather than passing a directory listing.

DG-04, the staleness alarm, as an AUTO-GATE.

Source SLA Why that number Fires on
FRAP perimeters 400 days one annual release cycle plus about five weeks, so a retrieval does not go stale on the eve of the version that would replace it 2027-09-11
CAL FIRE DINS 180 days about one fire season; no version string, appended to as inspections complete, so there is no release to wait for 2027-02-03

This gate is built to fire, and firing is it working. The message says what to do: re-acquire and rebuild site/, or move the SLA in the card with a reason. What it must not become is a number quietly raised to get a green build, which is the move CONTRIBUTING.md already forbids for a marker set.

The SLA is this project's declaration, and every card says so in those words, held by a test. Neither CAL FIRE nor FRAP promises a publication schedule and neither is asked to.

Where the clock is allowed to be

The obvious implementation is the one the standard's wording suggests: flag it in the report. That is exactly what must not happen here.

README.md promises the same inputs produce byte-identical output, tools/determinism.sh is the gate behind that promise, and test_no_wall_clock_leaks_into_an_artifact already asserts it. A page computing its own staleness would read the wall clock at build time, two builds a day apart would differ, and of the two claims in conflict the one that would quietly lose is determinism, because its failure is invisible.

So the artifacts publish retrieved and staleness_sla_days and compute nothing from them. A consumer holding both does the subtraction against its own clock and gets the same answer this build would, whenever it asks. One function reads date.today(), one caller passes it, and both sides of the SLA boundary are tested against fixed dates. ADR-0009 records that trade and names what it does not solve: ADR-0003's worry that a published marker basis can go stale when a publisher revises a domain is a different check, and a file-level SLA only bounds the window.

Proof each new check can fail

Against the pre-change source with docs/data/ absent, all 25 tests in the new file fail:

FAILED tests/test_data_cards.py::test_the_card_says_the_sla_is_this_projects_declaration[frap_perimeters]
FAILED tests/test_data_cards.py::test_the_cards_carry_no_em_dashes - Attribut...
FAILED tests/test_data_cards.py::test_no_published_retrieval_is_older_than_the_sla_its_card_states
FAILED tests/test_data_cards.py::test_the_staleness_gate_fires_once_the_sla_is_past
FAILED tests/test_data_cards.py::test_the_staleness_gate_fires_the_day_after_the_sla_and_not_the_day_of[dins_postfire]
25 failed in 0.14s

The staleness gate is a pure function of a date, so it is separately run against dates it must reject:

  • test_the_staleness_gate_fires_once_the_sla_is_past runs it at 2099-01-01 and requires every source flagged
  • test_the_staleness_gate_fires_the_day_after_the_sla_and_not_the_day_of pins both sides of the boundary per source, so an off-by-one is a failure rather than a gate that fires early or late

Positive control, passing identically before and after: tests/test_provenance.py, 14 tests, which reads the same sources.py and is unaffected.

The gate stacked underneath also earned its keep again: adding the policy fields without rebuilding site/ failed test_the_published_artifacts_carry_the_reviewed_provenance on both artifacts.

Gate output

$ make verify
...
Required test coverage of 90% reached. Total coverage: 100.00%
============================= 606 passed in 15.64s =============================
...
  8 passed (3.1s)
determinism: build/run-one and build/run-two are byte-identical (5 files)
MAKE_VERIFY_EXIT=0

Exit code read from $?.

site/ is rebuilt from the acquired files: four lines added to each artifact, all of them the policy fields. No measured number moves.

🤖 Generated with Claude Code

@ChelseaKR
ChelseaKR changed the base branch from docs/the-manifest-entry-exists to main August 29, 2026 00:15
@ChelseaKR
ChelseaKR force-pushed the feat/data-cards-and-a-staleness-gate branch from 3f5ad08 to 35cae44 Compare August 29, 2026 00:31
ChelseaKR and others added 3 commits August 29, 2026 10:04
Every figure these pages publish is a count of a file downloaded on one day. The
pages say the date, PROVENANCE.md says the version and the hash, and none of that
is the same thing as noticing. DINS carries no version string and is appended to
as inspections complete, so it differs from the acquired file the day after the
acquisition and increasingly every season after. Nothing here raised its hand, and
nothing could: no code in this repository had ever asked what today is.

DG-01 asks for a data card per ingest source with seven named rows. PROVENANCE.md
carried five of them. The two it had never carried are the ones the README's Data
Governance row recorded as missing: the tier, L1 for both, and the refresh cadence
with its staleness SLA. docs/data/frap-perimeters.md and docs/data/dins.md are
those cards, and tests/test_data_cards.py enumerates them against the declared
source list rather than against what is on disk, so adding a source without a card
fails rather than passing.

DG-04 makes the staleness alarm an AUTO-GATE, and this is it. 400 days for the
FRAP layer, which is one annual release cycle plus about five weeks so a retrieval
does not go stale on the eve of the version that would replace it. 180 days for
DINS, which is about one fire season. On the retrieval on record the gate fires
2027-02-03 for DINS and 2027-09-11 for FRAP. Firing is it working, and the failure
message says what to do: re-acquire and rebuild site/, or move the SLA in the card
with a reason. What it must not become is a number quietly raised to get a green
build, which is the move CONTRIBUTING.md already forbids for a marker set.

The clock is in the gate and not in the artifacts, on purpose. A page computing
its own staleness would read the wall clock at build time and two builds a day
apart would differ, which is in direct conflict with the byte-identical output
make determinism exists to protect, and determinism is the claim that would
quietly lose. So the artifacts publish retrieved and staleness_sla_days and
compute nothing from them; a consumer holding both does the subtraction against
its own clock and gets the same answer this build would. One function reads
date.today(), one caller passes it, and both sides of the SLA boundary are tested
against fixed dates.

The SLA is this project's declaration and every card says so. Neither CAL FIRE nor
FRAP promises a publication schedule and neither is asked to.

Measured on this tree. Against the pre-change source and with docs/data/ absent,
all 25 tests in tests/test_data_cards.py fail, on AttributeError for the fields
sources.py did not carry and on the cards not existing. tests/test_provenance.py
passes identically in both states, 14 tests. After, make verify exits 0 with 606
tests and 100% branch coverage.

site/ is rebuilt from the acquired files: four lines added to each artifact, the
policy fields, and no measured number moves.

See ADR-0009.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nine open PRs and four open issues had accumulated past what one person can
work through in a sitting, so each PR is now written up to be decidable on
its own: what it does, what was verified rather than taken from the body,
the defects found, and a single recommendation.

The findings worth surfacing: no PR in the queue is failing CI; codeql never
ran on the four PRs that target another branch rather than main, so their
CodeQL result is absent rather than green; the two three-PR stacks collide in
CHANGELOG.md, site/dins.html and render.py whichever order they land in;
merging a stack base with --delete-branch auto-closes its dependent, which
has already happened once here; and nothing in the queue should be closed.

This is the review, not its execution. Nothing was merged, closed, retargeted
or rebased to produce it, and no recommendation in it has been acted on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… SLA day

ADR-0009 said the staleness gate "fires for DINS on 2027-02-03 and for FRAP on
2027-09-11". Those are the last days it passes. The comparison in
tests/test_data_cards.py is `age > source.staleness_sla_days`, so on the SLA day
itself age equals the SLA and the source is not stale; the first failing day is
the one after. Retrieval on record is 2026-08-07, so DINS first fires 2027-02-04
and FRAP 2027-09-12.

The code was never wrong and is not touched here. Its own boundary test,
test_the_staleness_gate_fires_the_day_after_the_sla_and_not_the_day_of, asserts
both sides of exactly this line, and docs/PR-TRIAGE.md on the same branch reads
it correctly ("from 2027-02-04 every CI run goes red"). The ADR was the one
document in the set that stated it a day early, and an ADR is published output.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ChelseaKR
ChelseaKR force-pushed the feat/data-cards-and-a-staleness-gate branch from 35cae44 to ae0e2d6 Compare August 29, 2026 17:05
@ChelseaKR
ChelseaKR merged commit 5376bf2 into main Aug 29, 2026
6 checks passed
@ChelseaKR
ChelseaKR deleted the feat/data-cards-and-a-staleness-gate branch August 29, 2026 17:08
ChelseaKR added a commit that referenced this pull request Aug 29, 2026
This file said "Two of the five contexts in `main.json` do not exist on `main`
yet" and made landing them step 1 of the apply order. It said the same thing a
second time in the `bypass_actors` bullet. Both are now false.

The sentence was written on 2026-08-15 in 18827df, at 17:14. `zizmor`, one of
the two contexts it meant, landed on `main` in #16 (839557e) the same day, and
`sast` had been a job in ci.yml since ea06580 on 2026-08-07. So it was true when
written on its own branch and false within the day, and nothing here noticed for
two weeks, which is the shape #30 fixed in the Standards Conformance table and
ADR-0009 turned out to have in its fire dates.

Re-measured 2026-08-29 rather than reasoned about. `verify`, `secret-scan`,
`sast` and `zizmor` are jobs in .github/workflows/ci.yml on `main`;
`codeql (actions · python · javascript)` is the `name:` of the `analyze` job in
codeql.yml, which triggers on `pull_request: branches: [main]`. Read off the two
most recent pull request head commits, 62c406e (#30) and ae0e2d6 (#31), each
reported all five contexts.

The other dated claims in the file were re-read at the same time and are
unchanged: no ruleset is applied, `"protected": false`, and the branch
protection endpoint still 404s. The dependabot queue that step 2 said to drain
is drained, measured as 0 open. The two completed steps are kept as history
rather than deleted, so the order stays legible, and the remaining steps are
renumbered.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ChelseaKR added a commit that referenced this pull request Aug 29, 2026
#32)

* fix: the committed ruleset would lock the owner out on its first apply

No ruleset is applied on this repository. `gh api repos/ChelseaKR/perimeter/rulesets`
returned `[]` on 2026-08-15 and again on 2026-08-28, so nothing is broken yet;
what was committed was a first application that would have broken it.
`.github/rulesets/main.json` carried `"bypass_actors": []`, and the ruleset
README argued for it by name, calling the empty list "the stricter reading" of
CICD-15.

It is not the stricter reading. `bypass_actors` now holds exactly the
repository owner's standing bypass, `RepositoryRole` 5 with
`bypass_mode: always`, deliberately and permanently: an agent once applied a
ruleset with no bypass and locked the owner out of their own repository, and
restoring access took a sweep across eighteen repositories. An empty list
there is not a stricter gate, it is the lockout.

That matters more here than anywhere else in the portfolio, because this
profile has never met a live ruleset. It asks for `required_signatures`,
`required_linear_history`, a strict up-to-date policy and five required
contexts, two of which do not exist on `main` yet. That is a lot of ways for a
first application to wedge, and the empty list would have removed the only way
back in that does not go through GitHub support. `bypass_mode: always` rather
than CICD-15's `pull_request` for the same reason: a bypass that only works
inside a pull request is no use when the pull request is the thing that is
wedged.

The README bullet is replaced with the reversal and the reasoning rather than
quietly rewritten, the apply procedure gains a step that checks the bypass
actually came through, and "what is true today" now states plainly that no
ruleset is applied and that this file has therefore never been corrected by a
live one.

Nothing here touches a live setting; there is none to touch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: check the bypass before posting, not only after

The ordered list is what a reader follows, so the check belongs in it as a
step rather than only in the confirmation after the fence. Records that POST
adds a ruleset rather than replacing one: rules from every applicable ruleset
combine and bypass actors are per-ruleset, so posting twice without deleting
leaves an empty-bypass ruleset over `main` that blocks the owner whatever the
first one allows.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(rulesets): all five required contexts exist, measured 2026-08-29

This file said "Two of the five contexts in `main.json` do not exist on `main`
yet" and made landing them step 1 of the apply order. It said the same thing a
second time in the `bypass_actors` bullet. Both are now false.

The sentence was written on 2026-08-15 in 18827df, at 17:14. `zizmor`, one of
the two contexts it meant, landed on `main` in #16 (839557e) the same day, and
`sast` had been a job in ci.yml since ea06580 on 2026-08-07. So it was true when
written on its own branch and false within the day, and nothing here noticed for
two weeks, which is the shape #30 fixed in the Standards Conformance table and
ADR-0009 turned out to have in its fire dates.

Re-measured 2026-08-29 rather than reasoned about. `verify`, `secret-scan`,
`sast` and `zizmor` are jobs in .github/workflows/ci.yml on `main`;
`codeql (actions · python · javascript)` is the `name:` of the `analyze` job in
codeql.yml, which triggers on `pull_request: branches: [main]`. Read off the two
most recent pull request head commits, 62c406e (#30) and ae0e2d6 (#31), each
reported all five contexts.

The other dated claims in the file were re-read at the same time and are
unchanged: no ruleset is applied, `"protected": false`, and the branch
protection endpoint still 404s. The dependabot queue that step 2 said to drain
is drained, measured as 0 open. The two completed steps are kept as history
rather than deleted, so the order stays legible, and the remaining steps are
renumbered.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: an empty bypass list is a test failure, not a comment

The previous commit in this branch corrected `"bypass_actors": []` in
.github/rulesets/main.json. Correcting the file is not the fix, because nothing
in this repository reads that file. Measured before writing this: no test and no
source module referenced `ruleset` or `bypass_actors` anywhere, so the value
could regress to the empty list in one edit and every gate would stay green.

That is what made the committed file dangerous rather than merely wrong. It is
never applied, so no live ruleset has ever contradicted it, and nothing read it,
so nothing here could contradict it either. The apply command in the README
posts the file as it stands, which means following this repository's own
documented procedure was enough to lock the owner out of it.

tests/test_ruleset.py makes the empty list red. `lockout_risk` is a pure
function of a parsed document, run against the five shapes that lose the
bypass (empty list, absent key, wrong type, a different actor, and the owner
with `bypass_mode: pull_request`) as well as against the committed file, with a
positive control so it is not passing by refusing everything.

`load_ruleset` fails on a missing or unparseable file rather than returning an
empty document. A check that passes when its subject is absent is the defect
class this repository keeps finding, and it would be a poor joke to introduce
one here.

This does not apply the ruleset and does not ask anyone to. Applying it changes
a live repository setting and remains the owner's call.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): the guard, the re-measurement, and what is still not applied

Records the three things the ruleset entry did not yet cover: that nothing read
main.json before this branch so correcting it once would not have held, that the
apply order's claim about missing required contexts was re-measured and is
false, and that landing this changes nothing on the server.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant