Skip to content

The manifest entry this table said did not exist - #30

Merged
ChelseaKR merged 1 commit into
mainfrom
docs/the-manifest-entry-exists
Aug 29, 2026
Merged

The manifest entry this table said did not exist#30
ChelseaKR merged 1 commit into
mainfrom
docs/the-manifest-entry-exists

Conversation

@ChelseaKR

Copy link
Copy Markdown
Owner

Part of #15. Stacked on #29; base is feat/a11y-in-a-real-browser.

What was wrong

The Standards Conformance table opened with:

Read one caveat first. The standards program's applicability manifest has no entry for this repository, so nothing has ever decided which of the fifteen standards bind it.

The entry was added on 2026-08-15, in STANDARDS commit 36747be, "fix(applicability): correct the manifest, and register the repo that owns it". Verified at that repo's HEAD, applicability.yml:1171:

  perimeter:
   archetype: civic-data-tool
   publication: cleared   # repo is PUBLIC on GitHub since 2026-08-08
   tier: B+C              # GitHub Pages site + local Python/Node build
   flags: { html: true, dockerfile: false, llm: false, bilingual: false, hosted: true }
   standards:
     ...
     AIEV: { na: "deterministic coverage counting over published CAL FIRE/FRAP datasets; no LLM/model component" }

The sentence was true when it was written and false the same week.

Why nothing noticed for twelve days

def test_the_table_says_the_applicability_manifest_has_no_entry_for_this_repo() -> None:
    """FIX-02. Remove this test on the day the manifest entry exists, not before."""
    assert "applicability manifest has no entry" in section

That test cannot fail while the sentence is present, and cannot notice when the sentence stops being true, because the only thing it reads is the sentence. It was a permanently green check over a false claim about this repository's own governance, in the section whose whole job is to state what is true today. It is the same shape as the four checks ADR-0004 was written about, in a place nobody thought to look.

What the table says now

It transcribes the entry, with the date it was read, and says plainly that the registry lives in another repository so nothing in CI can check the transcription against its source. That is the discipline docs/MARKERS.md already applies to CAL FIRE's documents: a source that cannot be re-read at build time is transcribed with a date, and the date is part of the claim.

One row moved to match the registry. Observability was Applies (Tier C); the registry says B+C, which is also what OBS section 0 asks a repository with two surfaces to state.

The B half is a genuine gap and is recorded as one rather than talked around. OBS section 8 wants a Core Web Vitals RUM beacon from a Tier B frontend, which means shipping a script that reports readers of a civic-data page back to somebody. These pages ship no script and this project takes no telemetry. That refusal is a position, not an oversight, and the standard has no N/A for it, so the row says so. The Lighthouse-CI lab gate on LCP, INP and CLS needs no beacon, is also not met, and is named as the half that could be built.

Proof each new check can fail

Against the pre-change README, with the new tests in place:

FAILED tests/test_standards_conformance.py::test_the_table_does_not_claim_the_manifest_has_no_entry
FAILED tests/test_standards_conformance.py::test_the_table_transcribes_the_manifest_entry[civic-data-tool]
FAILED tests/test_standards_conformance.py::test_the_table_transcribes_the_manifest_entry[`B+C`]
FAILED tests/test_standards_conformance.py::test_the_table_transcribes_the_manifest_entry[cleared]
FAILED tests/test_standards_conformance.py::test_the_table_transcribes_the_manifest_entry[added 2026-08-15]
FAILED tests/test_standards_conformance.py::test_the_table_transcribes_the_manifest_entry[Read 2026-08-27]
FAILED tests/test_standards_conformance.py::test_the_transcription_says_it_cannot_be_checked_from_here
7 failed, 50 passed in 0.06s

Positive controls, 31 of them in the same file, passing identically before and after:

$ uv run pytest tests/test_standards_conformance.py -q -k "no_row_is_blank or every_standard_has_a_row or pin_names_a_released_tag"
31 passed, 26 deselected in 0.02s

test_the_table_marks_exactly_the_one_standard_the_manifest_marks_not_applicable is also a control: it passed before this change too, because the table already marked exactly one. It is added because it is the one part of the transcription a test inside this repository can check, and it fails the day a second row is marked N/A without the registry saying so.

Gate output

$ make verify
...
Required test coverage of 90% reached. Total coverage: 100.00%
============================= 581 passed in 27.51s =============================
...
determinism: build/run-one and build/run-two are byte-identical (5 files)
MAKE_VERIFY_EXIT=0

Exit code read from $?.

What is still open on #15, and why this PR does not close it

🤖 Generated with Claude Code

@ChelseaKR
ChelseaKR changed the base branch from feat/a11y-in-a-real-browser to main August 29, 2026 00:15
@ChelseaKR
ChelseaKR force-pushed the docs/the-manifest-entry-exists branch from de52e4a to b3ba67c Compare August 29, 2026 00:31
The Standards Conformance table opened by saying the standards program's
applicability manifest had no entry for this repository, so nothing had ever
decided which of the fifteen standards bind it, and that the scoping below was
this table's reading rather than the registry's.

The entry was added on 2026-08-15, in STANDARDS commit 36747be, "fix(applicability):
correct the manifest, and register the repo that owns it". The sentence was true
when it was written and false the same week.

Nothing here noticed for twelve days, and the reason is the shape worth recording:

    def test_the_table_says_the_applicability_manifest_has_no_entry_for_this_repo():
        assert "applicability manifest has no entry" in section

with the instruction "Remove this test on the day the manifest entry exists, not
before". That test cannot fail while the sentence is present and cannot notice
when the sentence stops being true, because the only thing it reads is the
sentence. It was a green check over a false claim about this repository's own
governance, in the section whose whole job is to state what is true today.

The table now transcribes what the registry records: archetype civic-data-tool,
publication cleared and public since 2026-08-08, tier B+C, flags html and hosted
true, fourteen standards applying and AI Evaluation N/A for "deterministic
coverage counting over published CAL FIRE/FRAP datasets; no LLM/model component".
It carries the date it was read and says that the registry lives in another
repository, so nothing in CI can check the transcription against its source. Same
discipline docs/MARKERS.md applies to CAL FIRE's documents: a source that cannot
be re-read at build time is transcribed with a date, and the date is part of the
claim.

One row moved to match the registry. Observability was recorded as Tier C and the
registry says B+C, which is also what OBS section 0 asks a repository with two
surfaces to state. The B half is a real gap: section 8 wants a Core Web Vitals RUM
beacon, which means shipping a script that reports readers of a civic-data page
back to somebody, and these pages ship no script and this project takes no
telemetry. That refusal is a position rather than an oversight, and the standard
has no N/A for it, so it is recorded as not met and says why. The Lighthouse-CI
lab gate needs no beacon, is also not met, and is the half that could be built.

The replacement tests refuse the specific false claim, hold the transcription's
shape and its date, and check the one thing that is mechanically checkable from
inside this repository: that the table marks exactly the one standard the registry
marks N/A.

Measured on this tree. Against the pre-change README, seven of the new tests fail,
including:

    FAILED test_the_table_does_not_claim_the_manifest_has_no_entry
    FAILED test_the_table_transcribes_the_manifest_entry[civic-data-tool]
    FAILED test_the_transcription_says_it_cannot_be_checked_from_here
    7 failed, 50 passed

and 31 positive controls in the same file pass in both states.

make verify exits 0: 581 tests, 100% branch coverage, determinism byte-identical.

Part of #15. The other half of that issue, applying .github/rulesets/main.json
and enabling Dependabot alerts, is a live repository setting and is not something
a pull request can do.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ChelseaKR
ChelseaKR force-pushed the docs/the-manifest-entry-exists branch from b3ba67c to 62c406e Compare August 29, 2026 17:01
@ChelseaKR
ChelseaKR merged commit 5abbcc0 into main Aug 29, 2026
6 checks passed
@ChelseaKR
ChelseaKR deleted the docs/the-manifest-entry-exists branch August 29, 2026 17:04
ChelseaKR added a commit that referenced this pull request Aug 29, 2026
This file said "Two of the five contexts in `main.json` do not exist on `main`
yet" and made landing them step 1 of the apply order. It said the same thing a
second time in the `bypass_actors` bullet. Both are now false.

The sentence was written on 2026-08-15 in 18827df, at 17:14. `zizmor`, one of
the two contexts it meant, landed on `main` in #16 (839557e) the same day, and
`sast` had been a job in ci.yml since ea06580 on 2026-08-07. So it was true when
written on its own branch and false within the day, and nothing here noticed for
two weeks, which is the shape #30 fixed in the Standards Conformance table and
ADR-0009 turned out to have in its fire dates.

Re-measured 2026-08-29 rather than reasoned about. `verify`, `secret-scan`,
`sast` and `zizmor` are jobs in .github/workflows/ci.yml on `main`;
`codeql (actions · python · javascript)` is the `name:` of the `analyze` job in
codeql.yml, which triggers on `pull_request: branches: [main]`. Read off the two
most recent pull request head commits, 62c406e (#30) and ae0e2d6 (#31), each
reported all five contexts.

The other dated claims in the file were re-read at the same time and are
unchanged: no ruleset is applied, `"protected": false`, and the branch
protection endpoint still 404s. The dependabot queue that step 2 said to drain
is drained, measured as 0 open. The two completed steps are kept as history
rather than deleted, so the order stays legible, and the remaining steps are
renumbered.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ChelseaKR added a commit that referenced this pull request Aug 29, 2026
#32)

* fix: the committed ruleset would lock the owner out on its first apply

No ruleset is applied on this repository. `gh api repos/ChelseaKR/perimeter/rulesets`
returned `[]` on 2026-08-15 and again on 2026-08-28, so nothing is broken yet;
what was committed was a first application that would have broken it.
`.github/rulesets/main.json` carried `"bypass_actors": []`, and the ruleset
README argued for it by name, calling the empty list "the stricter reading" of
CICD-15.

It is not the stricter reading. `bypass_actors` now holds exactly the
repository owner's standing bypass, `RepositoryRole` 5 with
`bypass_mode: always`, deliberately and permanently: an agent once applied a
ruleset with no bypass and locked the owner out of their own repository, and
restoring access took a sweep across eighteen repositories. An empty list
there is not a stricter gate, it is the lockout.

That matters more here than anywhere else in the portfolio, because this
profile has never met a live ruleset. It asks for `required_signatures`,
`required_linear_history`, a strict up-to-date policy and five required
contexts, two of which do not exist on `main` yet. That is a lot of ways for a
first application to wedge, and the empty list would have removed the only way
back in that does not go through GitHub support. `bypass_mode: always` rather
than CICD-15's `pull_request` for the same reason: a bypass that only works
inside a pull request is no use when the pull request is the thing that is
wedged.

The README bullet is replaced with the reversal and the reasoning rather than
quietly rewritten, the apply procedure gains a step that checks the bypass
actually came through, and "what is true today" now states plainly that no
ruleset is applied and that this file has therefore never been corrected by a
live one.

Nothing here touches a live setting; there is none to touch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: check the bypass before posting, not only after

The ordered list is what a reader follows, so the check belongs in it as a
step rather than only in the confirmation after the fence. Records that POST
adds a ruleset rather than replacing one: rules from every applicable ruleset
combine and bypass actors are per-ruleset, so posting twice without deleting
leaves an empty-bypass ruleset over `main` that blocks the owner whatever the
first one allows.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(rulesets): all five required contexts exist, measured 2026-08-29

This file said "Two of the five contexts in `main.json` do not exist on `main`
yet" and made landing them step 1 of the apply order. It said the same thing a
second time in the `bypass_actors` bullet. Both are now false.

The sentence was written on 2026-08-15 in 18827df, at 17:14. `zizmor`, one of
the two contexts it meant, landed on `main` in #16 (839557e) the same day, and
`sast` had been a job in ci.yml since ea06580 on 2026-08-07. So it was true when
written on its own branch and false within the day, and nothing here noticed for
two weeks, which is the shape #30 fixed in the Standards Conformance table and
ADR-0009 turned out to have in its fire dates.

Re-measured 2026-08-29 rather than reasoned about. `verify`, `secret-scan`,
`sast` and `zizmor` are jobs in .github/workflows/ci.yml on `main`;
`codeql (actions · python · javascript)` is the `name:` of the `analyze` job in
codeql.yml, which triggers on `pull_request: branches: [main]`. Read off the two
most recent pull request head commits, 62c406e (#30) and ae0e2d6 (#31), each
reported all five contexts.

The other dated claims in the file were re-read at the same time and are
unchanged: no ruleset is applied, `"protected": false`, and the branch
protection endpoint still 404s. The dependabot queue that step 2 said to drain
is drained, measured as 0 open. The two completed steps are kept as history
rather than deleted, so the order stays legible, and the remaining steps are
renumbered.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: an empty bypass list is a test failure, not a comment

The previous commit in this branch corrected `"bypass_actors": []` in
.github/rulesets/main.json. Correcting the file is not the fix, because nothing
in this repository reads that file. Measured before writing this: no test and no
source module referenced `ruleset` or `bypass_actors` anywhere, so the value
could regress to the empty list in one edit and every gate would stay green.

That is what made the committed file dangerous rather than merely wrong. It is
never applied, so no live ruleset has ever contradicted it, and nothing read it,
so nothing here could contradict it either. The apply command in the README
posts the file as it stands, which means following this repository's own
documented procedure was enough to lock the owner out of it.

tests/test_ruleset.py makes the empty list red. `lockout_risk` is a pure
function of a parsed document, run against the five shapes that lose the
bypass (empty list, absent key, wrong type, a different actor, and the owner
with `bypass_mode: pull_request`) as well as against the committed file, with a
positive control so it is not passing by refusing everything.

`load_ruleset` fails on a missing or unparseable file rather than returning an
empty document. A check that passes when its subject is absent is the defect
class this repository keeps finding, and it would be a poor joke to introduce
one here.

This does not apply the ruleset and does not ask anyone to. Applying it changes
a live repository setting and remains the owner's call.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): the guard, the re-measurement, and what is still not applied

Records the three things the ruleset entry did not yet cover: that nothing read
main.json before this branch so correcting it once would not have held, that the
apply order's claim about missing required contexts was re-measured and is
false, and that landing this changes nothing on the server.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant