Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,31 @@ Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this project a

## [Unreleased]

### Fixed, a caveat that was true when written, false a week later, and pinned by a test

- **The Standards Conformance table said the applicability manifest had no entry for
this repository. It has had one since 2026-08-15.** The entry records the archetype as
`civic-data-tool`, publication as cleared, the tier as `B+C`, and fourteen of the
fifteen standards as applying with AI Evaluation N/A. The table now transcribes that,
with the date it was read, and says plainly that the registry lives in another
repository so nothing in CI can check the transcription against its source.
- **The test guarding that sentence could only ever pass.** It read
`assert "applicability manifest has no entry" in section` and carried the instruction
"Remove this test on the day the manifest entry exists, not before". It was green for
twelve days over a claim that had become false, because the only thing reading the
sentence was a test asserting the sentence was still there. It is replaced by tests
that refuse the false claim, hold the transcription's shape and date, and check the
one part that is mechanically checkable from inside this repository: that the table
marks exactly the one standard the registry marks N/A.
- **Observability moves from Tier C to Tier B+C**, which is what the registry records and
what OBS section 0 asks a repository with two surfaces to state. The B half is a real
gap and is recorded as one: section 8 wants a Core Web Vitals RUM beacon, which means
shipping a script that reports readers of a civic-data page back to somebody, and
these pages ship no script and this project takes no telemetry. That refusal is a
position rather than an oversight and the standard has no N/A for it. The
Lighthouse-CI lab gate, which needs no beacon, is also not met and is the half that
could be built.

### Fixed, the three pages could not say which of them a search result was for

- **One meta description was shared by all three pages, and none of them had a canonical.**
Expand Down
28 changes: 22 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -235,21 +235,37 @@ here is how much of each published field is actually filled in, and what the bla
## Standards conformance

This repository is held to the portfolio's shared engineering standards, pinned in
`.standards-version` to `v2.0.0`. Every row states what is true on 2026-08-15, not what is
`.standards-version` to `v2.0.0`. Every row states what is true on 2026-08-27, not what is
intended. "Applies (not met)" is a recorded gap; a blank state is a defect and
`tests/test_standards_conformance.py` fails on one.

Read one caveat first. The standards program's applicability manifest has no entry for
this repository, so nothing has ever decided which of the fifteen standards bind it. The
scoping below was derived here, from each standard's own applicability section, and it is
this table's reading rather than the registry's. A manifest entry supersedes it.
**Which standards bind this repository is the registry's decision, and the registry has
made it.** The applicability manifest carries an entry for `perimeter`, added
2026-08-15. Read 2026-08-27, it records the archetype as `civic-data-tool`, publication
as cleared and public since 2026-08-08, the tier as `B+C` (the published Pages site is
the B surface, the local Python and Node build is the C one), and the flags as
`html: true`, `hosted: true`, `dockerfile: false`, `llm: false`, `bilingual: false`. Of
the fifteen standards it marks fourteen as applying and AI Evaluation as N/A, for
"deterministic coverage counting over published CAL FIRE/FRAP datasets; no LLM/model
component".

This table used to open by saying the manifest had no entry and that the scoping below
was this table's reading rather than the registry's. That was true when it was written
and stopped being true the same week, and nothing here noticed, because the only test
reading the sentence was one asserting it was still present. The scoping now matches the
registry on all fifteen rows. One row moved to get there: Observability was recorded as
Tier C, and the registry says `B+C`.

The registry lives in another repository, so this is a transcription with a date, the
same way `docs/MARKERS.md` transcribes CAL FIRE's documents rather than reading them at
build time. Nothing in CI can check it against the source.

| Standard | State |
|---|---|
| Code Quality | Applies: uv, ruff, mypy `--strict`, pytest with branch coverage at 100% over `src/` with nothing omitted, against a 90% floor. `make lock-check` runs `uv lock --check`, because CQ-09's prescribed `uv sync --frozen` exits 0 on lockfile drift (measured 2026-08-15) |
| Security & Supply-Chain | Applies: semgrep, gitleaks, pip-audit, `npm audit`, CodeQL over actions/python/javascript, every action SHA-pinned, `permissions: contents: read` at the top of every workflow, `persist-credentials: false` on every checkout. Not met: no SBOM, no OpenSSF Scorecard workflow, no `osv-scanner` alongside pip-audit (SEC-11, SEC-13), no scheduled trufflehog run (SEC-19), and Dependabot alerts are disabled on the repository, so SEC-15 has nothing to read |
| CI/CD | Applies (not met). `main` has no ruleset and no branch protection, so the gates report and block nothing. The `protect-main` profile is committed at `.github/rulesets/main.json` and deliberately not applied; applying it is a live repository setting |
| Observability | Applies (Tier C). A library and a CLI writing to stdout, plus static pages with no script. No hosted service, no telemetry, no SLO surface. Not met: no operations runbook |
| Observability | Applies (Tier B+C), which is what the manifest records and what OBS section 0 asks a repo with two surfaces to state. Tier C is the library and CLI writing to stdout: OTel is out of scope with no network surface, which section 10 allows a Tier C surface to declare. Tier B is the published Pages site. Not met, and the B half is the larger gap: section 8 asks a Tier B frontend for a Core Web Vitals RUM beacon, which means shipping a script that reports readers of a civic-data page back to somebody, and these pages ship no script and this project takes no telemetry; that refusal is a position, not an oversight, and the standard has no N/A for it. Also not met: the Lighthouse-CI lab gate on LCP, INP and CLS, which does not need a beacon and is the half that could be built; no `docs/ROADMAP.md` carrying the tier declaration section 0 requires; and no operations runbook |
| Accessibility | Applies: html-validate and axe-core over the built pages in CI, in jsdom and again in Chromium, plus SC 1.4.10 Reflow at 320 by 256, contrast measured arithmetically over both palettes, and the same structural floor asserted from Python. An axe rule that comes back undecided fails the jsdom gate unless it is declared with a reason and with where it is checked instead; the browser run declares nothing and fails on any undecided rule. Nothing is suppressed anywhere, so there is no `waivers.yml` and A11Y-06 and A11Y-09 are met rather than waived. Not met: no ACR, and the manual checks README names under "What still needs a person" are unverified. Measured 2026-08-27: the browser run found `scrollable-region-focusable`, serious, on both measurement pages, which the jsdom run cannot see; the scroll containers are now named sections with `tabindex="0"` |
| Internationalization | Applies (not met). Civic data presented to the public is in scope per I18N section 1, and these pages are English only with no catalog and no `docs/I18N.md` declaration |
| AI Evaluation | N/A (no model, no LLM, no generated text anywhere in the pipeline or the pages) |
Expand Down
78 changes: 72 additions & 6 deletions tests/test_standards_conformance.py
Original file line number Diff line number Diff line change
Expand Up @@ -139,11 +139,77 @@ def test_the_table_carries_no_standard_this_portfolio_does_not_have() -> None:
assert not unknown, f"unrecognised standards in the table: {sorted(unknown)}"


def test_the_table_says_the_applicability_manifest_has_no_entry_for_this_repo() -> None:
"""FIX-02. Remove this test on the day the manifest entry exists, not before.
# --- FIX-02: the manifest entry, and what happened to the test that pinned its absence -

The scoping above was derived from each standard's own applicability section rather
than read from the registry, and a reader is owed that caveat while it is true.

def conformance_section() -> str:
return README.split("## Standards conformance", 1)[1].split("\n## ", 1)[0]


def test_the_table_does_not_claim_the_manifest_has_no_entry() -> None:
"""The entry exists. This asserted the opposite for twelve days.

The test that stood here read ``assert "applicability manifest has no entry" in
section`` and carried the instruction "Remove this test on the day the manifest
entry exists, not before". The entry was added to the registry on 2026-08-15, the
same week the sentence was written. Nothing here noticed, because the only thing
reading the sentence was a test asserting it was still present: green, permanently,
over a claim that had become false. A test that can only fail when somebody edits
the sentence it is pinning is not a check on the sentence.

This is the direction that can be checked from inside this repository. The registry
lives in another one, so the table's transcription of it is dated prose and no test
can compare it to the source. What a test can do is refuse the specific false claim
that was made, and hold the transcription's shape.
"""
section = conformance_section()
assert "applicability manifest has no entry" not in section, (
"the manifest carries an entry for this repository; the table must not say "
"otherwise (FIX-02)"
)
assert "manifest carries an entry" in section


@pytest.mark.parametrize(
"fact",
[
"civic-data-tool",
"`B+C`",
"cleared",
"added 2026-08-15",
"Read 2026-08-27",
],
)
def test_the_table_transcribes_the_manifest_entry(fact: str) -> None:
"""What the registry records, in the table, with the date it was read.

Same discipline as ``docs/MARKERS.md``: a document this project cannot re-read at
build time is transcribed with a date, and the date is part of the claim. Matched
against the section with its line breaks flattened, so rewrapping the paragraph
cannot break a check about its content.
"""
section = README.split("## Standards conformance", 1)[1].split("\n## ", 1)[0]
assert "applicability manifest has no entry" in section
assert fact in " ".join(conformance_section().split())


def test_the_transcription_says_it_cannot_be_checked_from_here() -> None:
"""A transcription that reads like a live check is worse than no transcription."""
assert "Nothing in CI can check it against the source." in conformance_section()


def test_the_table_marks_exactly_the_one_standard_the_manifest_marks_not_applicable() -> (
None
):
"""The registry marks one N/A and fourteen applying. A second N/A here is a divergence.

This is the one part of the transcription that is mechanically checkable from inside
the repository, because the verdicts are in this table. It does not prove the table
agrees with the registry; it fails when the table stops agreeing with what it says
the registry said.
"""
not_applicable = sorted(
name for name, state in conformance_rows().items() if state.startswith("N/A")
)
assert not_applicable == ["AI Evaluation"], (
"the manifest marks AI Evaluation N/A and fourteen standards as applying; "
f"this table marks {not_applicable}"
)
Loading