fix(security,docs): the baseline gate saw no entries, and Art. 11 cited a missing index - #88
Merged
Merged
Conversation
…ed a missing index Three findings, one theme: a control that reports green because its scanner is narrower than the surface it guards. 1. scripts/check_baselines_expiry.py reported "OK — no expired or unannotated entries" while three HIGH-severity GKE checks sat suppressed in .security-baselines/tfsec.yml. It saw none of them: its yaml_entry pattern required a suppression id to start with an uppercase letter, which matches checkov ids (CKV_AWS_18) and misses every tfsec id (google-gke-enable-master-networks). Widening to lowercase alone would have been wrong — `framework: [terraform, kubernetes, dockerfile]` in checkov.yml is a sequence too. The scanner is now block-aware and treats only items under `exclude:` / `skip-check:` as entries, which is what its own comment had claimed since it was written. The three entries also carried `Review-by: 2027-01`, a format the gate does not parse. Normalised to `# expiry: 2027-01-01`, keeping the original justification prose. Verified: all three now seen and in-date, `--as-of 2027-06-01` fails all three, and `framework:` still yields no false positive. 2. .security-baselines/README.md step 4 has instructed reviewers to update docs/audit/baseline-review.md since the baselines were introduced. It did not exist. The first dated review now records all three suppressions with their compensating controls. All three are tool limitations rather than accepted risks: PSP was removed in Kubernetes 1.25 and PSS covers it via namespace labels; master_authorized_networks _config exists as a dynamic block tfsec cannot evaluate; the metadata attribute lives on the node pools, not the cluster tfsec inspects. 3. docs/COMPLIANCE_MAPPING.md cited an ADR index as EU AI Act Art. 11 evidence. The index did not exist, and the same row claimed "ADRs (37)" against 45 on disk — a compliance mapping pointing at a missing artefact and miscounting the one it has, falsifiable in one command. scripts/generate_adr_index.py generates docs/decisions/README.md from the files and --check fails CI when it goes stale, wired as a CI step and a pre-commit hook. It parses both heading conventions in use. The Art. 11 row now links the index instead of restating a count, removing the drift surface rather than correcting one instance of it. Baseline: two more entries resolved. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The gate suite is the local mirror of the CI contract; a gate that CI runs and `make verify` does not is the same blind spot in miniature. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
DuqueOM
force-pushed
the
fix/adr-index-and-baseline-review
branch
from
September 4, 2026 20:52
af8bbfd to
a335bac
Compare
This was referenced Sep 5, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third of three PRs closing the residue from #84. Three findings, one theme: a control that reports green because its scanner is narrower than the surface it guards.
1. The baseline expiry gate was watching nothing
scripts/check_baselines_expiry.pyreported:while three HIGH-severity GKE checks sat suppressed in
.security-baselines/tfsec.yml. It was not lying about the entries it saw — it saw none.Its
yaml_entrypattern required a suppression id to begin with an uppercase letter. That matches checkov ids (CKV_AWS_18) and misses every tfsec id (google-gke-enable-master-networks). Zero matches → zero expired → pass.Widening to lowercase alone would have been wrong.
checkov.ymlcontains aframework:list (terraform,kubernetes,dockerfile) — a sequence, but not a suppression. So the scanner is now block-aware and treats only items underexclude:/skip-check:as entries — exactly what its own comment had claimed since it was written:The three entries also carried
Review-by: 2027-01, a format the gate does not parse (it wants# expiry: YYYY-MM-DDon the entry line or the one directly above). Normalised, keeping the original justification prose intact.tfsec.yml--as-of 2027-06-01framework:2. The review record that was promised and never created
.security-baselines/README.mdstep 4 has instructed reviewers to "updatedocs/audit/baseline-review.md(next quarterly review)" since the baselines were introduced. The document did not exist, so three HIGH suppressions had a justification in a YAML comment and no review record anywhere. A suppression without a review record is indistinguishable from one nobody remembers making.The first dated review records all three with their compensating controls. Worth stating plainly: all three are tool limitations, not accepted risks —
master_authorized_networks_configis present, as adynamicblock. tfsec v1.28 does not evaluate dynamic blocks.remove_default_node_pool = true; tfsec inspects only the cluster.In all three the control exists and tfsec cannot see it. That is why the expiries are tied to the tfsec → trivy migration rather than to a risk-acceptance window.
3. An Art. 11 compliance claim an auditor could falsify in one command
docs/COMPLIANCE_MAPPING.mdcited an ADR index as EU AI Act Art. 11 evidence — "documentation sufficient to assess compliance". The index did not exist, and the same row claimed "ADRs (37)" against 45 on disk. Two defects in one compliance assertion.scripts/generate_adr_index.pygeneratesdocs/decisions/README.mdfrom the files themselves;--checkfails CI when it goes stale, wired as a CI step and a pre-commit hook. It handles both heading conventions in use (# ADR-001: Titleand# ADR-045 — Title) and cross-checks the number in the heading against the number in the filename.The Art. 11 row now links the index instead of restating a count — removing the drift surface rather than correcting one instance of it. Writing the index by hand would have reproduced the defect on the next ADR.
Baseline
Across the three PRs in this series,
.doc-path-baseline.ymlgoes from 7 entries to 2, and nounimplementedentries remain — only the two genuineruntime-artifactones.Evidence — Schema / Contract Test
scripts/generate_adr_index.py --checkis the contract for the index (verified in CI and pre-commit);scripts/check_baselines_expiry.py --as-ofis the contract for the expiry behaviour. Both are exercised below.Evidence — Real Execution Output
Evidence — CI Run Link
See the checks on this PR.
🤖 Generated with Claude Code