Skip to content

fix(docs,ci): two baseline entries were misclassified, and eight gates had no local hook - #86

Merged
DuqueOM merged 1 commit into
mainfrom
fix/reference-hygiene-and-verify
Sep 4, 2026
Merged

fix(docs,ci): two baseline entries were misclassified, and eight gates had no local hook#86
DuqueOM merged 1 commit into
mainfrom
fix/reference-hygiene-and-verify

Conversation

@DuqueOM

@DuqueOM DuqueOM commented Sep 4, 2026

Copy link
Copy Markdown
Owner

First of three PRs closing the residue from #84.

Two of my five "unimplemented" entries were misreadings

scripts/smoke_test.py was never missing. The release-checklist skill says, in so many words:

the template does not ship a scripts/smoke_test.py script (would compete with deploy-common.yml as SSOT)

It is a deliberate, documented non-existence. The path gate flagged it only because the sentence wrapped the path in a code span, and my baseline then recorded a correct design decision as a defect. Rewritten as plain prose, per the convention docs/governance/doc-path-references.md already states: a code span asserts this resolves, so a path you are talking about is not one.

scripts/load_test_services.py was a wrong path, not a missing file. The artefact exists:

Referenced Reality
locust -f scripts/load_test_services.py (×3 in the load-test workflow) templates/service/tests/load_test.py, run by make load-test as locust -f tests/load_test.py

An agent following that workflow fails on every one of the three commands. Corrected to tests/load_test.py — the form that is right from inside a generated service, which is where the workflow runs.

Baseline: 7 entries → 5.

A broken link that was suppressed, not unnoticed

templates/service/docs/ADOPTION.md linked [SECURITY.md](../SECURITY.md). That resolves to templates/service/SECURITY.md here, and to the service root in a generated service. Neither exists.

I previously reported this as "the Link Check does not flag it". That was wrong, and the truth is worse — .github/markdown-link-check.json carried a dedicated suppression for that exact link:

{ "pattern": "^\\.\\./SECURITY\\.md$" }

Someone saw it and silenced it. The sentence is about this template's disclosure SLA, so the link now points at the upstream SECURITY.md absolutely, and the suppression is deleted. The repo-root copy of the same document resolves correctly and is untouched.

The systemic half: eight gates with no local hook

Eight jobs in validate-templates.yml had no pre-commit hook at all:

doc-coherence · cicd-template-drift · vendored-runtime-drift · common-utils-drift · dashboard-inventory · baselines-expiry · test-clock-isolation · agentic-adapter-sync

So there was no way to run the full gate set locally, and the only feedback loop was push-and-read-CI. That is exactly how three separate breakages reached the remote on #84.

All eight are now pre-commit hooks, path-filtered so each fires only on the files it guards.

This is deliberately not a revival of the pre-push stage retired in R5-L4. That decision was about a 60-second scaffold hook training --no-verify on contributors, and it cites an industry budget of pre-commit < 5 s. All thirteen gates together run in 0.9 s, so the reasoning does not transfer. The justification is written next to the original decision in .pre-commit-config.yaml so the two are read together rather than looking like a silent reversal.

make verify runs the same thirteen in one command for the sweep-before-PR case, reporting every failure rather than stopping at the first — a contributor who broke three things should learn that in one run. Slow end-to-end stays in make smoke.

$ make verify
  check_doc_coherence                PASS
  check_doc_path_refs                PASS
  check_cicd_template_drift          PASS
  check_vendored_runtime_drift       PASS
  ... 9 more ...
✓ every gate green — safe to push

Verified negatively

Test Result
Introduce a dead path, run make verify ✅ exit 1, check_doc_path_refs FAIL, names the path
Break a vendored runbook's byte-identity, git add, run the hook ✅ blocked at commit time — the exact mistake that reached CI on #84
Clean tree ✅ all thirteen PASS

🤖 Generated with Claude Code

…s had no local hook

Reviewing the baseline I filed in #84 showed two of its five
"unimplemented" entries were my own misreadings.

scripts/smoke_test.py was never missing. The release-checklist skill says
in so many words that "the template does not ship a scripts/smoke_test.py
script (would compete with deploy-common.yml as SSOT)". It is a
deliberate, documented non-existence, and the baseline recorded a correct
design decision as a defect. The gate flagged it only because the sentence
wrapped the path in a code span; rewritten as plain prose per the
convention the governance doc already states.

scripts/load_test_services.py was a wrong path, not a missing file. The
artefact exists as templates/service/tests/load_test.py, run via
`make load-test`. The load-test workflow gave three
`locust -f scripts/load_test_services.py` commands that fail as written.
Corrected to tests/load_test.py, the form that is right from inside a
generated service. Baseline: 7 entries -> 5.

templates/service/docs/ADOPTION.md linked ../SECURITY.md, which resolves
to templates/service/SECURITY.md here and to the service root in a
generated service. Neither exists. It never failed CI because
.github/markdown-link-check.json carried a dedicated ignorePatterns entry
silencing that exact link — it was suppressed, not unnoticed. The sentence
is about this template's disclosure SLA, so the link now points upstream
absolutely and the suppression is deleted. The repo-root copy of the same
document was always correct and is untouched.

The systemic half: eight validate-templates.yml gates had no pre-commit
hook at all — doc-coherence, cicd-template-drift, vendored-runtime-drift,
common-utils-drift, dashboard-inventory, baselines-expiry,
test-clock-isolation and agentic-adapter-sync. There was no way to run the
full gate set locally, so the only feedback loop was push-and-read-CI,
which is how three breakages reached the remote on #84.

All eight are now pre-commit hooks, path-filtered so each fires only on
the files it guards. This is deliberately NOT a revival of the pre-push
stage retired in R5-L4: that decision was about a 60-second scaffold hook
training `--no-verify`, and all thirteen gates together run in 0.9 s,
inside the < 5 s budget this config targets. The reasoning is recorded
next to the original decision so the two are read together.

`make verify` runs the same thirteen in one command for the
sweep-before-PR case, reporting every failure rather than stopping at the
first. Slow end-to-end stays in `make smoke`.

Verified negatively: a dead path fails check_doc_path_refs through
`make verify`, and breaking the byte-identity of a vendored runbook — the
exact mistake that reached CI on #84 — is now caught by the
vendored-runtime-drift hook at commit time.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@DuqueOM
DuqueOM merged commit b73e0f4 into main Sep 4, 2026
25 checks passed
@DuqueOM
DuqueOM deleted the fix/reference-hygiene-and-verify branch September 4, 2026 20:46
DuqueOM added a commit that referenced this pull request Sep 5, 2026
…ternal links

Two of the three items from the trade-off review; the third (tfsec -> trivy)
is #90.

The two runtime-artifact entries in .doc-path-baseline.yml carried a
one-year expiry, and that was my own design mistake. Their condition never
changes, so the date could only ever be bumped — and a date that can only be
postponed trains reviewers to postpone dates, degrading the mechanism for the
entries where the deadline is the whole point.

Entries now declare an explicit kind:, and the two kinds are verified
differently because they are not the same claim.

  unimplemented    claims "we intend to build this". The only honest check on
                   an intention is a deadline, so it keeps expiry:.

  runtime-artifact claims "this resolves at runtime, and X creates it".
                   That is checkable NOW, so it carries created-by: instead.
                   The gate asserts the named creator exists and still
                   references the path.

If the skill that writes docs/concept_drift_log.md is deleted, or simply
stops mentioning it, the entry falsifies itself on the next run rather than
sitting valid until 2027. A runtime-artifact carrying an expiry is rejected
outright so the two mechanisms cannot be quietly mixed, and the
classification moves out of the reason: prose where it was an unenforced
string prefix.

The gate also now checks internal Markdown link targets. A target resolves
relative to the file containing it, not the repo root — the distinction that
produced the only broken link this repo had, which I introduced in #84 and CI
caught. Link Check is a real gate but triggers only on pull_request with
paths: **/*.md and passes check-modified-files-only on PRs, so a link that
breaks because its TARGET moved is invisible at PR time and surfaces up to
seven days later in the Monday scan, on main, blocking nobody. That is long
enough for someone to suppress it rather than fix it, which is what happened
to ../SECURITY.md until #86.

The split is by what each check needs, not by syntax: internal links are
deterministic and run in pre-commit; external URLs and site-root targets stay
with Link Check, off the merge critical path. Code spans are stripped before
link matching, since `[text](path)` in backticks is documentation about
links. Measured before enabling: one broken relative link repo-wide, and it
was that false positive in the gate's own governance doc.

Contract test at 49 cases.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant