Skip to content

feat(ci): verify baseline claims instead of dating them, and check internal links - #91

Merged
DuqueOM merged 1 commit into
mainfrom
feat/self-verifying-baseline-and-links
Sep 5, 2026
Merged

feat(ci): verify baseline claims instead of dating them, and check internal links#91
DuqueOM merged 1 commit into
mainfrom
feat/self-verifying-baseline-and-links

Conversation

@DuqueOM

@DuqueOM DuqueOM commented Sep 5, 2026

Copy link
Copy Markdown
Owner

The second and third items from the trade-off review. (The first, tfsec → Trivy, is #90.)

1. Baseline claims that can be verified now are verified, not dated

The two runtime-artifact entries carried a one-year expiry. That was my own design mistake, and it is worth naming precisely: their condition never changes, so the date could only ever be bumped. A date that can only be postponed trains reviewers to postpone dates — which degrades the mechanism for the entries where the deadline is the point.

Entries now declare an explicit kind:, and the two kinds are verified differently because they are not the same claim:

Kind The claim How it is checked
unimplemented "we intend to build this" expiry: — a deadline is the only honest check on an intention
runtime-artifact "this resolves at runtime, and X creates it" created-by: — the gate asserts that file exists and still names the path

So if the skill that writes docs/concept_drift_log.md is deleted, or simply stops mentioning it, the entry falsifies itself on the next run instead of sitting valid until 2027. That is the difference between governance and a calendar.

A runtime-artifact carrying an expiry: is rejected outright, so the two mechanisms cannot be quietly mixed. The classification also moves out of the reason: prose, where it had been a string prefix nothing enforced.

Test Result
created-by names a file that does not exist ✅ fails, names it
created-by exists but no longer references the path ✅ fails: "nothing is producing it at runtime any more"
runtime-artifact with an expiry: ✅ exit 2, kinds cannot be mixed
Clean tree ✅ exit 0

2. Internal Markdown link targets

A link target resolves relative to the file that contains it, not the repo root. That distinction produced the only broken link this repo had — templates/service/README.md pointing at templates/service/docs/CCDS_MAPPING.md, which from inside that directory means a doubled path. I introduced it in #84 by rewriting path text without accounting for relativity, and CI caught it. Which is the argument for catching it locally.

Link Check is a real gate (only Markdown Lint is warn-only), but its coverage has a shape worth stating:

  • it triggers only on pull_request with paths: **/*.md, so a PR that moves or deletes a file without touching a .md never runs it;
  • on a PR it passes check-modified-files-only: yes, so it sees only changed files.

A link breaks when its target moves, not when the linking file changes. That case is invisible at PR time and surfaces up to seven days later in the Monday 06:00 UTC scan — on main, as a red scheduled run that blocks nobody. Long enough for someone to suppress it instead of fixing it, which is exactly what happened to ../SECURITY.md, silenced by a dedicated ignorePatterns entry until #86.

The split is by what each check needs, not by syntax:

Checked by Why
Internal relative links this gate deterministic, no network, runs in pre-commit
http(s)://, mailto:, /-rooted Link Check needs the network; latency and third-party flakiness are tolerable off the merge critical path

Code spans are stripped before link matching — [text](path) inside backticks is documentation about links, and it was the single false positive the first run produced. Typographic now counts as an ellipsis alongside ....

Measured before enabling: one broken relative link across the whole repo, and it was that false positive in the gate's own governance doc. Hard gate from day one, no baseline needed.

Test Result
Plant the exact #84 bug ✅ fails, names both file and target
External URL, #anchor, /root link ✅ ignored
[text](path) inside a code span ✅ not treated as a link

Evidence — Schema / Contract Test

templates/tests/unit/test_doc_path_refs_contract.py, now 49 cases. Two existing tests were migrated to the new baseline shape rather than deleted: the expired-entry case is now constructed, because the repo no longer has any unimplemented entries — every promise has been built or corrected.

Evidence — Real Execution Output

$ python3 scripts/check_doc_path_refs.py
[doc-path-refs] OK — 595 documents + 330 code files scanned, every repo path
reference resolves (2 baselined, all justified).

$ # creator no longer names the path
::error::baseline entry 'docs/concept_drift_log.md': its declared creator
'agentic/skills/rollback/SKILL.md' no longer references that path, so nothing
is producing it at runtime any more

$ # runtime-artifact given a date
::error::baseline entry 'docs/concept_drift_log.md' is a runtime-artifact and
carries an expiry: — its claim is verified against created-by, not a date

$ # the #84 link bug, replanted
  - templates/service/README.md -> templates/service/docs/CCDS_MAPPING.md

$ python3 -m pytest templates/tests/unit/test_doc_path_refs_contract.py -q
49 passed

$ make verify
✓ every gate green — safe to push

Evidence — CI Run Link

See the checks on this PR.

🤖 Generated with Claude Code

…ternal links

Two of the three items from the trade-off review; the third (tfsec -> trivy)
is #90.

The two runtime-artifact entries in .doc-path-baseline.yml carried a
one-year expiry, and that was my own design mistake. Their condition never
changes, so the date could only ever be bumped — and a date that can only be
postponed trains reviewers to postpone dates, degrading the mechanism for the
entries where the deadline is the whole point.

Entries now declare an explicit kind:, and the two kinds are verified
differently because they are not the same claim.

  unimplemented    claims "we intend to build this". The only honest check on
                   an intention is a deadline, so it keeps expiry:.

  runtime-artifact claims "this resolves at runtime, and X creates it".
                   That is checkable NOW, so it carries created-by: instead.
                   The gate asserts the named creator exists and still
                   references the path.

If the skill that writes docs/concept_drift_log.md is deleted, or simply
stops mentioning it, the entry falsifies itself on the next run rather than
sitting valid until 2027. A runtime-artifact carrying an expiry is rejected
outright so the two mechanisms cannot be quietly mixed, and the
classification moves out of the reason: prose where it was an unenforced
string prefix.

The gate also now checks internal Markdown link targets. A target resolves
relative to the file containing it, not the repo root — the distinction that
produced the only broken link this repo had, which I introduced in #84 and CI
caught. Link Check is a real gate but triggers only on pull_request with
paths: **/*.md and passes check-modified-files-only on PRs, so a link that
breaks because its TARGET moved is invisible at PR time and surfaces up to
seven days later in the Monday scan, on main, blocking nobody. That is long
enough for someone to suppress it rather than fix it, which is what happened
to ../SECURITY.md until #86.

The split is by what each check needs, not by syntax: internal links are
deterministic and run in pre-commit; external URLs and site-root targets stay
with Link Check, off the merge critical path. Code spans are stripped before
link matching, since `[text](path)` in backticks is documentation about
links. Measured before enabling: one broken relative link repo-wide, and it
was that false positive in the gate's own governance doc.

Contract test at 49 cases.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@DuqueOM
DuqueOM force-pushed the feat/self-verifying-baseline-and-links branch from 3c44926 to 6897e31 Compare September 5, 2026 17:39
@DuqueOM
DuqueOM merged commit 1247267 into main Sep 5, 2026
25 checks passed
@DuqueOM
DuqueOM deleted the feat/self-verifying-baseline-and-links branch September 5, 2026 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant