Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,7 @@ repos:
entry: python3 scripts/check_doc_path_refs.py
language: system
pass_filenames: false
files: \.(md|txt)$|^scripts/check_doc_path_refs\.py$|^\.doc-path-baseline\.yml$
files: \.(md|txt|py|ya?ml|sh)$|(^|/)Makefile$|^\.doc-path-baseline\.yml$
- id: adr-index
name: ADR index is current
description: |
Expand Down
2 changes: 1 addition & 1 deletion .security-baselines/tfsec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ exclude:
# PSP (PodSecurityPolicy) was deprecated in K8s 1.21 and REMOVED in K8s
# 1.25. GKE clusters on REGULAR channel run 1.27+; enabling PSP would
# cause a Terraform error. Pod Security Standards (PSS) are enforced via
# namespace labels in every overlay (D-29, templates/k8s/overlays/*/
# namespace labels in every overlay (D-29, templates/service/k8s/overlays/*/
# namespace.yaml). Re-evaluate when tfsec migrates to trivy/PSS.
# expiry: 2027-01-01 reason: ADR-024 §Review — PSS via namespace labels (D-29)
- google-gke-enforce-pod-security-policy
Expand Down
68 changes: 68 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,74 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and [Sem

## [Unreleased]

### Fixed — the service CLAUDE.md described the template repo, not the service

- `templates/service/CLAUDE.md` ships into every scaffolded service and
still described the **pre-migration template repo**: a `templates/` tree
with `cicd/`, `monitoring/` and `common_utils/` that ADR-030 dissolved in
June, commands telling the adopter to scaffold a service from inside
their service, and a surface claim of "18 rules + 26 skills + 18
workflows" against a live 19/27/20.
- Rewritten to the service's own layout and command set, with the
template-repo audit history replaced by an actionable **Upstream
template** section.
- **C4 now reconciles both `CLAUDE.md` files.** The service copy was only
ever checked *inside* a generated service — by which point it had already
shipped wrong. `check_doc_coherence.py` reconciles each against the
agentic surface beside it.

### Fixed — `make scaffold-update` ran `copier update` unpinned, defaulting to `main`

- The service `Makefile` ran `copier update --trust --defaults` with no
`--vcs-ref`. `/scaffold-update` was pinned in #71; the Makefile target —
the other entry point to the same operation — was not.
- Worse, the target reused `REF`, which the `ci-green` target defines as
`REF ?= main`. A bare `make scaffold-update` therefore updated the service
from the **moving development branch**, not a release.
- Now uses `TEMPLATE_REF` with **no default**: the target refuses to run
unpinned (exit 2, without invoking copier).
- **`check_adopter_scaffold_ref.py` could not see it.** Its scan was keyed
on `SCAN_EXTENSIONS`, and a `Makefile` has no extension — while its own
comment warned that "a guard whose coverage is a literal list is only ever
as complete as the moment someone last remembered to edit it". Scoping by
extension was the same mistake. Extensionless build files are now scanned.
- The guard's failure message also described a catastrophe ADR-045 already
removed (the v1.x downgrade). Corrected: unpinned now means jumping to a
release nobody chose, with the catastrophic form prevented by the tag
namespace staying clean rather than by the command.

### Fixed — four sys.path bridges and a contract test that self-disabled

- Three bridges in `templates/service/tests/` probed directories where
`common_utils` has not lived since ADR-030 — dead safety nets that could
never fire, masked because the import resolves by another route in the
contexts CI exercises. All now use `parents[1]`, which is the service root
in **both** layouts.
- `test_drills_reproducible.py` carried an unreachable `DRILL_PYTHONPATH`
fallback for a layout split that the migration closed; replaced by an
assertion that would notice if the layouts ever diverge again.
- `test_memory_contracts.py` resolved the service root as
`REPO_ROOT/"templates"/"service"`, which does not exist inside an
adopter's service — so the invariant *"serving code must not import
`common_utils.memory_types`"* **silently skipped in exactly the
environment it protects**. Now context-adaptive, and verified to fail on
a planted violation.

### Changed — the path gate now reads code comments

- Scan extended from `.md`/`.txt` to tracked `.py`, `.yml`, `.yaml`, `.sh`
and `Makefile` comments. Measured before widening: 15 unresolved paths
across 309 code files — including `.security-baselines/tfsec.yml`
justifying three suppressed HIGH findings against a deleted directory.
All 15 resolved.
- Three filters the first run demanded, each pinned by a test: glob and
brace shorthands (`deploy-*.yml`, `deploy-{gcp,aws}.yml`) no longer
report a truncated prefix; uppercase stand-ins (`ADR-XXX.md`) and
`*.local.*` paths are not claims; and punctuation stripping no longer
scrubs `templates/templates/...` into a clean-looking one.
- String literals stay out of scope: a path built at runtime is program
logic, not a claim.

### Fixed — two baseline entries were misclassified, not unimplemented

- **`scripts/smoke_test.py` was never missing.** The release-checklist
Expand Down
38 changes: 29 additions & 9 deletions docs/governance/doc-path-references.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,16 @@

## What this gate enforces

Every repo-relative path named inside backticks in living documentation
must resolve to a file or directory that exists.
Every repo-relative path named inside backticks in living documentation —
or inside a comment in a tracked `.py`, `.yml`, `.yaml`, `.sh` or `Makefile`
— must resolve to a file or directory that exists.

The code-comment half was added on 2026-09-04. The gate shipped scanning
`.md`/`.txt` only, which left `.security-baselines/tfsec.yml` justifying
three suppressed HIGH findings against a directory ADR-030 had deleted, and
a set of test comments describing a layout that had not existed since June.
Measured before widening: **15 unresolved paths across 309 code files** —
small enough for a hard gate.

## Why this gate exists

Expand Down Expand Up @@ -90,9 +98,21 @@ that never leave the template repo resolve against the repo root only.
ellipses are illustrative, not claims about the tree.
- **Paths outside this repo's top-level directories.** A runbook naming
`src/main.py` is describing the adopter's tree, not ours.
- **Link targets in Markdown link syntax.** Only backticked code spans
are scanned. Extending to `[text](path)` is a natural follow-up and
would live in the same script.
- **Link targets in Markdown link syntax.** In documents, only backticked
code spans are scanned. Extending to `[text](path)` is a natural
follow-up and would live in the same script; the `Link Check` job in
`docs-quality.yml` covers those today.
- **String literals in code.** Only comments are scanned. A path built at
runtime is program logic, not a claim, and matching it would report
every `Path(...) / "templates"` expression.
- **Glob and brace shapes.** `deploy-*.yml` and `deploy-{gcp,aws}.yml` are
legitimate shorthand. A negative lookahead drops them rather than
reporting the truncated prefix `.../deploy-` as dead — the false
positive that this widening produced on its first run.
- **Stand-ins and files that must not exist.** Uppercase placeholders
(`ADR-XXX.md`) and any `*.local.*` path are excluded; the latter is
gitignored by contract, so a comment naming one is describing something
that is *supposed* to be absent.
- **Paths written as plain prose.** This is a deliberate convention, not
a hole: a code span asserts *this resolves today, from this document's
perspective*. A document discussing a path that has been removed, or
Expand Down Expand Up @@ -132,10 +152,10 @@ stopped `f219895`.

## Operational cost

593 tracked documents, one regex pass each, plus a filesystem `exists()`
per candidate token. Tens of milliseconds on a CI runner, dominated by
Python startup. The pre-commit hook fires only on `*.md` / `*.txt`
changes and on edits to the script or baseline.
595 documents plus 330 code files, one regex pass each, and a filesystem
`exists()` per candidate token. Tens of milliseconds on a CI runner,
dominated by Python startup. The pre-commit hook fires on `*.md`, `*.txt`,
`*.py`, `*.yml`, `*.yaml`, `*.sh`, any `Makefile`, and the baseline.

## Revisit triggers

Expand Down
2 changes: 1 addition & 1 deletion examples/minimal/train.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@
# every run and dilute the signal of the gate.
#
# This is an EXPLICIT exception, not a contradiction. Any scaffolded
# service (`templates/service/training/train.py`) inherits the 0.80
# service (`templates/service/src/<service_slug>/training/train.py`) inherits the 0.80
# default; only this small-N example uses 0.70.
DEMO_FAIRNESS_THRESHOLD = 0.70 # demo only — see comment above
PRODUCTION_FAIRNESS_THRESHOLD = 0.80 # what real services use (D-12)
Expand Down
18 changes: 15 additions & 3 deletions scripts/check_adopter_scaffold_ref.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,12 @@
HISTORICAL_FILES = ("CHANGELOG.md", "VALIDATION_LOG.md", "MIGRATION.md")

SCAN_EXTENSIONS = (".md", ".yml", ".yaml", ".sh")
# Extensionless files that still hold executable instructions. A Makefile has
# no suffix, so the extension filter below skipped it — and that is exactly
# where `make scaffold-update` lived, unpinned, while this guard passed. The
# comment above says coverage must not be a literal list; scoping by
# extension was the same mistake wearing a different hat.
SCAN_FILENAMES = ("Makefile", "makefile", "GNUmakefile", "Justfile", "justfile")

SCAFFOLD_CMD = re.compile(r"copier\s+copy\s+(?P<args>[^\n]*?)https://github\.com/[^\s]+")
# `copier update` is the DESTRUCTIVE path: unpinned it does not merely
Expand Down Expand Up @@ -122,7 +128,9 @@ def main() -> int:
# the target release varies so a placeholder ref is legitimate.
update_cmds_seen = 0
for path in sorted(REPO_ROOT.rglob("*")):
if not path.is_file() or path.suffix not in SCAN_EXTENSIONS:
if not path.is_file():
continue
if path.suffix not in SCAN_EXTENSIONS and path.name not in SCAN_FILENAMES:
continue
rel = path.relative_to(REPO_ROOT).as_posix()
if rel.startswith(".git/") or "__pycache__" in rel:
Expand Down Expand Up @@ -152,8 +160,12 @@ def main() -> int:
if match and "--vcs-ref" not in match.group("args"):
problems.append(
f"{rel}:{lineno}: executable `copier update` without --vcs-ref. "
f"Unpinned it DOWNGRADES the service to a frozen v1.x snapshot and "
f"deletes .copier-answers.yml, removing the update path itself."
f"Copier resolves an unpinned source to the highest-sorting tag, so the "
f"service jumps to a release nobody chose — including across a major. "
f"ADR-045 moved the frozen v1.x snapshots out of the version namespace, "
f"which removed the catastrophic form of this (627 files -> 435, "
f".copier-answers.yml deleted); the safety now rests entirely on that "
f"namespace staying clean. Pin the ref instead of relying on it."
)

if not update_cmds_seen:
Expand Down
48 changes: 36 additions & 12 deletions scripts/check_doc_coherence.py
Original file line number Diff line number Diff line change
Expand Up @@ -212,29 +212,53 @@ def check_anti_pattern_count() -> list[str]:
return problems


def check_surface_counts() -> list[str]:
"""C4 — live agentic surface counts must match CLAUDE.md's claim."""
claude = _read(CLAUDE)
if claude is None or not RULES_DIR.is_dir():
_SURFACE_CLAIM = re.compile(r"(\d+)\s*rules\s*\+\s*(\d+)\s*skills\s*\+\s*(\d+)\s*workflows")


def _reconcile_surface(claude_path: Path, rules_dir: Path, label: str) -> list[str]:
"""Compare one CLAUDE.md's surface claim against the tree beside it."""
claude = _read(claude_path)
if claude is None or not rules_dir.is_dir():
return [] # no agentic surface / no CLAUDE.md here → nothing to reconcile

rules = len(list(RULES_DIR.glob("*.md")))
skills = len(list(SKILLS_DIR.glob("*/SKILL.md"))) if SKILLS_DIR.is_dir() else 0
workflows = len(list(WORKFLOWS_DIR.glob("*.md"))) if WORKFLOWS_DIR.is_dir() else 0
skills_dir = rules_dir.parent / "skills"
workflows_dir = rules_dir.parent / "workflows"
actual = (
len(list(rules_dir.glob("*.md"))),
len(list(skills_dir.glob("*/SKILL.md"))) if skills_dir.is_dir() else 0,
len(list(workflows_dir.glob("*.md"))) if workflows_dir.is_dir() else 0,
)

m = re.search(r"(\d+)\s*rules\s*\+\s*(\d+)\s*skills\s*\+\s*(\d+)\s*workflows", claude)
m = _SURFACE_CLAIM.search(claude)
if not m:
return [] # CLAUDE.md doesn't claim a surface count → nothing to verify
return [] # doesn't claim a surface count → nothing to verify
claimed = (int(m.group(1)), int(m.group(2)), int(m.group(3)))
actual = (rules, skills, workflows)
if claimed != actual:
return [
f"CLAUDE.md claims {claimed[0]} rules + {claimed[1]} skills + {claimed[2]} workflows; "
f"agentic/ has {actual[0]} rules + {actual[1]} skills + {actual[2]} workflows."
f"{label} claims {claimed[0]} rules + {claimed[1]} skills + {claimed[2]} workflows; "
f"the surface beside it has {actual[0]} rules + {actual[1]} skills + {actual[2]} workflows."
]
return []


def check_surface_counts() -> list[str]:
"""C4 — live agentic surface counts must match CLAUDE.md's claim.

Two CLAUDE.md files carry a surface claim, and only one of them used to
be checked. ``templates/service/CLAUDE.md`` ships into every scaffolded
service; inside that service this same script reconciles it correctly,
but by then it has already shipped. It sat at "18 rules + 26 skills +
18 workflows" against a live 19/27/20 because in the template repo
nothing looked at it. Both are reconciled here, each against the
surface that sits beside it.
"""
problems = _reconcile_surface(CLAUDE, RULES_DIR, "CLAUDE.md")
service_claude = REPO_ROOT / "templates" / "service" / "CLAUDE.md"
service_rules = REPO_ROOT / "templates" / "service" / "agentic" / "rules"
problems += _reconcile_surface(service_claude, service_rules, "templates/service/CLAUDE.md")
return problems


def check_adr_traceability() -> list[str]:
"""C5 — no silent ADR gaps; every number ≤ max has a file or a tombstone."""
if not ADR_DIR.is_dir():
Expand Down
67 changes: 63 additions & 4 deletions scripts/check_doc_path_refs.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#!/usr/bin/env python3
"""Contract: a repo-relative path named in living documentation must resolve.
"""Contract: a repo-relative path named in living documentation — or in a
code comment — must resolve.

Why this script exists
----------------------
Expand Down Expand Up @@ -138,12 +139,39 @@

_BACKTICKED = re.compile(r"`([^`\n]+)`")

# Code files carry the same claims in comments. The scan was `.md`/`.txt`
# only at first, which left `.security-baselines/tfsec.yml` justifying three
# HIGH suppressions against a directory ADR-030 had deleted, and a handful of
# test comments describing a layout that no longer existed. Measured before
# widening: 15 unresolved paths across 309 code files — small enough for a
# hard gate, unlike a naive scan of every string literal.
_CODE_SUFFIXES = (".py", ".yml", ".yaml", ".sh")
_CODE_FILENAMES = ("Makefile", "makefile", "GNUmakefile")
# A path-shaped token in a comment. Anchored on the comment marker so code
# that legitimately builds a path at runtime is not mistaken for a claim.
# The negative lookahead drops tokens that continue into a glob or brace set
# (`deploy-*.yml`, `deploy-{gcp,aws}.yml`): the character class stops at the
# metacharacter, and without the lookahead the truncated prefix
# `.../workflows/deploy-` would be reported as a dead path.
_COMMENT_PATH = re.compile(
r"(?:^|\s|\(|`)((?:templates|scripts|docs|agentic|examples|\.github)/[A-Za-z0-9_./-]+)(?![A-Za-z0-9_./-]*[{*?\[])"
)

# Placeholders that survive `_is_literal_path` because they contain no
# rejected character: `ADR-XXX.md`, `v0.NN.0`. They are stand-ins, not claims.
_UPPER_PLACEHOLDER = re.compile(r"(?:^|[-_/.])(?:XXX+|NNN?|YYYY|MM|DD)(?:[-_/.]|$)")


def _tracked_docs() -> list[str]:
out = subprocess.run(["git", "ls-files", "-z"], cwd=REPO_ROOT, capture_output=True, text=True, check=True).stdout
return [p for p in out.split("\0") if p.endswith((".md", ".txt"))]


def _tracked_code() -> list[str]:
out = subprocess.run(["git", "ls-files", "-z"], cwd=REPO_ROOT, capture_output=True, text=True, check=True).stdout
return [p for p in out.split("\0") if p and (p.endswith(_CODE_SUFFIXES) or Path(p).name in _CODE_FILENAMES)]


def _is_frozen(path: str) -> bool:
return path.startswith(FROZEN_PREFIXES) or path in FROZEN_FILES

Expand All @@ -155,7 +183,11 @@ def _is_dual(path: str) -> bool:
def _is_literal_path(token: str) -> bool:
if not token.startswith(REPO_PREFIXES):
return False
if "..." in token:
if "..." in token or _UPPER_PLACEHOLDER.search(token):
return False
# `*.local.*` files are gitignored by contract — a comment naming one is
# describing something that must NOT exist, not claiming that it does.
if ".local." in token:
return False
return not (_NON_LITERAL & set(token))

Expand All @@ -182,6 +214,33 @@ def collect_unresolved() -> dict[str, set[str]]:
if not _is_literal_path(token) or _resolves(token, dual):
continue
found.setdefault(token, set()).add(doc)

for src in _tracked_code():
if _is_frozen(src):
continue
path = REPO_ROOT / src
if not path.is_file(): # Copier-token filenames are not on disk
continue
try:
text = path.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
continue
dual = _is_dual(src)
for line in text.splitlines():
stripped = line.lstrip()
comment = None
if stripped.startswith(("#", "//")):
comment = stripped
elif " # " in line:
comment = line.split(" # ", 1)[1]
if comment is None:
continue
for match in _COMMENT_PATH.finditer(comment):
raw = match.group(1).strip()
token = raw if raw.endswith("...") else raw.rstrip(",.;:").rstrip("/")
if not _is_literal_path(token) or _resolves(token, dual):
continue
found.setdefault(token, set()).add(src)
return found


Expand Down Expand Up @@ -287,9 +346,9 @@ def main() -> int:
if new or expired or obsolete:
return 1

checked = len(_tracked_docs())
docs_n, code_n = len(_tracked_docs()), len(_tracked_code())
print(
f"[doc-path-refs] OK — {checked} tracked documents scanned, "
f"[doc-path-refs] OK — {docs_n} documents + {code_n} code files scanned, "
f"every repo path reference resolves ({len(baseline)} baselined, all in-date)."
)
return 0
Expand Down
2 changes: 1 addition & 1 deletion scripts/check_service_adr_references.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@
# without saying where to look is not a resolution.
UPSTREAM_MARKER = "docs/decisions"

# Placeholder used in templates/examples, never a real ADR.
# Placeholder used in template example content, never a real ADR.
PLACEHOLDER_IDS = {"099"}


Expand Down
2 changes: 1 addition & 1 deletion templates/k8s/policies/pod-security-standards.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ metadata:
# * capabilities.drop: [ALL]
# * seccompProfile.type: RuntimeDefault
# * no hostPath / hostNetwork / hostPID / hostIPC
# See templates/k8s/base/deployment.yaml for the securityContext block.
# See templates/service/k8s/base/deployment.yaml for the securityContext block.
apiVersion: v1
kind: Namespace
metadata:
Expand Down
Loading
Loading