Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .devin/skills/release-checklist/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ kill %1

For a multi-service / multi-cloud sweep, drive the loop from a
runbook page or a CI workflow-dispatch — the template does not ship
a `scripts/smoke_test.py` script (would compete with `deploy-common.yml`
a scripts/smoke_test.py script (would compete with `deploy-common.yml`
as SSOT).

## Rollback Plan
Expand Down
6 changes: 3 additions & 3 deletions .devin/workflows/load-test.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Determine which service and environment to test:

## 2. Configure Locust

Verify `scripts/load_test_services.py` has the correct endpoints and payloads for the target service.
Verify `tests/load_test.py` has the correct endpoints and payloads for the target service.

Before starting load, verify the FastAPI contract with the same auth
posture production clients use:
Expand All @@ -40,7 +40,7 @@ running Locust. Do not load-test a payload that returns 422.
## 3. Run Load Test (GCP)

```bash
locust -f scripts/load_test_services.py \
locust -f tests/load_test.py \
--host http://${GCP_ENDPOINT} \
--users 100 \
--spawn-rate 10 \
Expand All @@ -52,7 +52,7 @@ locust -f scripts/load_test_services.py \
## 4. Run Load Test (AWS)

```bash
locust -f scripts/load_test_services.py \
locust -f tests/load_test.py \
--host http://${AWS_ENDPOINT} \
--users 100 \
--spawn-rate 10 \
Expand Down
8 changes: 0 additions & 8 deletions .doc-path-baseline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,6 @@ entries:
expiry: 2027-09-04

# --- unimplemented --------------------------------------------------------
- path: scripts/smoke_test.py
reason: unimplemented — the release-checklist skill instructs running it; no such script exists in either root
expiry: 2026-12-01

- path: scripts/load_test_services.py
reason: unimplemented — the load-test workflow instructs running it; no such script exists in either root
expiry: 2026-12-01

- path: templates/service/tests/test_context_files_hygiene.py
reason: unimplemented — docs/agentic/contextualization.md cites it as the enforcing test; the test was never written
expiry: 2026-12-01
Expand Down
3 changes: 1 addition & 2 deletions .github/markdown-link-check.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@
{ "pattern": "^https://github\\.com/\\{ORG\\}" },
{ "pattern": "youtu\\.be/XXX" },
{ "pattern": "example\\.com" },
{ "pattern": "^https://ai\\.google\\.dev/" },
{ "pattern": "^\\.\\./SECURITY\\.md$" }
{ "pattern": "^https://ai\\.google\\.dev/" }
],
"replacementPatterns": [
{
Expand Down
77 changes: 77 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,19 @@

# Single-stage install — pre-push stage was retired in R5-L4 (May 2026);
# `make smoke` + CI pr-smoke-lane.yml replace it.
#
# 2026-09-04: eight validate-templates.yml gates had no local hook at all
# (doc-coherence, cicd-template-drift, vendored-runtime-drift,
# common-utils-drift, dashboard-inventory, baselines-expiry,
# test-clock-isolation, agentic-adapter-sync). That gap — not the missing
# pre-push stage — is what let three breakages reach the remote in one
# afternoon. They are added below as pre-commit hooks rather than by
# reviving pre-push: all thirteen gates together run in 0.9 s, well inside
# the < 5 s pre-commit budget this file targets, so R5-L4's reasoning
# (a 60 s hook trains `--no-verify`) does not apply to them.
#
# `make verify` runs the same set in one command, for the sweep-before-PR
# case where you want every gate regardless of which files you touched.
default_install_hook_types: [pre-commit]

# fail_fast: false is the DEFAULT (shown explicit for intent) — we
Expand Down Expand Up @@ -195,6 +208,70 @@ repos:
language: system
pass_filenames: false
files: ^(VERSION|README\.md|QUICK_START\.md|docs/TUTORIAL\.md|docs/PROGRESSION\.md|scripts/check_adopter_scaffold_ref\.py)$
- id: doc-coherence
name: documentation coherence (rule 16, ADR-031)
description: |
C1-C7: VERSION vs CHANGELOG, llms.txt version, anti-pattern
counts, agentic surface counts, ADR traceability, release-note
existence, English-only + private-repo guard.
entry: python3 scripts/check_doc_coherence.py
language: system
pass_filenames: false
files: ^(VERSION|CHANGELOG\.md|llms\.txt|AGENTS\.md|README\.md|CLAUDE\.md|docs/decisions/|releases/|agentic/|scripts/check_doc_coherence\.py)
- id: cicd-template-drift
name: template action versions match runtime
description: |
Dependabot only scans .github/workflows/, so action references
under templates/ age silently and ship to every scaffolded
service. See docs/governance/cicd-templates-drift.md.
entry: python3 scripts/check_cicd_template_drift.py
language: system
pass_filenames: false
files: \.ya?ml$|^scripts/check_cicd_template_drift\.py$
- id: vendored-runtime-drift
name: vendored copies match their canonical originals
description: |
Files vendored byte-identical into templates/service/ (AGENTS.md,
the agentic surface, several scripts and runbooks) must not
diverge. Editing one side only is anti-pattern Q-06.
entry: python3 scripts/check_vendored_runtime_drift.py
language: system
pass_filenames: false
files: ^(scripts/|docs/runbooks/|agentic/|templates/service/|templates/config/|AGENTS\.md|AGENT_CONTEXT\.md)
- id: common-utils-drift
name: common_utils distribution contract (ADR-025)
entry: python3 scripts/check_common_utils_drift.py
language: system
pass_filenames: false
files: common_utils/
- id: dashboard-inventory
name: Grafana dashboard inventory is complete
entry: python3 scripts/check_dashboard_inventory.py
language: system
pass_filenames: false
files: ^(templates/service/monitoring/|docs/observability/|scripts/check_dashboard_inventory\.py)
- id: baselines-expiry
name: security baseline entries are dated and in-date
entry: python3 scripts/check_baselines_expiry.py
language: system
pass_filenames: false
files: ^(\.security-baselines/|scripts/check_baselines_expiry\.py)
- id: test-clock-isolation
name: tests do not depend on the wall clock
entry: python3 scripts/check_test_clock_isolation.py
language: system
pass_filenames: false
files: (^|/)tests?/.*\.py$|^scripts/check_test_clock_isolation\.py$
- id: agentic-adapter-sync
name: generated adapter surfaces are in sync
description: |
.devin/, .cursor/, .claude/ and .codex/ are generated from
agentic/ by sync_agentic_adapters.py. Editing agentic/ without
re-running the sync ships a stale adapter surface.
entry: python3 scripts/sync_agentic_adapters.py --check
language: system
pass_filenames: false
files: ^(agentic/|\.devin/|\.cursor/|\.claude/|\.codex/|templates/config/agentic_manifest\.yaml)
- id: doc-path-refs
name: documentation path references resolve
description: |
Expand Down
52 changes: 52 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,58 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and [Sem

## [Unreleased]

### Fixed — two baseline entries were misclassified, not unimplemented

- **`scripts/smoke_test.py` was never missing.** The release-checklist
skill says in so many words that *"the template does not ship a
`scripts/smoke_test.py` script (would compete with `deploy-common.yml`
as SSOT)"*. It is a deliberate, documented non-existence. The path
reference gate flagged it only because the sentence wrapped it in a code
span, and the baseline then recorded a correct design decision as a
defect. Rewritten as plain prose per the convention in
`docs/governance/doc-path-references.md`: a code span asserts *this
resolves*, so a path you are talking about is not one.
- **`scripts/load_test_services.py` was a wrong path, not a missing
file.** The artefact exists as `templates/service/tests/load_test.py`,
run via `make load-test`. The load-test workflow gave three
`locust -f scripts/load_test_services.py` commands that fail as written.
Corrected to `tests/load_test.py`, the form that is right from inside a
generated service.
- Baseline down from 7 entries to 5.

### Fixed — a broken link was suppressed rather than repaired

- `templates/service/docs/ADOPTION.md` linked `[SECURITY.md](../SECURITY.md)`,
which resolves to `templates/service/SECURITY.md` in this repo and to the
service root in a generated one. Neither exists.
- It never failed CI because `.github/markdown-link-check.json` carried a
dedicated `ignorePatterns` entry, `^\.\./SECURITY\.md$`, silencing that
exact link. The sentence is about *this template's* disclosure SLA, so
the link now points at the upstream `SECURITY.md` absolutely, and the
suppression is deleted. The repo-root copy of the same document was
always correct and is untouched.

### Added — `make verify`, and the eight CI gates that had no local hook

- Eight jobs in `validate-templates.yml` had no pre-commit hook at all:
doc-coherence, cicd-template-drift, vendored-runtime-drift,
common-utils-drift, dashboard-inventory, baselines-expiry,
test-clock-isolation and agentic-adapter-sync. There was no way to run
the full gate set locally, so the only feedback loop was push-and-read-CI.
- All eight are now pre-commit hooks, path-filtered so each fires only on
the files it guards. This is deliberately **not** a revival of the
pre-push stage retired in R5-L4: that decision was about a 60-second
scaffold hook training `--no-verify`, and all thirteen gates together
run in 0.9 s — inside the < 5 s budget this config targets. The
reasoning is recorded next to the original decision.
- `make verify` runs the same thirteen in one command for the
sweep-before-PR case, reporting **every** failure rather than stopping at
the first. Slow end-to-end stays in `make smoke`.
- Verified negatively: introducing a dead path fails `check_doc_path_refs`
through `make verify`, and breaking the byte-identity of a vendored
runbook — the exact mistake that reached CI on #84 — is now caught by the
`vendored-runtime-drift` hook at commit time.

### Fixed — the Copier migration relocated the template tree and the prose never followed

- **Root cause, traced.** ADR-030's migration (commit `fe89e92`,
Expand Down
56 changes: 55 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# make test-examples # Run example regression tests

.PHONY: help install-dev lint-all format-all validate-templates \
validate-agentic bootstrap smoke \
validate-agentic bootstrap smoke verify \
mcp-check mcp-doctor mcp-render-docs \
report-validate report-example \
demo-minimal test-examples clean
Expand Down Expand Up @@ -155,6 +155,60 @@ eda-validate: ## Validate EDA pipeline: syntax + run against example dataset
validate-templates: lint-all validate-k8s validate-agentic test-scaffold eda-validate ## Validate all templates (lint + K8s + agentic + scaffold + EDA)
@echo "$(GREEN)✓ All templates validated$(NC)"

# ═══════════════════════════════════════════════
# Gate suite — the CI contract, runnable locally
# ═══════════════════════════════════════════════
# Every check below is a job in validate-templates.yml. Before this target
# existed the only way to run the full set was to push and read CI, which
# is how three separate breakages in one afternoon (Link Check, vendored
# byte-identity, evidence-check) reached the remote instead of dying on the
# contributor's machine.
#
# Deliberately excludes the slow ones: `make smoke` (scaffolder E2E, ~2.5
# min) and the pytest suites. This target is the fast contract — every gate
# here is sub-second — so it is cheap enough to sit on pre-push.
#
# Runs ALL gates and reports every failure, rather than stopping at the
# first. A contributor who broke three things should learn that in one run.

GATES := \
check_doc_coherence \
check_doc_path_refs \
check_cicd_template_drift \
check_vendored_runtime_drift \
check_common_utils_drift \
check_dashboard_inventory \
check_baselines_expiry \
check_gitleaks_pin \
check_adopter_scaffold_ref \
check_service_adr_references \
check_test_clock_isolation \
validate_agentic

verify: ## Run every fast CI gate locally (the pre-push contract). Slow E2E lives in `make smoke`.
@echo "$(GREEN)Running the CI gate suite...$(NC)"
@failed=""; \
for gate in $(GATES); do \
if python3 scripts/$$gate.py >/tmp/$$gate.out 2>&1; then \
printf ' %-34s PASS\n' "$$gate"; \
else \
printf ' %-34s FAIL\n' "$$gate"; \
failed="$$failed $$gate"; \
fi; \
done; \
if python3 scripts/sync_agentic_adapters.py --check >/tmp/adapter_sync.out 2>&1; then \
printf ' %-34s PASS\n' "adapter-sync"; \
else \
printf ' %-34s FAIL\n' "adapter-sync"; \
failed="$$failed sync_agentic_adapters"; \
fi; \
if [ -n "$$failed" ]; then \
echo "$(RED)✗ failing gates:$$failed$(NC)"; \
for g in $$failed; do echo "--- $$g ---"; tail -20 /tmp/$$g.out; done; \
exit 1; \
fi; \
echo "$(GREEN)✓ every gate green — safe to push$(NC)"

# ═══════════════════════════════════════════════
# Example (Fraud Detection)
# ═══════════════════════════════════════════════
Expand Down
2 changes: 1 addition & 1 deletion agentic/skills/release-checklist/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ kill %1

For a multi-service / multi-cloud sweep, drive the loop from a
runbook page or a CI workflow-dispatch — the template does not ship
a `scripts/smoke_test.py` script (would compete with `deploy-common.yml`
a scripts/smoke_test.py script (would compete with `deploy-common.yml`
as SSOT).

## Rollback Plan
Expand Down
6 changes: 3 additions & 3 deletions agentic/workflows/load-test.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Determine which service and environment to test:

## 2. Configure Locust

Verify `scripts/load_test_services.py` has the correct endpoints and payloads for the target service.
Verify `tests/load_test.py` has the correct endpoints and payloads for the target service.

Before starting load, verify the FastAPI contract with the same auth
posture production clients use:
Expand All @@ -40,7 +40,7 @@ running Locust. Do not load-test a payload that returns 422.
## 3. Run Load Test (GCP)

```bash
locust -f scripts/load_test_services.py \
locust -f tests/load_test.py \
--host http://${GCP_ENDPOINT} \
--users 100 \
--spawn-rate 10 \
Expand All @@ -52,7 +52,7 @@ locust -f scripts/load_test_services.py \
## 4. Run Load Test (AWS)

```bash
locust -f scripts/load_test_services.py \
locust -f tests/load_test.py \
--host http://${AWS_ENDPOINT} \
--users 100 \
--spawn-rate 10 \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ kill %1

For a multi-service / multi-cloud sweep, drive the loop from a
runbook page or a CI workflow-dispatch — the template does not ship
a `scripts/smoke_test.py` script (would compete with `deploy-common.yml`
a scripts/smoke_test.py script (would compete with `deploy-common.yml`
as SSOT).

## Rollback Plan
Expand Down
6 changes: 3 additions & 3 deletions templates/service/agentic/workflows/load-test.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Determine which service and environment to test:

## 2. Configure Locust

Verify `scripts/load_test_services.py` has the correct endpoints and payloads for the target service.
Verify `tests/load_test.py` has the correct endpoints and payloads for the target service.

Before starting load, verify the FastAPI contract with the same auth
posture production clients use:
Expand All @@ -40,7 +40,7 @@ running Locust. Do not load-test a payload that returns 422.
## 3. Run Load Test (GCP)

```bash
locust -f scripts/load_test_services.py \
locust -f tests/load_test.py \
--host http://${GCP_ENDPOINT} \
--users 100 \
--spawn-rate 10 \
Expand All @@ -52,7 +52,7 @@ locust -f scripts/load_test_services.py \
## 4. Run Load Test (AWS)

```bash
locust -f scripts/load_test_services.py \
locust -f tests/load_test.py \
--host http://${AWS_ENDPOINT} \
--users 100 \
--spawn-rate 10 \
Expand Down
3 changes: 2 additions & 1 deletion templates/service/docs/ADOPTION.md
Original file line number Diff line number Diff line change
Expand Up @@ -283,7 +283,8 @@ The template explicitly does **not** address:

## 7. Disclosure SLA (R4 audit M4 clarification)

The vulnerability response timeline in [`SECURITY.md`](../SECURITY.md)
The vulnerability response timeline in
[`SECURITY.md`](https://github.com/DuqueOM/ml-service-template/blob/main/SECURITY.md)
is **operational** for this template:

- The maintainer commits to the response times listed (Critical 48h /
Expand Down
Loading