Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .changelog/unreleased/99-substance-only-contract.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- **Substance-only PR verification contract.** `scripts/pr-contract.mjs` now counts any non-empty bullet or checkbox in `## Verification` as a verification item (`missing_verification_item` replaces `missing_checked_verification`); unchecked boxes and missing evidence blocks are advisories instead of failures; heading presence/order is advisory while section substance stays enforced; placeholder and generic "tests pass"/"CI green" claims still hard-fail; fenced content inside the Verification section is no longer misread as items; advisories now surface in the job summary on success. Templates (`.github/PULL_REQUEST_TEMPLATE.md`, `contracts/pr-template.md`), README contract description, and pr-policy header docs reconciled; retired persona language removed from AGENTS.md. Caller inputs unchanged — consumers pick this up at the next `@v1` retag. (#99)
28 changes: 18 additions & 10 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,17 @@
<!--
PULL_REQUEST_TEMPLATE for ArchonVII/repo-template.

repo-template has no language-runtime CI surface — verification commands
here target the actionlint workflow and git-hook shell parsing.

Non-draft PRs are validated by the shared PR contract. Replace every
placeholder before marking the PR ready for review.
PULL_REQUEST_TEMPLATE for ArchonVII/github-workflows.

This repo hosts the reusable workflow library. Its own CI surface is the
Node/vitest self-test (scripts/** changes) and the Go-lane self-test —
verification commands here usually cite `npm test` runs and actionlint.

Non-draft PRs are validated by the shared PR contract (substance-only since
#99): the Verification section needs at least one substantive item — a plain
bullet or a checkbox — recording what was actually run or checked. Evidence
blocks are the recommended shape and are validated when present, but their
absence is advisory, not blocking. Placeholders (TODO/TBD) and generic
claims ("tests pass", "CI green") still hard-fail. Replace every placeholder
before marking the PR ready for review.
-->

## Summary
Expand All @@ -14,7 +20,7 @@ TODO: What changed and why?

## Verification

- [ ] TODO: Replace with an exact command, CI check, or manual smoke test.
- TODO: Replace with the exact command, CI check, or manual smoke test you ran and its result.

```evidence
command: TODO
Expand All @@ -25,9 +31,9 @@ TODO: What changed and why?

### Verification Notes

Each checked box below must be backed by exactly one fenced `evidence` block. The PR-policy parser (warning-only in Phase 1, will hard-fail in Phase 2+) reads them.
Record concrete detail: command output, CI check names, or manual smoke-test notes. Generic "tests passed / CI green" statements are rejected.

Required fields: `command`, `location` (one of `local` / `ci` / `manual`), `result`, `timestamp`. Optional: `check` (used when `location: ci` and the check-run name differs from the command).
Evidence-block fields when you use one: `command`, `location` (one of `local` / `ci` / `manual`), `result`, `timestamp`. Optional: `check` (used when `location: ci` and the check-run name differs from the command).

TODO: Summarize the exact verification evidence and any manual review.

Expand All @@ -37,6 +43,8 @@ TODO: Record the changelog fragment, direct CHANGELOG edit, docs update, or no-c

Plan/status artifacts: TODO: closed, narrowed to remaining scoped work, marked deprecated/superseded with the current source of truth, or not applicable because none were created or used by this lane.

Owner decisions this lane: appended / none

## Linked Issue

TODO: Closes #\_\_\_
Expand Down
10 changes: 6 additions & 4 deletions .github/workflows/pr-policy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@ name: PR Policy (reusable)

# Reusable workflow that enforces a minimum PR body contract:
# 1. PR body links an issue (Closes / Fixes / Refs #N)
# 2. PR body contains a Verification heading
# 3. PR body contains a Verification Notes heading
# 4. At least one verification checkbox is checked (`- [x]`)
# 2. PR body contains substantive Verification content
# 3. PR body contains substantive Verification Notes content
# 4. At least one substantive verification item — a plain bullet or a
# checkbox recording what was actually run (substance-only since #99;
# the exact `- [x]` format is no longer required)
#
# Doc-only PRs (every changed file matches the doc filter) skip the ceremony.
# Draft PRs are advisory only.
Expand All @@ -21,7 +23,7 @@ on:
type: boolean
default: true
require-checked-box:
description: "Require at least one `- [x]` checked box in the body."
description: "Require at least one substantive verification item (bullet or checkbox) in the body. Name kept for caller compatibility; substance-only since #99."
type: boolean
default: true
strict-body-contract:
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ Optional capabilities (claims #14, close-scan #28) are reported as "not installe

## Owner Maintenance Lane

When the working tree contains only add-only safe maintenance files, agents must not invoke Issue-Admiral, Project-Captain, Project-Lieutenant, Release-Admiral, claim records, handoff blocks, or full CI. Either report `owner maintenance present, no action required` or, if explicitly asked to commit, commit directly on `main` with `docs(owner): ...` or `chore(owner): ...`.
When the working tree contains only add-only safe maintenance files, agents must not start a delivery lane (issue/branch/PR), write claim records or handoff blocks, or trigger full CI. Either report `owner maintenance present, no action required` or, if explicitly asked to commit, commit directly on `main` with `docs(owner): ...` or `chore(owner): ...`. (Persona-role language — Issue-Admiral / Project-Captain / Project-Lieutenant / Release-Admiral — is retired; see README "role separation".)

Safe owner-maintenance paths are:

Expand Down
20 changes: 14 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ full immutability can pin to a commit SHA instead of `@v1`.

## Workflow Inventory

This repo currently contains 19 reusable `workflow_call` workflows and 2
This repo currently contains 20 reusable `workflow_call` workflows and 2
provider self-test workflows.

### Required Gate And PR Policy
Expand Down Expand Up @@ -161,13 +161,21 @@ docs/changelog, and issue-link content.
[`scripts/pr-contract.mjs`](scripts/pr-contract.mjs) for the shared PR contract.
The contract can require:

- `Summary`, `Verification`, `Verification Notes`, `Docs / Changelog`, and
`Linked Issue` sections.
- A `Closes #N`, `Fixes #N`, or `Refs #N` issue link.
- At least one checked verification item.
- Non-placeholder verification notes.
- Substantive `Summary`, `Verification`, `Verification Notes`, and
`Docs / Changelog` content (headings are located by name; presence/order of
the exact heading structure is advisory since #99).
- A `Closes #N`, `Fixes #N`, or `Refs #N` issue link anywhere in the body.
- At least one substantive verification item — a plain bullet or a checkbox
recording what was actually run or checked (substance-only since #99;
the exact `- [x]` format is no longer required).
- Non-placeholder, non-generic verification content ("tests pass" /
"CI green" claims still hard-fail).
- Branch names matching the configured convention.

Evidence blocks remain the recommended verification shape: they are validated
when present, and a checked item without one now draws an advisory warning
instead of a failure.

Doc-only PRs can skip the strict body ceremony when every changed file matches
the configured doc-only extensions or prefixes.

Expand Down
37 changes: 34 additions & 3 deletions contracts/pr-template.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,50 @@
<!--
Canonical PR body scaffold for ArchonVII repos (substance-only contract, #99).

Filling this out verbatim — with every TODO replaced by real content —
passes `repo-required-gate / pr-contract`. The rules:
- `## Verification` needs at least one substantive item (bullet or checkbox)
recording what was actually run or checked.
- Evidence blocks are the recommended shape; validated when present,
advisory when absent.
- Placeholders (TODO/TBD/N-slash-A) and generic claims ("tests pass",
"CI green") hard-fail.
- Heading order is advisory; section substance is not.
-->

## Summary

TODO: Replace with a concise description of the change.

## Verification

- [ ] TODO: Run required verification and replace this line.
- TODO: The exact command, CI check, or manual smoke test you ran, with its result.

```evidence
command: TODO
location: local
result: TODO
timestamp: TODO
```

### Verification Notes

TODO: Record concrete command output, CI check names, or manual smoke-test notes.

## Docs / Changelog

TODO: Record docs/changelog handling.
TODO: Record docs/changelog handling (fragment, direct edit, docs update, or no-changelog label).

Plan/status artifacts: TODO: closed, narrowed, superseded-with-pointer, or not applicable.

Owner decisions this lane: appended / none

## Linked Issue

TODO: Closes #___
TODO: Closes #\_\_\_

## Risks

- Risk level:
- Rollback:
- Follow-ups:
9 changes: 9 additions & 0 deletions docs/repo-update-log.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,15 @@ This log records agent-visible repository changes that should be easy to audit l
- **Propagation:** none | pending <repo/path> | completed <repo/path>
```

## 2026-07-01 - Substance-only PR verification contract

- **Issue/PR:** #99 / (pending)
- **Branch:** agent/claude/99-substance-only-contract
- **Changed paths:** scripts/pr-contract.mjs, scripts/pr-contract.test.mjs, .github/workflows/pr-policy.yml (comments/description only), .github/PULL_REQUEST_TEMPLATE.md, contracts/pr-template.md, README.md, AGENTS.md, .changelog/unreleased/99-substance-only-contract.md
- **What changed:** Verification items are substance-only — any non-empty bullet or checkbox counts (`missing_verification_item` replaces `missing_checked_verification`); unchecked boxes, missing evidence blocks, and heading order demote to advisories; placeholder + generic-claim rejection and section-substance checks stay hard; fenced content masked from item collection; advisories print on success. Templates, README contract description, and pr-policy header reconciled; mislabeled github-workflows PR template fixed (claimed to be repo-template's, denied this repo's CI surface); retired persona language removed from AGENTS.md; workflow inventory count corrected 19→20.
- **Verification:** `npm test` — vitest 9 files / 168 tests / 168 pass on the lane worktree (baseline before change: 159; net +9 covering bullets-count, unchecked-warns, evidence-warns, order-advisory, fence-masking, generic-bullet-fails, advisories-on-success).
- **Propagation:** pending — `@v1` retag (owner-gated) reaches all consumers' CI; repo-template re-vendor of `scripts/pr-contract.mjs` + parity test is the next friction-arc lane; archon-setup snapshot refresh follows.

## 2026-06-21 - Force LF for .mjs (Windows shebang loader fix)

- **Issue/PR:** #89 / (pending)
Expand Down
108 changes: 79 additions & 29 deletions scripts/pr-contract.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,14 @@ const TITLE_RE = /^(feat|fix|refactor|test|docs|style|chore|perf|ci|build|revert
const ISSUE_RE = /\b(Closes|Fixes|Refs)\s+#\d+\b/i;
// "placeholder" is valid completed prose; reject explicit unfilled markers.
const PLACEHOLDER_RE = /\b(TODO|TBD|FIXME|FILL ME|FILL IN|REPLACE THIS|NOT YET|N\/A|NONE YET)\b|#\s*(?:___|<[^>]+>)|<set-before-merge>/i;
const CHECKED_RE = /^\s*-\s+\[[xX]\]\s+(.+?)\s*$/;
const UNCHECKED_RE = /^\s*-\s+\[\s\]\s+(.+?)\s*$/;
const CHECKED_RE = /^\s*[-*]\s+\[[xX]\]\s+(.+?)\s*$/;
const UNCHECKED_RE = /^\s*[-*]\s+\[\s\]\s+(.+?)\s*$/;
// Substance-only contract (owner decision 2026-07-01, #99): any non-empty
// bullet in `## Verification` counts as a verification item — the contract
// requires that something substantive is recorded, not a specific checkbox
// format. The negative lookahead keeps checkbox lines out of the plain-item
// match so each line is classified exactly once.
const ITEM_RE = /^\s*[-*]\s+(?!\[[ xX]\]\s)(.+?)\s*$/;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Sync auto-inject with plain-bullet verification

With this regex a plain - Ran npm test line now satisfies the PR contract, but I checked .github/workflows/pr-body-autoinject.yml and its skip test still requires checked: /-\s+\[[xX]\]/ before it avoids prepending the TODO scaffold. For bot-authored non-doc PRs that already have the new valid plain-bullet verification, the reusable auto-inject workflow will still add TODO placeholders, and the same contract then fails on placeholder_text; update the auto-inject satisfiability check and stub to use the new item semantics.

Useful? React with 👍 / 👎.

const GENERIC_VERIFICATION_RE = /\b(automated ci checks? green|ci[- ]?green|ci checks? pass(?:ed|es)?|checks? green|all checks? pass(?:ed|es)?|tests? pass(?:ed|es)?)\b/i;

/**
Expand Down Expand Up @@ -69,29 +75,39 @@ export function validatePrContract(input, options = {}) {
validateBody(data.body, rules, errors, warnings);
}

const items = collectVerificationItems(data.body);
return {
ok: errors.length === 0,
errors,
warnings,
facts: {
docsOnly,
checkedVerificationCount: countCheckedVerificationItems(data.body),
checkedVerificationCount: items.checked.length,
verificationItemCount: [...items.checked, ...items.unchecked, ...items.plain]
.filter((item) => item.claim.length > 0).length,
},
};
}

export function formatPrContractResult(result) {
if (result.ok) {
const suffix = result.facts.docsOnly ? ' (docs-only body ceremony skipped)' : '';
return `PR contract passed${suffix}.`;
const lines = [`PR contract passed${suffix}.`];
if (result.warnings.length > 0) {
lines.push('', 'Advisories (non-blocking):');
for (const item of result.warnings) {
lines.push(`- [${item.code}] ${item.message}`);
}
}
return lines.join('\n');
}

const lines = ['PR contract failed.', '', 'Required fixes:'];
for (const item of result.errors) {
lines.push(`- [${item.code}] ${item.message}`);
}
if (result.warnings.length > 0) {
lines.push('', 'Warnings:');
lines.push('', 'Advisories (non-blocking):');
for (const item of result.warnings) {
lines.push(`- [${item.code}] ${item.message}`);
}
Expand Down Expand Up @@ -188,7 +204,10 @@ function validateBody(body, rules, errors, warnings) {
));
}

validateHeadingOrder(headings, rules.requiredHeadings, errors);
// Heading presence/order is advisory since #99: sections are located by
// name, so the substance checks below still hard-fail when a section's
// content is genuinely missing — only the exact structure is soft.
validateHeadingOrder(headings, rules.requiredHeadings, warnings);

const summary = sectionContent(body, headings, 'Summary');
if (rules.requireSummary && !hasSubstantiveContent(summary)) {
Expand Down Expand Up @@ -216,44 +235,41 @@ function validateBody(body, rules, errors, warnings) {
errors.push(error('empty_docs_changelog', '`## Docs / Changelog` must describe docs or changelog handling.', 'body'));
}

const verification = sectionContent(body, headings, 'Verification', { stopBefore: 'Verification Notes' });
const checkedItems = verification.split(/\r?\n/).map((line, index) => {
const match = line.match(CHECKED_RE);
return match ? { claim: match[1].trim(), index } : null;
}).filter(Boolean);
const uncheckedItems = verification.split(/\r?\n/).map((line) => {
const match = line.match(UNCHECKED_RE);
return match ? match[1].trim() : null;
}).filter(Boolean);

if (rules.requireCheckedVerification && checkedItems.length === 0) {
const { verification, checked, unchecked, plain } = collectVerificationItems(body);
const allItems = [...checked, ...unchecked, ...plain];
const substantiveItems = allItems.filter((item) => item.claim.length > 0);

if (rules.requireCheckedVerification && substantiveItems.length === 0) {
errors.push(error(
'missing_checked_verification',
'`## Verification` must include at least one checked verification item.',
'missing_verification_item',
'`## Verification` must include at least one verification item — a bullet or checkbox recording what was actually run or checked.',
'body',
));
}

for (const claim of uncheckedItems) {
errors.push(error(
'unchecked_required_box',
`Unchecked verification item must be completed or removed before ready-for-review: "${claim}".`,
for (const item of unchecked) {
warnings.push(error(
'unchecked_verification_item',
`Verification item is unchecked; it still counts as an item, but reads as not-done: "${item.claim}".`,
'body',
));
}

for (const item of checkedItems) {
for (const item of allItems) {
if (GENERIC_VERIFICATION_RE.test(item.claim)) {
errors.push(error(
'generic_verification',
`Checked verification item is too generic: "${item.claim}". Record the actual command, check, or manual action.`,
`Verification item is too generic: "${item.claim}". Record the actual command, check, or manual action.`,
'body',
));
}
}

for (const item of checked) {
if (rules.requireEvidenceBlocks && !hasEvidenceBlockAfter(verification, item.index)) {
errors.push(error(
warnings.push(error(
'missing_evidence_block',
`Checked verification item "${item.claim}" must be followed by a fenced \`\`\`evidence block.`,
`Checked verification item "${item.claim}" is not followed by a fenced \`\`\`evidence block (recommended shape; advisory since #99).`,
Comment on lines +270 to +272

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Align evidence enforcement with advisory semantics

This demotes missing evidence to a PR-contract warning, but in .github/workflows/pr-policy.yml the separate Evidence check still calls parseEvidence, whose missing-evidence result remains an error and calls core.setFailed when enforce-evidence is true. Consumers that opted into enforce-evidence: true and follow the new template/docs by using a checked item without an evidence block will still have the reusable workflow fail, so the evidence parser/workflow needs the same advisory semantics or the docs must exclude enforce-evidence callers.

Useful? React with 👍 / 👎.

'body',
));
}
Expand Down Expand Up @@ -375,10 +391,44 @@ function hasEvidenceBlockAfter(section, checkedLineIndex) {
return false;
}

function countCheckedVerificationItems(body) {
// Classify every line of the `## Verification` section (fenced blocks masked
// so evidence field values are never misread as items) into checked boxes,
// unchecked boxes, and plain bullets. Line indexes are preserved through the
// masking so evidence-adjacency checks still run against the raw section.
function collectVerificationItems(body) {
const headings = parseHeadings(body || '');
const verification = sectionContent(body || '', headings, 'Verification', { stopBefore: 'Verification Notes' });
return verification.split(/\r?\n/).filter((line) => CHECKED_RE.test(line)).length;
const checked = [];
const unchecked = [];
const plain = [];
maskFencedLines(verification).forEach((line, index) => {
let match = line.match(CHECKED_RE);
if (match) {
checked.push({ claim: match[1].trim(), index });
return;
}
match = line.match(UNCHECKED_RE);
if (match) {
unchecked.push({ claim: match[1].trim(), index });
return;
}
match = line.match(ITEM_RE);
if (match) plain.push({ claim: match[1].trim(), index });
});
return { verification, checked, unchecked, plain };
}

// Blank out fence delimiters and fenced content while preserving line count,
// so indexes into the raw section stay valid.
function maskFencedLines(text) {
let inFence = false;
return String(text || '').split(/\r?\n/).map((line) => {
if (/^\s*```/.test(line)) {
inFence = !inFence;
return '';
}
return inFence ? '' : line;
});
}

function isDocsOnly(files, rules) {
Expand Down
Loading