feat(pr-contract): substance-only verification contract + template/doc reconciliation#100
Conversation
Any non-empty bullet or checkbox in ## Verification now counts as a verification item; unchecked boxes, missing evidence blocks, and heading presence/order demote to advisories; placeholder and generic-claim rejection plus section-substance checks stay hard. Fenced content is masked from item collection so evidence output is never misread as a claim. Advisories surface in the formatted result on success. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cTBuec3roYwjTcVNToC4u
…act (#99) Fix the mislabeled PULL_REQUEST_TEMPLATE (claimed to be repo-template's and denied this repo's CI surface), make contracts/pr-template.md a scaffold that passes the gate it models, and update pr-policy.yml header comments and the require-checked-box description (input name kept for caller compatibility). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cTBuec3roYwjTcVNToC4u
AGENTS.md still instructed the retired Issue-Admiral/Project-Captain/ Project-Lieutenant/Release-Admiral personas, contradicting the README; README said 19 reusables where 20 exist and described the pre-#99 checked-box rule. Adds the changelog fragment and update-log entry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cTBuec3roYwjTcVNToC4u
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 70f984293b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // requires that something substantive is recorded, not a specific checkbox | ||
| // format. The negative lookahead keeps checkbox lines out of the plain-item | ||
| // match so each line is classified exactly once. | ||
| const ITEM_RE = /^\s*[-*]\s+(?!\[[ xX]\]\s)(.+?)\s*$/; |
There was a problem hiding this comment.
Sync auto-inject with plain-bullet verification
With this regex a plain - Ran npm test line now satisfies the PR contract, but I checked .github/workflows/pr-body-autoinject.yml and its skip test still requires checked: /-\s+\[[xX]\]/ before it avoids prepending the TODO scaffold. For bot-authored non-doc PRs that already have the new valid plain-bullet verification, the reusable auto-inject workflow will still add TODO placeholders, and the same contract then fails on placeholder_text; update the auto-inject satisfiability check and stub to use the new item semantics.
Useful? React with 👍 / 👎.
| warnings.push(error( | ||
| 'missing_evidence_block', | ||
| `Checked verification item "${item.claim}" must be followed by a fenced \`\`\`evidence block.`, | ||
| `Checked verification item "${item.claim}" is not followed by a fenced \`\`\`evidence block (recommended shape; advisory since #99).`, |
There was a problem hiding this comment.
Align evidence enforcement with advisory semantics
This demotes missing evidence to a PR-contract warning, but in .github/workflows/pr-policy.yml the separate Evidence check still calls parseEvidence, whose missing-evidence result remains an error and calls core.setFailed when enforce-evidence is true. Consumers that opted into enforce-evidence: true and follow the new template/docs by using a checked item without an evidence block will still have the reusable workflow fail, so the evidence parser/workflow needs the same advisory semantics or the docs must exclude enforce-evidence callers.
Useful? React with 👍 / 👎.
Summary
Substance-only PR verification contract (owner decision, 2026-07-01 friction arc): the parser now requires that something substantive is recorded in
## Verification— a plain bullet or a checkbox both count — and stops hard-failing on exact format. June data motivating this: PR-body format failures caused multi-round fix-commit loops, and each round re-ran the full 6–7-workflow fan-out per consumer PR.scripts/pr-contract.mjs:missing_verification_item(replacesmissing_checked_verification) fires only when zero substantive items exist; unchecked boxes → advisoryunchecked_verification_item; missing evidence fence after a checked item → advisorymissing_evidence_block; heading presence/order → advisory (substance checks still locate sections by name and hard-fail when content is missing); placeholder + generic-claim ("tests pass"/"CI green") rejection unchanged; fenced content inside Verification is masked so evidence output is never misread as a claim; advisories now print on success.@v1retag (owner-gated) — the gate's pr-contract job imports this script from the checked-out library; no caller-input changes needed (require-checked-boxname kept)..github/PULL_REQUEST_TEMPLATE.mdfixed (claimed to be repo-template's and denied this repo's vitest CI surface);contracts/pr-template.mdnow passes the gate it models; README inventory 19→20 + contract description updated; retired persona language removed fromAGENTS.md;pr-policy.ymlheader/description comments updated.Verification
Ran the full vitest suite in the lane worktree: 9 files / 168 tests / 168 pass (baseline pre-change: 159; +9 net new cases covering bullets-count,
*-bullets, unchecked-warns, evidence-warns, order-advisory-passes, renamed-heading-still-fails-via-substance, fence-masking, generic-bullet-fails, advisories-print-on-success).Exercised the CLI against this very PR body shape:
node scripts/pr-contract.mjs --body-file -accepts a bullets-plus-evidence body and rejects an items-free body withmissing_verification_item(covered by the body-file acceptance tests inscripts/pr-contract.test.mjs).Verification Notes
The acceptance table from the #81/#82/#91 work is preserved unchanged — the "still fails" cases (placeholder wording in Summary, an unfilled command field inside a real evidence fence, a generic checked claim, a docs section containing only a not-applicable marker) all still fail, so the parser is more tolerant of format, not weaker on substance.
validatePrTemplatestill passes on the shipped templates.Docs / Changelog
Changelog fragment:
.changelog/unreleased/99-substance-only-contract.md. Update-log entry appended todocs/repo-update-log.md(includes the propagation plan). README + AGENTS.md updated in this PR.Plan/status artifacts: friction-arc plan is owner-side (
C:\Users\josep\.claude\plans\please-review-what-is-fluttering-lobster.md); no repo plan docs created or used by this lane.Owner decisions this lane: appended — substance-only relaxation ratified by owner 2026-07-01 (AskUserQuestion session record).
Linked Issue
Closes #99
Risks
@v1retag. Mitigations: substance checks unchanged or strengthened (fence masking), acceptance-table regression tests preserved, error-code rename verified unreferenced outside this repo's own test (repo-template's vendored copy is re-vendored in the next lane).@v1back; no schema or state changes.@v1retag is owner-gated and should ride after this merges.