Skip to content

feat(pr-contract): substance-only verification contract + template/doc reconciliation#100

Merged
ArchonVII merged 3 commits into
mainfrom
agent/claude/99-substance-only-contract
Jul 2, 2026
Merged

feat(pr-contract): substance-only verification contract + template/doc reconciliation#100
ArchonVII merged 3 commits into
mainfrom
agent/claude/99-substance-only-contract

Conversation

@ArchonVII

@ArchonVII ArchonVII commented Jul 2, 2026

Copy link
Copy Markdown
Owner

Summary

Substance-only PR verification contract (owner decision, 2026-07-01 friction arc): the parser now requires that something substantive is recorded in ## Verification — a plain bullet or a checkbox both count — and stops hard-failing on exact format. June data motivating this: PR-body format failures caused multi-round fix-commit loops, and each round re-ran the full 6–7-workflow fan-out per consumer PR.

  • scripts/pr-contract.mjs: missing_verification_item (replaces missing_checked_verification) fires only when zero substantive items exist; unchecked boxes → advisory unchecked_verification_item; missing evidence fence after a checked item → advisory missing_evidence_block; heading presence/order → advisory (substance checks still locate sections by name and hard-fail when content is missing); placeholder + generic-claim ("tests pass"/"CI green") rejection unchanged; fenced content inside Verification is masked so evidence output is never misread as a claim; advisories now print on success.
  • Behavior reaches every consumer at the next @v1 retag (owner-gated) — the gate's pr-contract job imports this script from the checked-out library; no caller-input changes needed (require-checked-box name kept).
  • Templates/docs reconciled: mislabeled .github/PULL_REQUEST_TEMPLATE.md fixed (claimed to be repo-template's and denied this repo's vitest CI surface); contracts/pr-template.md now passes the gate it models; README inventory 19→20 + contract description updated; retired persona language removed from AGENTS.md; pr-policy.yml header/description comments updated.

Verification

  • Ran the full vitest suite in the lane worktree: 9 files / 168 tests / 168 pass (baseline pre-change: 159; +9 net new cases covering bullets-count, *-bullets, unchecked-warns, evidence-warns, order-advisory-passes, renamed-heading-still-fails-via-substance, fence-masking, generic-bullet-fails, advisories-print-on-success).

    command: npm test
    location: local
    result: Test Files 9 passed (9); Tests 168 passed (168)
    timestamp: 2026-07-02T00:30:18Z
    
  • Exercised the CLI against this very PR body shape: node scripts/pr-contract.mjs --body-file - accepts a bullets-plus-evidence body and rejects an items-free body with missing_verification_item (covered by the body-file acceptance tests in scripts/pr-contract.test.mjs).

Verification Notes

The acceptance table from the #81/#82/#91 work is preserved unchanged — the "still fails" cases (placeholder wording in Summary, an unfilled command field inside a real evidence fence, a generic checked claim, a docs section containing only a not-applicable marker) all still fail, so the parser is more tolerant of format, not weaker on substance. validatePrTemplate still passes on the shipped templates.

Docs / Changelog

Changelog fragment: .changelog/unreleased/99-substance-only-contract.md. Update-log entry appended to docs/repo-update-log.md (includes the propagation plan). README + AGENTS.md updated in this PR.

Plan/status artifacts: friction-arc plan is owner-side (C:\Users\josep\.claude\plans\please-review-what-is-fluttering-lobster.md); no repo plan docs created or used by this lane.

Owner decisions this lane: appended — substance-only relaxation ratified by owner 2026-07-01 (AskUserQuestion session record).

Linked Issue

Closes #99

Risks

  • Risk level: Medium — this changes gate behavior for every consumer at the next @v1 retag. Mitigations: substance checks unchanged or strengthened (fence masking), acceptance-table regression tests preserved, error-code rename verified unreferenced outside this repo's own test (repo-template's vendored copy is re-vendored in the next lane).
  • Rollback: revert the three commits and retag @v1 back; no schema or state changes.
  • Follow-ups: repo-template re-vendor + parity test (next friction-arc lane); archon-setup snapshot refresh; @v1 retag is owner-gated and should ride after this merges.

ArchonVII and others added 3 commits July 1, 2026 19:29
Any non-empty bullet or checkbox in ## Verification now counts as a
verification item; unchecked boxes, missing evidence blocks, and heading
presence/order demote to advisories; placeholder and generic-claim
rejection plus section-substance checks stay hard. Fenced content is
masked from item collection so evidence output is never misread as a
claim. Advisories surface in the formatted result on success.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017cTBuec3roYwjTcVNToC4u
…act (#99)

Fix the mislabeled PULL_REQUEST_TEMPLATE (claimed to be repo-template's
and denied this repo's CI surface), make contracts/pr-template.md a
scaffold that passes the gate it models, and update pr-policy.yml header
comments and the require-checked-box description (input name kept for
caller compatibility).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017cTBuec3roYwjTcVNToC4u
AGENTS.md still instructed the retired Issue-Admiral/Project-Captain/
Project-Lieutenant/Release-Admiral personas, contradicting the README;
README said 19 reusables where 20 exist and described the pre-#99
checked-box rule. Adds the changelog fragment and update-log entry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017cTBuec3roYwjTcVNToC4u

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 70f984293b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/pr-contract.mjs
// requires that something substantive is recorded, not a specific checkbox
// format. The negative lookahead keeps checkbox lines out of the plain-item
// match so each line is classified exactly once.
const ITEM_RE = /^\s*[-*]\s+(?!\[[ xX]\]\s)(.+?)\s*$/;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Sync auto-inject with plain-bullet verification

With this regex a plain - Ran npm test line now satisfies the PR contract, but I checked .github/workflows/pr-body-autoinject.yml and its skip test still requires checked: /-\s+\[[xX]\]/ before it avoids prepending the TODO scaffold. For bot-authored non-doc PRs that already have the new valid plain-bullet verification, the reusable auto-inject workflow will still add TODO placeholders, and the same contract then fails on placeholder_text; update the auto-inject satisfiability check and stub to use the new item semantics.

Useful? React with 👍 / 👎.

Comment thread scripts/pr-contract.mjs
Comment on lines +270 to +272
warnings.push(error(
'missing_evidence_block',
`Checked verification item "${item.claim}" must be followed by a fenced \`\`\`evidence block.`,
`Checked verification item "${item.claim}" is not followed by a fenced \`\`\`evidence block (recommended shape; advisory since #99).`,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Align evidence enforcement with advisory semantics

This demotes missing evidence to a PR-contract warning, but in .github/workflows/pr-policy.yml the separate Evidence check still calls parseEvidence, whose missing-evidence result remains an error and calls core.setFailed when enforce-evidence is true. Consumers that opted into enforce-evidence: true and follow the new template/docs by using a checked item without an evidence block will still have the reusable workflow fail, so the evidence parser/workflow needs the same advisory semantics or the docs must exclude enforce-evidence callers.

Useful? React with 👍 / 👎.

@ArchonVII
ArchonVII merged commit 6b39f55 into main Jul 2, 2026
1 check passed
@ArchonVII
ArchonVII deleted the agent/claude/99-substance-only-contract branch July 2, 2026 04:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(pr-contract): substance-only verification contract + template/doc reconciliation

1 participant