feat: F2/F10 evidence parser + warning-only pr-policy wiring#19
Merged
Conversation
Pure ES-module parser for fenced ```evidence``` blocks under checked verification items. Implements the 2026-05-19 amendment shape: - Per-block flat YAML (command/location/result/timestamp, optional check) - location: ci|local|manual - local: command must contain a recognised executable token - ci: command (or check:) must match a successful check-run on head SHA - timestamps: future guard >now+5min; local/manual must be >=head-5min - ci timestamp drift >5min vs check-run completed_at is a warning Parser has no SDK or network dependencies — the caller injects already- fetched data via the function signature. Hand-rolled key:value parser avoids adding a YAML dep. Also adds a minimal repo-root package.json with vitest as the sole devDep so the parser can be unit-tested in CI. Reported as a repo-contract change. Refs #10 Refs #12
15 cases covering all amendment-2026-05-19 requirements: happy paths (local/ci/manual), structural errors (missing block, double block, malformed YAML, unknown location), command validation (no token, missing check-run, failed check-run), and timestamp validation (future, pre-head, ci drift warning, unchecked, invalid ISO). Refs #10 Refs #12
Adds an `enforce-evidence` boolean input (default false for Phase 1) and a new evidence-check step in the existing policy job that: - skips on draft PRs (advisory) and doc-only PRs (existing detection) - fetches check-runs for the PR head SHA via github.rest.checks.listForRef - fetches the head commit time via github.rest.git.getCommit - dynamic-imports scripts/parse-evidence.mjs from a sibling checkout of ArchonVII/github-workflows (so callers don't need the script vendored) - writes errors and warnings into $GITHUB_STEP_SUMMARY - only calls core.setFailed when enforce-evidence === true AND errors exist A comment block above the step documents that Owner Maintenance Lane is direct-commit-only in Phase 1 and intentionally has no PR-path exemption. Refs #10 Refs #12
Source: PR #19 review patch 1 (2026-05-20). Adds common shell utility tokens so 'location: local' evidence rows that invoke grep/awk/sed/jq parse cleanly under enforce mode.
Source: PR #19 review patch 1 (2026-05-20). Directly guards against the dogfood regression where 'grep -nE ...' would hard-fail under enforce mode. 16/16 passing.
Source: PR #19 review patch 2 (2026-05-20). Pins the ArchonVII/github-workflows checkout that supplies parse-evidence.mjs to the same ref the caller pins this reusable workflow to. Default v1 keeps parser+workflow versioned together; callers using @main or @sha should pass workflow-library-ref to match. Previously the checkout had no ref and defaulted to the default branch (main), creating 'workflow from v1, parser from main' drift.
ArchonVII
marked this pull request as ready for review
May 20, 2026 18:36
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #10
Closes #12
Summary
Phase 1 of the F2 + F10 evidence-enforcement amendment (2026-05-19): adds a
pure ES-module parser for fenced
```evidence```blocks under checkedverification items, wires it into
pr-policy.ymlin warning-only mode, andadds a self-test CI workflow that runs the parser's vitest suite on PRs
touching
scripts/**.The amendment supersedes the earlier compact-one-liner / repo allow-list /
24h-window shape. This PR implements the amendment shape only.
What's in scope
scripts/parse-evidence.mjs— pure parser (no SDK imports). Caller injectsPR body, head commit time, "now", and check-runs via the function signature.
scripts/parse-evidence.test.mjs— 16 vitest cases covering happy paths,structural errors, command validation (local-token list + ci check-run
lookup), timestamp policy (future guard, post-head requirement, ci drift
warning, invalid ISO), plus a grep-evidence regression case.
.github/workflows/pr-policy.yml— addsenforce-evidenceinput(defaults to
false) and aworkflow-library-refinput (defaults tov1) that pins the parser-source checkout to the same ref callers pinthis reusable workflow to. Adds an evidence-check step that runs after
the existing checks. Skips drafts and doc-only PRs. Writes errors and
warnings to
$GITHUB_STEP_SUMMARY. Only callscore.setFailedwhenenforce-evidenceis true..github/workflows/self-test.yml— Node 20 vitest run onscripts/**changes.
package.jsonwithvitestas the sole devDep.What's NOT in scope (per task constraints)
direct-commit-only and never produces a PR; the exemption lives upstream.
A comment in
pr-policy.ymldocuments this so future-me doesn't add it.enforce-evidenceto true anywhere..agent/verification-commands.yml, orhook-authority.mdchanges — those are separate PRs in other repos.Verification
Verification Notes
Dogfooding note
Since
enforce-evidencedefaults tofalsein Phase 1, any malformedevidence in this PR will only surface as warnings in the job summary —
that's the point. Once the parser is live in Phase 2 we'll flip the
input for
github-workflows's own PRs first.