The unit tests cover the places where a mistake stays quiet — deduplication, the comment frame, the quality gate, the diff scope, the manifest agreeing with the workflow. None of them prove that a run works end to end, because none of them talks to GitHub or to a model.
The self-review on this repository's own pull requests is the closest thing there is, and it has a known hole: a called workflow's uses: cannot be an expression, so it exercises actions/agent@main rather than the branch (see CONTRIBUTING.md).
What to build
A scratch repository with a fixture pull request that deliberately contains one provable defect and one linked issue with acceptance criteria, and a documented walk-through: install the three workflows from examples/, set the variables, run each agent, check the comment, the inline findings, the criteria table, the artifact link and the exit code.
Automated as a nightly run if the model cost allows. As a hand-check once per release if it does not. For the acceptance test, a hand-check once per release is worth more than a brittle automated one.
Acceptance criteria
- The walk-through is written down as steps somebody other than the author can follow.
- It covers a re-run (the comment must be replaced, not duplicated), a second push (the findings from the first must not be reposted), and a run whose agent writes no report.
- The result of the last run is recorded somewhere a reader can find, with the date and the version.
The unit tests cover the places where a mistake stays quiet — deduplication, the comment frame, the quality gate, the diff scope, the manifest agreeing with the workflow. None of them prove that a run works end to end, because none of them talks to GitHub or to a model.
The self-review on this repository's own pull requests is the closest thing there is, and it has a known hole: a called workflow's
uses:cannot be an expression, so it exercisesactions/agent@mainrather than the branch (seeCONTRIBUTING.md).What to build
A scratch repository with a fixture pull request that deliberately contains one provable defect and one linked issue with acceptance criteria, and a documented walk-through: install the three workflows from
examples/, set the variables, run each agent, check the comment, the inline findings, the criteria table, the artifact link and the exit code.Automated as a nightly run if the model cost allows. As a hand-check once per release if it does not. For the acceptance test, a hand-check once per release is worth more than a brittle automated one.
Acceptance criteria