diff --git a/README.md b/README.md index 4c720e2..e69b354 100644 --- a/README.md +++ b/README.md @@ -3,8 +3,14 @@ [![CI](https://github.com/RobYed/pr-pitcrew/actions/workflows/ci.yml/badge.svg)](https://github.com/RobYed/pr-pitcrew/actions/workflows/ci.yml) [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE) -A pit crew for your pull requests. Several specialists, each with one job, all at once, and the car -leaves in better shape than it arrived. +Three review agents for your pull requests, installed as GitHub Actions workflows and run on your +own LLM endpoint. Open a pull request and it is read for bugs and for security defects, with each +finding written as a comment on the line it is about. Ask for the third one and an agent with a +browser drives your app, walks the linked issue's acceptance criteria, and leaves the +recording behind for you to check. + +A real pit crew: several specialists, each with one job, all at once, and the car leaves in better +shape than it arrived. Three of them ship today: @@ -18,9 +24,50 @@ They run on [OpenCode](https://opencode.ai) inside **your** runner, against **yo OpenAI-compatible endpoint, with **your** key. No third-party service sits in between, and no model is preselected: which model reads your code is a decision you make with your provider. +## What a pull request looks like with this installed + +**You push.** `Pitcrew / Bug Review` and `Pitcrew / Security Review` start on the new commits. +Minutes later the diff carries comments where the defects are, and one summary comment above them: +verdict, counts by severity, what was looked at. From `high` upward (configurable) the check is red, and it stays +red until somebody deals with the finding. + +**You push again.** Only the commits you added are reviewed, and a point one of the agents already +made is not made a second time. What you fixed disappears; what you did not stays red. + +**You ask for the acceptance test.** You request a review from the pitcrew account, which is the +gesture a human reviewer gets, and an agent with a browser drives your deployed app: it takes the +acceptance criteria from the linked issue, verbatim, does the steps by hand, and records them. What lands on +the pull request is a table of those criteria, each marked met or not and stamped with the moment it +happens in the recording, above a link to the video and the screenshots in the run's artifacts. + +**Whatever a verdict does not tell you** is in the run summary: every tool call the agent made, its +input, and what the agent wrote. + +## Who this is for + +**A team whose review bot bills more than it explains.** A hosted service charges per seat, per +repository or per review, and what was done for that money is not visible from the outside. Here the +bill arrives from your own provider, per token, for a model you picked, and the run summary shows +the work the verdict came from. + +**A team that has to answer for where the code goes.** Your runner, your endpoint, your key: the +only outside party that sees the diff is the provider you picked, and the workflow that sends it is +seventeen lines you can read. When that question comes from a customer or an audit, the answer is +yours to give rather than a vendor's. + +**A team whose issues carry acceptance criteria.** If the criteria are written down and there is a +deployed environment to point at, the acceptance test is where this pays off most: the manual +walk-through somebody does before every merge becomes a table on the pull request and a video to +check it against. + +**A team who wants to skip human review - YOLO!.** Review takes developer's time. You trust AI outputs +without any fear. Speed matters more than perfection. What should go wrong?! + ## Quickstart -One file in your repository. That is the whole installation. +One file per agent, and that file is the whole of the agent's installation: the scripts, the prompts +and the permission profiles travel with the package instead of into your repository. This is the bug +review: ```yaml # .github/workflows/pitcrew-bug-review.yml @@ -48,7 +95,7 @@ jobs: api-key: ${{ secrets.PITCREW_LLM_API_KEY }} ``` -Then, under **Settings → Secrets and variables → Actions**: +Then, once for all three agents, under **Settings → Secrets and variables → Actions**: | | | | --- | --- | @@ -58,10 +105,12 @@ Then, under **Settings → Secrets and variables → Actions**: Open a pull request. There is no fourth step. -`examples/` has the same file for all three agents. Enable one, two or all three; none of them needs -the others. +The other two are that same file with another name in it. `examples/` has all three, ready to copy: +the security review is installed exactly like this one, and the acceptance test differs in what +starts it and in two variables of its own, which the section below explains. Beyond the endpoint, +the key and the model, they share nothing, so enable one, two or all three. -## What you get +## How the output behaves **Findings at the code, not in a wall of text.** Each finding becomes a comment on the line it is about. Above them, one summary comment in a frame the package writes: heading, verdict, counts by @@ -82,10 +131,10 @@ call, its input behind a fold, and what the agent wrote. The verdict and the wor questions. **Read-only by construction.** The two review agents have no shell, no `webfetch`, no `websearch` -and no sub-agents; they cannot read `/proc`, `/sys` or `.git/`; they may write only inside a -git-ignored run directory. That is not a promise made to the model in a prompt, it is a permission -profile that ships with this package, so a pull request cannot grant its own reviewer a shell even -by editing every file it can reach. See [`docs/threat-model.md`](docs/threat-model.md). +and no sub-agents; they may write only inside a git-ignored run directory. That is not a promise +made to the model in a prompt, it is a permission profile that ships with this package, so a pull +request cannot grant its own reviewer a shell even by editing every file it can reach. +See [`docs/threat-model.md`](docs/threat-model.md). ## Updating, and pinning @@ -98,7 +147,7 @@ uses: RobYed/pr-pitcrew/.github/workflows/bug-review.yml@a1b2c3d4... # v1.0.0 Both are supported. `@v1` is the convenient one, a SHA is the deliberate one. -## The acceptance test +## The acceptance test agent This one costs more than the other two: a container, a browser, up to half an hour, and real operations against a real environment. So **it does not start itself**. It starts when somebody @@ -106,6 +155,10 @@ requests a review from the account in `PITCREW_ACCEPTANCE_REVIEWER`, which is th human reviewer gets. Removing that reviewer and adding them back is how you ask for a second run. `/acceptance` in a comment does the same. +It leaves one comment behind: the criteria from the issue, each marked met or not and stamped with +the moment it happens in the recording, and a link to the video and the screenshots, which are +uploaded with the run. + It needs `PITCREW_ACCEPTANCE_TARGET_URL` (the deployed app), and takes credentials for it if it has a login. Everything it needs is named `PITCREW_ACCEPTANCE_*`, so a repository running only the two diff reviews can see at a glance that none of it applies. It refuses to run in a public repository