Code Review Action: openlibrary-bot-driven PR review (Copilot replacement)

[mekarpeles]

Summary

Build a GitHub Action, using the same Claude-token pattern as issue enrichment (#13075) and the planned PR auto-responder (#13162), that gives PRs an automated code review similar to what GitHub Copilot used to provide before free Copilot credits for OSS maintainers were discontinued. Sub-task of #13160.

Trigger

When @openlibrary-bot is added a PR reviewer, or

Staff-only, for security:

• at-mentioning @openlibrary-bot directly in a PR comment

The permission-gate bypass problem found in #13161 hopefully doesn't apply here, but should be verified once this is built rather than assumed. It should work at least for me...

Behavior

• Post inline suggestion-style comments on specific code blocks/lines, similar to Copilot's review UX (not just a single summary comment).
• On re-trigger for the same PR, review only the diff since its last review pass — not the whole PR again. Needs a persisted "last reviewed SHA" marker, likely reusing the <!-- ol-pr-bot -->-style idempotency marker convention already used in pr-prereview.md.
• Must ignore openlibrary-bot's own prior comments/actions when re-triggered, to avoid the same class of feedback-loop bug as openlibrary-bot comments trigger "Needs: Response" labeling #13138.
• Maps onto the existing Revere identity/role (deep code review across correctness, security, a11y, tests, style — see pm repo AGENTS.md) — the gap being closed here is automation transport, not review logic. Findings should follow the same safety rules as pr-prereview.md (no value judgments, never mention Claude/Anthropic, idempotency guard).

Research needed before design

• How does Copilot structure its inline review comments technically (REST/GraphQL shape for line-anchored suggestions)? Needed to know what's achievable via gh api / claude-code-action vs. what needs a custom script.
• Confirm whether anthropics/claude-code-action supports posting line-anchored review comments natively, or whether this needs direct GitHub Review API calls (POST /repos/{owner}/{repo}/pulls/{pull_number}/reviews with comments[]) wrapped around it.

Related

• Epic: AI Workflows — GitHub Actions automation for the issue/PR lifecycle #13160 — Epic: AI Workflows (parent)
• Port PR pre-review/welcomer script to a GitHub Action #13162 — PR Auto-Responder port — currently assumes Copilot is assignable as a reviewer; that step needs to point here once this ships
• Issue Enrichment: allowed_non_write_users bypass not taking effect for non-write actors #13161 — non-write-actor permission bug found in issue enrichment; different trigger model here, but worth a sanity check once built
• openlibrary-bot comments trigger "Needs: Response" labeling #13138 — bot-comment feedback-loop bug (ignore-openlibrary-bot's-own-actions class)

[openlibrary-bot]

Warning

This issue requires access to internal infrastructure, production systems, or institutional knowledge unavailable to community contributors. It can only be resolved by a maintainer or staff member.

⚠️ Contributors, this issue will be ready to work on once:

• Triage — maintainers will add Lead: @*
• Needs: Investigation — root cause or approach unknown; research required first
• Needs: Staff / Internal — requires production access or institutional knowledge

---

Context

• The idempotency-marker convention this issue wants to reuse (<!-- ol-pr-bot -->, from pr-prereview.md) does not exist anywhere in this repository — a repo-wide grep for pr-prereview and ol-pr-bot returns zero hits. Per Port PR pre-review/welcomer script to a GitHub Action #13162's body, that file and the pam.py script that runs it live in local, staff-only directories (~/Projects/openlibrary-pam, ~/Projects/pm), not in a public repo. A community contributor has no way to inspect its conventions or safety rules without a staff member surfacing the content first — this is the basis for the Needs: Staff / Internal label.
• .github/workflows/issue_enrichment.yml#L39-L51 is the existing claude-code-action@v1 pattern this issue explicitly wants to mirror. It's also where the actor-permission bypass (allowed_non_write_users) lives — the exact mechanism Issue Enrichment: allowed_non_write_users bypass not taking effect for non-write actors #13161 found broken for the issue-enrichment case. This issue already correctly notes the PR-reviewer trigger path differs (staff-initiated, so the actor already has write access), but since Issue Enrichment: allowed_non_write_users bypass not taking effect for non-write actors #13161 is still open and unresolved, the "likely doesn't apply here" assumption can't be verified against a working reference implementation yet.
• The repo already has two precedents for PR-triggered GitHub Actions, neither of which fits this issue's requirements: pr_changes_requested.yml (pull_request_review: submitted, plain github-script labeling, no Claude involved) and new_pr_labeler.yml (pull_request_target: opened). Neither posts line-anchored review comments, and neither is triggered by a review request or an @-mention — so the two "Research needed before design" questions in the issue body (Copilot's inline-comment API shape, and whether claude-code-action supports the GitHub Review API's comments[] natively) have no existing answer anywhere in this codebase; they still need to be answered from outside the repo before implementation can start.
• Related to the trigger design: the two staff-initiated paths described ("requesting as reviewer" vs. "@-mentioning in a comment") map to different GitHub webhook events (pull_request_review_requested vs. issue_comment/pull_request_review_comment). The issue describes the user-facing behavior but not which event(s) the workflow's on: block should bind to — worth pinning down as part of the design research, since it affects what github.event fields are available to the skip-check logic.

Issue Assessment

Open questions

• Which GitHub event(s) should the workflow trigger on for each of the two paths (reviewer-request vs. @-mention), and does a single workflow file cleanly handle both, or are two triggers/jobs needed?
• Does anthropics/claude-code-action support posting line-anchored review comments (via the GitHub Review API's comments[]) natively, or will this need direct gh api / REST calls wrapped around the action? (This is one of the two research items already flagged in the issue body — surfacing it here since it blocks design.)
• How should "diff since last review pass" be computed and persisted — a marker comment storing the last-reviewed SHA (as proposed), or something derived from the PR's own review history via the API?

Action items

• Source the actual content of pr-prereview.md (or an equivalent internal writeup) from staff, since it isn't accessible from this repo and the issue's safety-rule and idempotency requirements depend on it.
• Once Issue Enrichment: allowed_non_write_users bypass not taking effect for non-write actors #13161 is resolved, re-verify whether its actor-permission finding has any bearing on the reviewer-request/@-mention trigger paths here, rather than relying on the "likely doesn't apply" assumption in the issue body.

---

Stakeholders

No additional stakeholder beyond the issue's own author/assignee was identified — git blame/git log on the related workflow files (issue_enrichment.yml, issue_enrichment.md) show @mekarpeles as the sole contributor to the pattern this issue builds on, and no Lead: label currently exists for the Theme: Bots area.

---

Full triage checklist

• Triage — correct labels applied or removed
• Problem / opportunity — problem is clear, audience identified, actionable
• Scope — issue is well-scoped; does not try to do too many things
• Justification — reasoning and impact stated; sufficient for leads to prioritize (qualitative only — no data on review-coverage gap or turnaround-time impact since Copilot credits ended)
• Success & testing criteria — defines what "done" looks like, verifiable by a contributor
• Proposal & breakdown — concrete plan present; checklist actionable for a contributor (design explicitly gated on the two open research questions)
• If bug: environment & repro steps — N/A, this is a feature request
• Risks, concerns, open questions — identified and surfaced
• Richy context — relevant code, commits, PRs reviewed; no duplicate found
• References — screenshots, designs, docs, and links present where needed

---

Note

This comment was automatically generated by Richy, Open Library's issue enrichment assistant, on behalf of @mekarpeles. This is NOT a sign-off. This issue needs lead review before work begins.

[mekarpeles]

Opened #13167 — trigger scaffold only (reviewer-request + staff @-mention paths, both staff-gated, openlibrary-bot explicitly excluded by login). Posts a NOP placeholder comment to verify the wiring independently of the review-generation work, which is still blocked on the open research questions above. Does not close this issue.

[mekarpeles]

Updated #13167 per Mek's 3 refinements: draft PRs never trigger review (any path); non-draft PRs auto-get openlibrary-bot requested as reviewer if missing (auto_assign_reviewer.yml); PRs authored by openlibrary-bot itself still get reviewed, via a direct trigger rather than a reviewer-request — GitHub's API rejects requesting a PR's own author as a reviewer (422), so that case can't literally go through the 'add as reviewer' path. Split into two workflows as suggested: one that auto-assigns the reviewer, one that reacts to the assignment (or the self-authored/mention cases) and does the (still-NOP) review.