Skip to content

qa-review: tune the review job, and record that it is on hold (with why and how to resume) - #118

Merged
lihsaa591 merged 7 commits into
masterfrom
feat/qa-review-tune
Oct 9, 2026
Merged

lihsaa591 merged 7 commits into
masterfrom
feat/qa-review-tune

Conversation

@lihsaa591

Copy link
Copy Markdown
Contributor

Summary

Merges the tuning of the manual-only QA review job and a CHANGELOG entry recording the decision to put it ON HOLD. Nothing triggers it automatically (no n8n flow, no webhook).

Tuning (verified on real public PRs #1428, #1393, #1392 of themegrill/user-registration, dry runs only):

  • findings capped by change size in code; tighter prompt
  • strict structured output (fixes the model dropping the test title key; optional 4th param on chatJSONWithUsage, existing Crisp/wp.org callers unchanged)
  • no "none found" when risks were raised but failed verification; discard reasons logged for public repos only
  • allowlist: themegrill/user-registration (public)

Hold decision: the reviewer never runs the plugin and overlaps Copilot / security-review / PHPCS; claudegrill already covers deterministic e2e and its agent tier is Claude-based and deliberately off. CHANGELOG has the reasons and a concrete resume path (local spike first, private repo for any agent run because this repo is public).

Testing

  • npm test: 57/57.
  • Live runs from this branch on #1428, #1393, #1392 (about 11 runs of the latter).

Not changed / known

  • Test-title matching is still exact (case-sensitive); a near-miss like different capitalization is discarded. Left for if this resumes.

@lihsaa591
lihsaa591 merged commit 2a1c815 into master Oct 9, 2026
@lihsaa591
lihsaa591 deleted the feat/qa-review-tune branch October 9, 2026 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant