Skip to content

Commit 70d7538

Browse files
Pigbibicodex
andauthored
fix: make PR review contract checks holistic (#89)
Co-authored-by: Codex <noreply@openai.com>
1 parent d0141e4 commit 70d7538

3 files changed

Lines changed: 24 additions & 1 deletion

File tree

prompts/pr_review.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,12 @@ You are reviewing a pull request for a **production quantitative trading and dat
1818
- Minor refactoring opportunities
1919
- Test coverage suggestions
2020

21+
## Review completeness
22+
23+
- Review the entire diff holistically and report all independent actionable findings in one response. Do not stop after the first blocking issue.
24+
- Do not invent backward-compatibility requirements that are absent from the repository and PR contract. If both explicitly define a clean-slate namespace, check for accidental legacy fallback instead of requesting dual-read or migration. This never overrides security or data-integrity findings.
25+
- For public JSON/wire contracts, systematically check optional-key presence versus explicit null, recursive JSON-safe types, every identity-bearing integer range, one canonical timestamp representation, deterministic encode/decode round-trips and digests, deep immutability, and identifier/path safety.
26+
2127
## Severity definitions
2228

2329
| Severity | Definition | Example |

scripts/run_codex_pr_review.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -325,7 +325,10 @@ def build_review_prompt(diff: str, pr_title: str, pr_body: str, repo: str) -> st
325325
1. Focus on **security vulnerabilities, logic errors, data corruption, crash bugs, race conditions, and API compatibility breaks**.
326326
2. Do NOT flag: code style, formatting, naming suggestions, minor refactoring preferences, or documentation issues.
327327
3. Do not emit a finding that concludes no code change is needed. For OIDC, `job_workflow_ref` is absent for explicit direct callers; flag a bypass only when a non-direct repository can reach the direct-caller path despite the allowlists.
328-
4. For each finding, classify its severity:
328+
4. Review the entire diff holistically and report all independent actionable findings in one response. Do not stop after the first blocking issue.
329+
5. Do not invent backward-compatibility requirements that are absent from the repository and PR contract. When the repository and PR explicitly define a clean-slate namespace with legacy compatibility out of scope, review that boundary for accidental fallback instead of requesting dual-read or migration. This never overrides security or data-integrity findings.
330+
6. For public JSON/wire contracts, systematically check optional-key presence versus explicit null, recursive JSON-safe types, every identity-bearing integer range, one canonical timestamp representation, deterministic encode/decode round-trips and digests, deep immutability, and identifier/path safety.
331+
7. For each finding, classify its severity:
329332
- **critical**: security vulnerability, data loss, production crash
330333
- **high**: logic error that produces wrong results, API break, memory/connection leak
331334
- **medium**: missing error handling, performance degradation, race condition

tests/test_run_codex_pr_review.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,20 @@ def test_review_prompt_states_direct_oidc_contract(self) -> None:
3636
self.assertIn("`job_workflow_ref` is absent for explicit direct callers", prompt)
3737
self.assertIn("Do not emit a finding that concludes no code change is needed", prompt)
3838

39+
def test_review_prompt_requires_holistic_contract_review(self) -> None:
40+
prompt = run_codex_pr_review.build_review_prompt(
41+
"diff",
42+
"clean-slate contract",
43+
"Legacy compatibility is explicitly out of scope.",
44+
"org/repo",
45+
)
46+
self.assertIn("report all independent actionable findings in one response", prompt)
47+
self.assertIn("Do not stop after the first blocking issue", prompt)
48+
self.assertIn("clean-slate", prompt)
49+
self.assertIn("optional-key presence versus explicit null", prompt)
50+
self.assertIn("every identity-bearing integer", prompt)
51+
self.assertIn("one canonical timestamp representation", prompt)
52+
3953
def test_review_script_never_imports_from_the_pr_checkout(self) -> None:
4054
source = Path(run_codex_pr_review.__file__).read_text(encoding="utf-8")
4155
self.assertNotIn("SOURCE_ROOT = BRIDGE_ROOT.parent / \"source\"", source)

0 commit comments

Comments
 (0)