Problem
OCR is currently pinned to concurrency 2 (down from the upstream default of 8). This was set conservatively, but the measured cost is significant:
| Run |
Concurrency |
Files reviewed |
OCR step time |
| actions/runs/29921065524 |
8 (default) |
62 |
13:44 |
| actions/runs/29923648413 |
2 (explicit) |
62 |
49:59 |
| actions/runs/29946076923 |
2 (explicit) |
63 |
46:31 |
The 3.6x slowdown correlates with the 4x concurrency reduction.
StepFun constraint
Our LLM provider (StepFun) caps concurrent API requests at 5 per account. This is a per-account limit, not per-session — multiple simultaneous OCR runs across different PRs share the same budget.
OCR's --concurrency flag controls file-level parallelism (number of files reviewed simultaneously). Each concurrent file review makes independent LLM API calls. OCR also has auxiliary LLM calls (planning, fact-checking, summarization) that add to the concurrent load.
Concurrency budget analysis
| OCR concurrency |
1 PR active |
2 PRs active |
3 PRs active |
| 2 |
~2-3 requests (safe) |
~4-6 (borderline) |
~6-9 (exceeds) |
| 3 |
~3-4 (safe) |
~6-8 (exceeds) |
~9-12 (exceeds) |
| 4 |
~4-5 (borderline) |
~8-10 (exceeds) |
— |
| 5 |
~5+ (at limit) |
— |
— |
Key insight
StepFun does not increase its concurrency limit because there are multiple PRs. Two simultaneous OCR runs at concurrency 3 each would attempt 6 concurrent requests against a limit of 5.
Proposed experiment
Controlled concurrency canary with StepFun awareness:
Experiment design
- Pick a single PR with a known diff size (~60 files, similar to the reference runs).
- Lock OCR version, model, rules, and base ref.
- Run OCR at concurrency 2, 3, and 4 on the same PR head (via manual
/ocr trigger to avoid interference).
- For each run, record:
- Wall-clock OCR step time.
- Total tokens consumed.
- Number of 429/rate-limit responses (if any).
- Number of retry/backoff events.
- Finding count and category distribution (to verify quality is not affected).
- Between runs, verify no other OCR runs are active on other PRs.
Decision criteria
- If concurrency 3 completes without 429s and shows >30% speedup over concurrency 2: adopt 3 as the new default.
- If concurrency 3 triggers 429s: keep 2 and investigate whether StepFun's limit can be raised.
- If concurrency 4 is significantly faster than 3 without 429s: consider 4 for single-PR scenarios, with a guard that reduces to 2 when concurrent runs are detected.
Future: dynamic concurrency
The ideal solution is dynamic: detect the current StepFun load (or number of concurrent OCR runs) and adjust --concurrency accordingly. This is not possible with GitHub Actions today, but once the auto-review limit (#2666) reduces simultaneous runs, concurrency 3 becomes safer because fewer PRs will trigger OCR simultaneously.
Acceptance criteria
Relationship to other issues
Problem
OCR is currently pinned to concurrency 2 (down from the upstream default of 8). This was set conservatively, but the measured cost is significant:
The 3.6x slowdown correlates with the 4x concurrency reduction.
StepFun constraint
Our LLM provider (StepFun) caps concurrent API requests at 5 per account. This is a per-account limit, not per-session — multiple simultaneous OCR runs across different PRs share the same budget.
OCR's
--concurrencyflag controls file-level parallelism (number of files reviewed simultaneously). Each concurrent file review makes independent LLM API calls. OCR also has auxiliary LLM calls (planning, fact-checking, summarization) that add to the concurrent load.Concurrency budget analysis
Key insight
StepFun does not increase its concurrency limit because there are multiple PRs. Two simultaneous OCR runs at concurrency 3 each would attempt 6 concurrent requests against a limit of 5.
Proposed experiment
Controlled concurrency canary with StepFun awareness:
Experiment design
/ocrtrigger to avoid interference).Decision criteria
Future: dynamic concurrency
The ideal solution is dynamic: detect the current StepFun load (or number of concurrent OCR runs) and adjust
--concurrencyaccordingly. This is not possible with GitHub Actions today, but once the auto-review limit (#2666) reduces simultaneous runs, concurrency 3 becomes safer because fewer PRs will trigger OCR simultaneously.Acceptance criteria
Relationship to other issues