fix(triage): accept note-only evidence; raise research max-turns to 200 - #97
Conversation
The triage handler's verdictSchema required every evidence[] entry to have a non-empty `file` string. Cross-cutting observations (negative grep results, "no usage anywhere" claims) legitimately have no single file pointer, and on issue #76 the agent produced a VALID verdict that was thrown away (11 turns, 93s, $0.575) when two such entries failed validation. Make `file` optional but require at least one of `file` or `note` via .refine() so empty {} still fails. Update the prompt example + rules to match. Bundles a small unrelated bump: research workflow `--max-turns` 80→200.
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (4)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Review rate limit: 0/1 reviews remaining, refill in 31 minutes and 35 seconds.Comment |
There was a problem hiding this comment.
Pull request overview
This PR makes the triage workflow more tolerant of valid note-only evidence entries and increases the scheduled research workflow's turn budget so longer investigations can complete. It fits into the existing bot/workflow infrastructure by adjusting agent-facing contracts rather than changing core orchestration behavior.
Changes:
- Relax the triage verdict schema so
evidence[]entries can be note-only, while still rejecting empty evidence objects. - Add regression tests covering accepted note-only evidence and rejected evidence entries missing both
fileandnote. - Raise the scheduled research workflow's Claude turn cap from 80 to 200 and reflow the manual input description.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/workflows/handlers/triage.ts |
Updates triage verdict validation and agent prompt guidance for note-only evidence. |
test/workflows/handlers/triage.test.ts |
Adds regression coverage for the new evidence-schema behavior. |
.github/workflows/research.yml |
Increases research workflow turn budget and reformats the dispatch input description. |
…n budget Address two Copilot findings on PR #97: - triage.ts: standing rule said 'a claim without a file:line citation is a guess' which contradicted the new note-only allowance and would push the agent back to the old behavior. Restate as: prefer file:line citations; a claim with neither a citation nor a concrete cross-cutting observation is a guess. - research.yml: embedded prompt still told the agent it had ~80 turns, even though --max-turns was raised to 200. Updated to ~200 to match.
# [1.9.0](v1.8.0...v1.9.0) (2026-05-03) ### Bug Fixes * **checkout:** fetch PR base branch so origin/<baseBranch> resolves (closes [#74](#74)) ([#96](#96)) ([71f83a6](71f83a6)) * **fetcher:** paginate GraphQL connections + MAX_FETCHED_* caps (closes [#66](#66)) ([#95](#95)) ([f728ecd](f728ecd)) * **triage:** accept note-only evidence; raise research max-turns to 200 ([#97](#97)) ([3b6036c](3b6036c)) * **workflow:** fix release.yml ([#98](#98)) ([cb43d69](cb43d69)) ### Features * **workflows:** publish SLSA provenance + SBOM attestations on every release tag (closes [#58](#58)) ([#94](#94)) ([95856bc](95856bc))
|
🎉 This PR is included in version 1.9.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
Summary
Two unrelated but small changes shipping together.
1. Triage schema no longer discards valid verdicts over note-only evidence. On issue #76 (daemon log:
5c88cc8f7b-5nztr), the triage agent produced a VALID verdict in 11 turns / 93s / $0.575 with a real citation atsrc/orchestrator/ws-server.ts:52, then was thrown away byverdictSchemabecause two evidence entries (cross-cutting observations like a negativegrep timingSafeEqual src/result) had nofile. The whole run was discarded with a generic "see server logs" message and the ship cascade halted at step 0. Per the team's brief ("any workflow shouldn't fail due to strict enforcing"),evidence[].fileis now optional, with a.refine()keeping the schema honest by requiring at least one offileornote. The prompt is updated to match so the agent knows note-only entries are acceptable for cross-cutting evidence whilefile+lineremains preferred.2. Research workflow
--max-turnsraised from 80 → 200. Daily research runs were hitting the 80-turn cap before producing usable issues. Plus a YAML reflow on thefocus_areadescription.Diagram
```mermaid
flowchart TD
Agent["agent writes
TRIAGE_VERDICT.json
w/ note-only entry"] --> Before["BEFORE
z.string.min1
file required"]:::bad
Before --> Reject["whole run discarded
cost wasted
ship halted"]:::bad
Agent --> After["AFTER
file optional
refine: file or note"]:::good
After --> Accept["valid verdict accepted
cross-cutting evidence kept
empty objects still rejected"]:::good
classDef good fill:#22863a,stroke:#0b3d18,color:#ffffff
classDef bad fill:#b31d28,stroke:#5b0a0e,color:#ffffff
```
Changes
Related Issues
Test plan