Restore the usage-scan budget on the server that enforces it - #189
Merged
Conversation
The v0.3.2 evaluation freeze published 27 failures where v0.3.0 had 17. The
regression was ours, not Bifrost's: 23 of those cases carry
`scan_usages_incomplete: reason=time_budget` and a `partial` usage set.
`max_duration_secs` was never only a request argument; it was how the frontend
raised Bifrost's per-request analyzer budget. Removing the argument did not
remove the budget, it removed our ability to raise it, so every scan fell back
to Bifrost's cold-workspace default:
// crates/bifrost-mcp/src/mcp_common.rs
const COLD_WORKSPACE_REQUEST_BUDGET: Duration = Duration::from_millis(4_500);
4.5 seconds is a budget meant for interactive use. The fixture corpus never
noticed -- it is warm and small, about 1.5 seconds a case -- but a cold clone of
a real project cannot finish a scan inside it, and answers with a partial usage
set that scores as a failure rather than as the timeout it is.
The replacement is `BIFROST_MCP_REQUEST_BUDGET_SECS`, read when the server
starts. Deadline policy is still the frontend's; it is chosen once per server
rather than once per call. The runner now sets it from the requested scan
budget, so the flag, its recorded invocation metadata, and the 300-second
default all keep the meaning they had before v0.10.8.
Bifrost panics on a zero budget, so zero means "leave Bifrost's default alone"
rather than reaching the server as a literal zero.
The runner's own deadline becomes the budget plus a minute instead of matching
it exactly. Equal deadlines race, and the client winning that race is the worse
outcome: Bifrost's budget ends a long scan by returning structured incomplete
evidence, where the client deadline can only abandon the call.
Verified on the case this fix is for. `real-project-v2-rust-01-1` failed in
v0.3.2 with `reason=time_budget`; against a cold clone under v0.10.8 with this
change it passes with 9 true positives, no false positives or negatives, and
both queries exact. Analyzer query time was 3.2 seconds -- far inside 300, far
outside 4.5.
Refs #185
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
DavidBakerEffendi
added a commit
that referenced
this pull request
Sep 2, 2026
Prerequisite for the corrected real-project-v2 freeze. `freeze.yml:94` hard-asserts the citation version matches the snapshot version, so preflight fails until this lands. v0.3.3 supersedes v0.3.2, which measured the slice under Bifrost's 4.5-second cold-workspace budget and recorded 27 failures against v0.3.0's 17. #189 fixed the cause; this is the version metadata for re-running the freeze under it. Still a **patch**: the corpus, its 36 cases, and their reviewed ground truth are unchanged across all three freezes. Only harness behaviour moved. v0.3.2 stays published — releases are immutable by design, so a corrected measurement supersedes it rather than replacing it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
v0.3.2 is an invalid measurement. It published 27 failures where v0.3.0 had
17, and the regression is ours, not Bifrost's. 23 of those cases carry
scan_usages_incomplete: reason=time_budgetwith apartialusage set.Root cause
max_duration_secswas never only a request argument — it was how thefrontend raised Bifrost's per-request analyzer budget. Removing the argument
did not remove the budget; it removed our ability to raise it. Every scan then
fell back to Bifrost's cold-workspace default:
4.5 seconds is a budget for interactive use. This is the "five-second
interactive budget" the README used to warn about — still there, one layer
above
ScanUsagesExecutionContext, which is why it was missed when theargument was dropped.
The fixture corpus never noticed: warm, small, ~1.5s a case. A cold clone of a
real project cannot finish a scan in 4.5s and answers with a partial usage set,
which scores as a failure rather than as the timeout it is.
The v0.3.2 timings confirm it was never our clock:
reason=time_budget55 seconds of query time across the whole run — the 300s client deadline never
came close to firing.
The fix
BIFROST_MCP_REQUEST_BUDGET_SECS, read when the server starts, is thereplacement. Deadline policy is still the frontend's; it is chosen once per
server instead of once per call. The runner sets it from the requested scan
budget, so the flag, its recorded invocation metadata, and the 300-second
default keep the meaning they had before v0.10.8.
Two details worth review:
budget, so it must not travel as a literal zero.
Equal deadlines race, and the client winning is the worse outcome: Bifrost's
budget ends a long scan with structured incomplete evidence, where the client
deadline can only abandon the call.
Verified on the case it is for
real-project-v2-rust-01-1failed in v0.3.2 withreason=time_budget. Againsta cold clone under v0.10.8 with this change:
3.2 seconds — far inside 300, far outside 4.5.
Follow-up
it stays published and a corrected freeze supersedes it.
bad number.
the bad result was approved on expectation rather than evidence. Printing
pass/fail counts and a diff against the previous release into the job summary
would make that gate able to catch this.
Verification
cargo test --locked— 253 passed (3 new), 0 failedpython3 -m unittest discover tests— 42 tests, OKvalidate-reproduction-contract.sh— exit 0🤖 Generated with Claude Code