Pin Bifrost v0.10.8, holding the usage-scan deadline in the runner - #178
Conversation
Bifrost v0.10.8 removed `max_duration_secs` from the scan tools and now rejects
it outright:
code -32602: scan_usages_by_location does not accept `max_duration_secs`;
deadline policy belongs to the frontend
The runner sent it on every scan, so under v0.10.8 each usage scan failed and
every case with a declaration-to-usages scan reported `error`. Since the freeze
gates require `errors == 0`, a v0.10.8 freeze would not have produced a wrong
number -- it would have failed outright.
The removal is deliberate on the Bifrost side: its own test asserts the
property is absent from the tool schema, and `ScanUsagesExecutionContext` no
longer carries a wall clock at all, only a cancellation token and the file,
byte, and callsite caps. So there is no implicit server budget left to inherit,
and the frontend holding its own clock is the intended arrangement.
`ToolClient` gains `set_request_timeout`, defaulting to a no-op so clients with
no clock of their own are unaffected. `McpSession` honours it for subsequent
requests and clamps it to the existing ten-minute process envelope, which stays
the outer bound. The Bifrost runner sets it from
`--scan-usages-max-duration-secs` before each scan and stops sending the
argument. The flag, its recorded invocation metadata, and the 300-second
default all keep their meaning.
One semantic does change, and the docs now say so. A budget expiry used to come
back as structured incomplete evidence from the server; the only lever an MCP
client has now is to stop waiting, so an exhausted budget aborts the call and
reports a case error instead of scoring truncated evidence. For a benchmark
that is the safer direction, and the freeze gates surface it loudly.
Verified against v0.10.8: `rust-struct-construction` returns the same usage set
as under v0.10.6 -- 4 true positives, 2 false negatives, `rawStatuses: ["found"]`
-- so dropping the argument costs no coverage.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`v0.10.8` is `8ddf1362`. The revision moves in the same five places as previous bumps, for the same reasons: reference-image.sh refuses to build when the candidate registry and the reference-environment manifest disagree, the reproduction contract and the freeze-shard test assert the exact pair, and the historical v1 page names the active candidate so readers know its own tables are older evidence. The v0.10.2 pin for candidates-v0.3.0.json is deliberately untouched; that registry belongs to a frozen release. This pin depends on the preceding commit. v0.10.8 rejects the per-request scan deadline the runner used to send, so without that fix every case carrying a usage scan reports `error`. Behavioural verification did complete this time, against the repository tree rather than a staged corpus -- staged runs on this machine remain broken for reasons established earlier and unrelated to any release. The proxy was validated when v0.10.5 reproduced its staged CI result exactly. The two Rust cases in #170 are unchanged under v0.10.8, measured with a v0.10.6 control on the same tree in the same session: rust-struct-construction failed under both, missing the capital-Self usages at src/service.rs:20:49 and 21:9 rust-parity-module-declaration-definition failed under both; v0.10.8 answers src/lib.rs:1:1 where the case authors 1:9, so module navigation returns the file anchor rather than the `mod` declaration name The control matters: both cases report `failed` with `rawStatuses: ["found"]` under v0.10.6, which is exactly what #170 recorded, so the environment is clean and the earlier all-`error` reading was the rejected argument alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Added a third commit retiring the published semantic-pack cases, closing #180. Why retirement rather than a republishThe republish was tried first and got one gate further before failing. The v0.10.8 requires schema 2 on exact equality ( The published-pack surface was a casualty of the open-core migration rather than What is deliberately kept
Effects
No promotion or cohort artifact moves — the legacy cohort already skipped these Local: |
1 participant
v0.10.8 cannot be pinned on its own: it removed
max_duration_secsfrom thescan tools and now rejects it.
The runner sent it on every scan, so under v0.10.8 every case with a
declaration-to-usages scan reported
error. The freeze gates requireerrors == 0, so this would not have produced a wrong number — the freezewould have failed outright. Two commits, fix first.
1. Hold the deadline in the runner
The removal is deliberate upstream: Bifrost's own test asserts the property is
gone from the tool schema, and
ScanUsagesExecutionContextno longer carries awall clock at all — only a cancellation token and the file, byte, and callsite
caps. There is no implicit server budget left to inherit, so the frontend
holding its own clock is the intended arrangement.
ToolClientgainsset_request_timeout, defaulting to a no-op so clientswith no clock of their own are unaffected.
McpSessionhonours it and clamps to the existing ten-minute processenvelope, which stays the outer bound.
--scan-usages-max-duration-secsand stopssending the argument.
The flag, its recorded invocation metadata, and the 300-second default all keep
their meaning.
One semantic does change, and the docs now say so: a budget expiry used to
return structured incomplete evidence from the server. The only lever an MCP
client has now is to stop waiting, so an exhausted budget aborts the call and
reports a case error rather than scoring truncated evidence. For a benchmark
that is the safer direction, and the gates surface it loudly.
Verified no coverage is lost — under v0.10.8 the scan returns the same set as
under v0.10.6: 4 TP, 2 FN,
rawStatuses: ["found"].2. Pin to v0.10.8
8ddf1362, moving the same five places as previous bumps.candidates-v0.3.0.jsonstays on v0.10.2 — that registry belongs to a frozenrelease.
#170 is unchanged under v0.10.8
Measured on the repository tree with a v0.10.6 control, same machine, same
session:
rust-struct-constructionfailed— missingSelfatsrc/service.rs:20:49,21:9rust-parity-module-declaration-definitionfailedfailed— answerssrc/lib.rs:1:1where the case authors1:9The second confirms #170's hypothesis: module navigation returns the file
anchor rather than the
moddeclaration name.The control is the point. Both report
failedwithrawStatuses: ["found"]under v0.10.6, exactly what #170 recorded — so the environment is clean and the
earlier all-
errorreading was the rejected argument alone, not a regression.#170 stays open and stays Bifrost-side.
Staged-corpus runs on this machine remain broken for reasons established
earlier and unrelated to any release; the repository-tree proxy was validated
when v0.10.5 reproduced its staged CI result exactly.
Verification
cargo test --locked— 247 passed, 0 failedpython3 -m unittest discover tests— 42 tests, OKvalidate-reproduction-contract.sh— exit 0🤖 Generated with Claude Code