Skip to content

fix(runner): record a failed run when the provider name is unknown - #10

Merged
sturlese merged 1 commit into
mainfrom
fix/bughunt-unknown-provider-crashes
Jul 12, 2026
Merged

fix(runner): record a failed run when the provider name is unknown#10
sturlese merged 1 commit into
mainfrom
fix/bughunt-unknown-provider-crashes

Conversation

@sturlese

Copy link
Copy Markdown
Owner

Bug

A model whose provider field names no registered adapter (e.g. provider: cohere) is valid, routable config — ModelSpec.provider is an unconstrained str, so it passes loading, the data-policy gate, and routing. When such a model is selected, execute() crashed with an uncaught ProviderError instead of recording a failed run.

Root cause

runner.execute() resolved the adapter with active_provider = provider or get_provider(spec.provider) outside the try/except ProviderError block that converts vendor errors into recorded failed runs. get_provider raises ProviderError for an unknown name, so the exception propagated out of execute() — no store row, no run_failed ledger event, just a traceback. That violates the runner's own contract:

Every path lands in the store AND the ledger: completed runs with cost and usage, blocked runs with the gate that stopped them, failed runs with the vendor error.

The existing test_provider_failure_is_a_failed_run_not_a_crash only covers a resolved provider whose complete() raises — it passes a FailingProvider instance, so get_provider is never exercised. The unknown-provider path was unpinned.

Fix

Move the provider resolution inside the existing try so an unknown provider is caught and recorded via the same _terminal("failed", ...) path as any other vendor error. One-line move; the complete() failure path is unchanged.

Test

test_unknown_provider_is_a_failed_run_not_a_crash: a routable model with provider: cohere, run offline → asserts status == "failed", the reason names cohere, and both the store row and the last ledger event (run_failed) are recorded. It raises ProviderError on main and passes with the fix (verified via git stash).

Full suite: 162 passed, 92.18% coverage (gate 85%); ruff clean; offline flightdeck demo + audit verify intact.

Found and fixed by an autonomous bughunt iteration.

`execute()` resolved the provider adapter with
`get_provider(spec.provider)` OUTSIDE the try/except that turns vendor
errors into recorded failed runs. A model whose `provider` names no
registered adapter is valid, routable config (`ModelSpec.provider` is an
unconstrained str) that clears policy and routing — then `get_provider`
raised `ProviderError` uncaught, crashing the whole call.

That violates the runner's contract ("Every path lands in the store AND
the ledger ... failed runs with the vendor error"): no store row, no
`run_failed` event, just a traceback. Move the resolution inside the try
so an unknown provider is recorded as a failed run like any other vendor
error. The `complete()` failure path is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@sturlese

Copy link
Copy Markdown
Owner Author

bughunt gate — /code-review high ✅ clean

Reviewed across 8 angles (line-by-line, removed-behavior, cross-file tracer, reuse, simplification, efficiency, altitude, CLAUDE.md conventions) — no correctness or quality findings.

  • The except ProviderError block references only variables bound before the try (spec, tokens_in/out=0, redactions, clock) — no NameError; spec.cost(0, 0) == 0.
  • Happy path and the existing complete()-failure path are unchanged; active_provider is used only inside the loop within the try, so no possibly-unbound risk.
  • Callers cli.run / cli.tick already handle a failed return (exit 1 / exit 0) — they previously crashed on the uncaught traceback; strictly better.
  • Regression test fails on main (raises ProviderError) and passes with the fix.

Full local gate: 162 passed, 92.18% coverage, ruff clean, offline demo intact. Full mode: watching CI to merge on green.

@sturlese
sturlese merged commit 64f466b into main Jul 12, 2026
5 checks passed
@sturlese
sturlese deleted the fix/bughunt-unknown-provider-crashes branch July 12, 2026 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant