Free-tier and BYOK model resilience on 404/402

[mvanhorn]

Issue

None.

Version and model info

None.

Problem

Free-tier and BYOK users hit vanishing model IDs and 402/credit errors. Onboarding still pins openrouter/deepseek/deepseek-r1:free. Related issues (#4289, #3264, #3936) show the pain; none implement a live catalog probe + explicit fallback when the selected model hard-fails.

Proposed approach

1. Centralize failure classification in exceptions.py / models send path.
2. Add suggest_free_fallback(model) using OpenRouter manager.
3. Hook from base coder retry path; update onboarding default to "resolve at runtime" rather than a single pinned ID.
4. Tests with fixtures for 404/402 and catalog stubs.

I'm happy to open a PR implementing this if the direction works for you.

[seven7763]

Wire-level data points from running a multi-model gateway, in case they sharpen the design:

1. 404 and 402 are different failure classes and want different handling.

• 404 on a chat request usually means the model ID isn't in the account-scoped catalog — the static litellm model list drifts from what any given provider account can actually reach (retired IDs, region/account gating, :free variants that vanish without notice). A live GET /v1/models is the only reliable source, but with a caveat: some gateways return their full catalog rather than the account-scoped subset, so membership in /models is necessary, not sufficient — the robust probe is GET /v1/models + one minimal completion against the selected ID.
• 402/credit errors won't be fixed by switching model IDs on the same key — they need a different action (top-up notice, or provider switch if the key spans providers). Classifying both as "hard-fail → fallback" risks masking a billing problem as a model problem, which is exactly the confusing state users hit with pinned :free IDs.

2. For suggest_free_fallback: log the resolved model, not the requested alias.
A fallback chain that records "we asked for X and got an answer" can't tell you whether the degraded leg served the task well — the quiet failure mode is a fallback that answers but answers wrong (valid schema, hallucinated payload). Per-call attribution of which model actually served turns "why did this output look dumb" into a grep. (Same class of bug just bit the OpenAI Responses path in pydantic/pydantic-ai#8045: the terminal event carries the served model, the client keeps the router alias.)

3. Onboarding default "resolve at runtime" — +1, with one UX note.
Runtime resolution should surface why a model was chosen (catalog hit vs fallback suggestion), otherwise users blame the fallback's quality for what was a catalog miss.

FWIW: I run an OpenAI-compatible gateway (daoxe), so this failure taxonomy is a weekly occurrence on our side — nothing above depends on it; the steps apply to OpenRouter/litellm directly.