Gateway error clarity: local stop-limit rejection + output-ceiling clamp

[SilenNaihin]

Gateway error clarity: two shipped fixes, three scoped follow-ups

Each fix is independent (drop either commit). Both trace to failure modes reproduced end to end against the live gateway this week, and both are Python on the admission/reservation path that survives the Rust data-plane migration (#629). No provider content, secrets, or free-text bodies enter any error, ledger, or log.

Shipped

1. Reject over-limit stop lists locally with a named parameter error (3c40b4c3)
A route can support stop sequences yet cap the count — Gemini's direct lane accepts at most 5. preflight_gateway_request only checked the boolean supports_stop_sequences, so a longer list was forwarded to Google and returned the opaque provider rejected the request; verify the request fields 4xx.

• Reproduced in production: stop with 6 sequences 400s on the gemini direct lane; 5 passes (byte-identical to the incident rows).
• Adds maximum_stop_sequences to GatewayDeploymentCapabilities (default None = unbounded, non-breaking) and enforces it in preflight with ProviderParameterError(param="stop", code="too_many_stop_sequences") naming the limit — matching every other pre-dispatch parameter rejection.
• Before: forwarded → provider 400 → opaque invalid_request. After: local named rejection an agent can self-correct from.
• Follow-up: populating the per-lane value (gemini = 5) is a catalog-authoring step (set maximum_stop_sequences on the gemini deployment capability in the launch catalog).

3. Clamp the caller output ceiling to the deployment before the cost worst case (c44f052b)
maximum_attempt_cost_micro_usd used the raw caller maximum_output_tokens, so an unbounded/huge value inflated the worst case past MAXIMUM_MICRO_USD, returned None, and failed a fundable request closed — mis-terminalized as a quota refusal (the caller-driven twin of the catalog-capability None source fixed in platform #877).

• The physical call can never emit more than the deployment ceiling, so clamp to it before the estimate. Settlement still charges actual tokens; this only tightens a conservative bound.

Validation

ruff format/check clean, ty check clean, and 826 passed across exp/common/models, exp/runtime/gateway, exp/runtime/models/providers (incl. the two added regression tests). Nothing merged.

Scoped follow-ups (Rust data plane — not in this PR)

These are engine-owned in the native gateway and need the Rust build/parity-golden cycle; documented with exact targets rather than shipped unverified:

• Admission/internal cause tags (exp/runtime/gateway/native/src/admission.rs): the admit and reserve seams both surface PublicError::internal() ("gateway admission failed before provider dispatch") with no way to tell authority vs accounting vs wire-resolution apart. Add a coarse content-free cause tag. (M4, reproduced exactly via bounded fault injection.)
• Fail-fast on unresponsive routes (exp/runtime/gateway/native/src/upstream.rs): dead lanes hang ~25s to the prod ceiling before failing (glm-5-turbo, qwen3.8-27b, others). Add a bounded connect/first-byte timeout — scoped to connect/first-byte, not total generation, so slow reasoning models are unaffected. This is genuinely a tuning knob: recommend a config value with a conservative default rather than a hardcode.
• Generic 4xx carries the provider machine code (upstream.rs, mirrored in exp/runtime/models/providers/errors.py): the generic invalid_request mapping discards the provider's sanitized status/machine code that's already available — include the code/param (still no free-text body) so agents self-correct. (Same family as fix Skeleton: LLM-as-environment world model harness #1; the live 4xx for direct lanes is emitted in Rust.)

Note on scope vs. the original request

The brief named Python symbols (gemini_requests.py, the transport classifier) as the serving encoders. Post-#629 the live data plane is Rust: GatewayWireProfile's own docstring confirms the native plane builds provider payloads and parses streams, so the outbound stop-sequence emission and generic 4xx mapping live in Rust (native/src/), while the Python provider modules serve the resolved-client/offline path. Fix #1 is therefore placed in preflight_gateway_request — the Python admission guard that runs before dispatch regardless of encoder — which is both correct and encoder-independent. Fixes #2/#4/#5 are Rust and listed above.

🤖 Generated with Claude Code

[greptile-apps]

Greptile Summary

The PR adds an optional deployment-level stop-sequence limit with a named local parameter error and makes attempt-cost estimation clamp caller output limits to the deployment ceiling.

• Adds maximum_stop_sequences to deployment capabilities and enforces configured limits during gateway preflight.
• Clamps worst-case output-token cost calculations to physical deployment capacity.
• Adds focused regression coverage for both behaviors.

Confidence Score: 5/5

The PR appears safe to merge with no actionable correctness or security failures identified.

Native admission rejects output limits above the resolved route capacity before reservation and dispatch, while the new optional stop limit remains backward compatible and is enforced whenever catalog authors configure it.

Important Files Changed

Filename	Overview
exp/common/models/catalog.py	Adds a backward-compatible optional stop-sequence count capability with positive-value validation.
exp/runtime/models/providers/protocol.py	Rejects configured over-limit stop lists locally with a stable parameter-specific error.
exp/runtime/models/providers/protocol_test.py	Covers over-limit rejection, at-limit acceptance, and the default unbounded behavior.
exp/runtime/gateway/budgets.py	Uses the deployment output ceiling when calculating the conservative maximum attempt cost; native admission independently prevents larger limits from reaching dispatch.
exp/runtime/gateway/budgets_test.py	Verifies huge caller output limits remain bounded by deployment capacity instead of producing an unknown cost.

_{Reviews (1): Last reviewed commit: "Gateway: clamp caller output ceiling to ..." | Re-trigger Greptile}