Skip to content

fix(ai): one session-layer retry owner for provider failures - #2045

Open
snimu wants to merge 4 commits into
mainfrom
sebastian/retry-owner-2026-09-04
Open

fix(ai): one session-layer retry owner for provider failures#2045
snimu wants to merge 4 commits into
mainfrom
sebastian/retry-owner-2026-09-04

Conversation

@snimu

@snimu snimu commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Makes the agent session's auto-retry loop the single owner of provider retries (Linear: RES-1271; discussion evidence: #1474, #1405, #1534, #1981-retry-half, #1553(3)).

Why

Retry policy lived in N+1 places that drifted independently:

What

Mechanism (one owner):

  • Every SDK client is constructed with retries off: OpenAI/Azure/Anthropic maxRetries: 0, Bedrock maxAttempts: 1. Google sets no retryOptions (SDK only retries when configured) and Mistral already used retries: { strategy: "none" } — verified, unchanged.
  • Codex's hand-rolled retry loop is deleted; a non-OK response throws a structured CodexApiError (code, status, retryAfterMs).
  • StreamFailureInfo gains retryAfterMs, extracted from Retry-After/retry-after-ms headers (seconds, ms, or HTTP-date) or Codex resets_at.
  • The session retry loop now waits max(backoff, retryAfterMs), capped by retry.provider.maxRetryDelayMs (default 60s, 0 disables): a longer server-requested wait fails immediately with an informative error — exactly the documented-but-dead semantics.
  • openai-completions and codex now record provider_stream_failure diagnostics like every other provider; codex parses nested error payloads into the friendly usage-limit message.
  • Structured invalid_request/refusal failures are permanent at attempt 0 (no more one pointless retry); auth keeps its single-retry hedge so a transient auth hiccup does not immediately mark auth stale.

Deletion:

  • Codex retry loop + isRetryableError + sleep helper.
  • Dead maxRetries/maxRetryDelayMs plumbing through StreamOptions, simple-options, Agent, streamProxy, sdk.ts, side-question, and the retry.provider.maxRetries setting (its only effect was re-enabling invisible SDK-internal retries).

Provider-local retry/backoff inventory (before → after)

Behavior Where After
OpenAI SDK internal retries, uncapped Retry-After sleeps openai-responses, openai-completions clients maxRetries: 0
Azure OpenAI SDK same azure-openai-responses client maxRetries: 0
Anthropic SDK same anthropic (4 client sites) maxRetries: 0
Per-request maxRetries override plumbing types/simple-options/4 request sites/settings deleted
Codex hand-rolled loop (4 attempts, retried deterministic 4xx) openai-codex-responses deleted; single attempt + structured error
Bedrock AWS SDK standard retry (3 attempts) BedrockRuntimeClient config maxAttempts: 1
Google GenAI pRetry (only if retryOptions set — never set) google, google-vertex unchanged (verified no retries)
Mistral SDK retries mistral already strategy: "none" (unchanged)
Codex websocket→SSE transport fallback openai-codex-responses kept (transport fallback before message start, not a retry/backoff)
Session _handleRetryableError (visible auto_retry events) agent-session the single owner; now honors capped Retry-After

Network-level failures previously retried inside providers are covered by the session loop, which retries any non-permanent error stop with visible auto_retry_start/end events.

Size

Net src LOC: +170/−142 across packages/ai, packages/agent, packages/coding-agent (mechanism ≈ +150 in Retry-After extraction/cap + structured codex errors; deletion ≈ −140 of SDK retry plumbing and the codex loop). Tests: 7 new pins, 1 reworked (the "retries structured permanent failures once" pin now asserts no retry for invalid_request/refusal).

Validation

  • Sandbox: root tsgo clean; packages/ai suite pass; packages/agent suite pass; packages/coding-agent suite matches the documented 82-test/14-file environmental baseline exactly (no new failures).
  • Targeted local runs: agent-session-retry-events, 4491, 3317, settings-manager, telemetry, model-registry all pass.

Linear: RES-1271


Note

Medium Risk
Changes retry semantics on every LLM provider path and removes SDK-level recovery; regressions would show up as extra failures or wrong backoff, but behavior is heavily pinned in new tests.

Overview
Provider retries are no longer handled inside SDKs or Codex’s old loop — each provider call is a single attempt that surfaces structured provider_stream_failure diagnostics (including retryAfterMs from Retry-After headers or Codex usage-limit resets_at).

The session auto-retry loop (and a new shared provider-retry module) becomes the only place that backs off and retries: delay is max(exponential backoff, provider retryAfterMs), capped by retry.provider.maxRetryDelayMs; longer waits fail fast with a clear error. invalid_request / refusal no longer get a retry; auth still gets one hedge retry.

Removed maxRetries / maxRetryDelayMs from stream options, Agent, proxy serialization, and retry.provider.maxRetries. Side questions, compaction, branch summarization, and refinement now use completeWithProviderRetry so they don’t silently become single-shot after SDK retry removal.

Reviewed by Cursor Bugbot for commit 0d2a6f0. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Consolidate provider retries into the session layer and cap retry delays

  • Configures all AI provider SDKs with zero internal retries, moving retry orchestration to the session layer via AgentSession and the new completeWithProviderRetry wrapper.
  • Introduces completeWithProviderRetry and ProviderRetryPolicy to apply shared retry settings to standalone completions: compaction, refinement, branch summaries, and side questions.
  • Adds HTTP status and Retry-After header parsing to StreamFailureInfo and CodexApiError, including extraction of nested usage-limit details from Codex SSE error events.
  • Calculates retry delays using the maximum of exponential backoff and provider-requested delays, stopping retries if the requested delay exceeds the configured maxRetryDelayMs cap.
  • Risk: maxRetries and maxRetryDelayMs fields removed from StreamOptions in types.ts, AgentOptions in agent.ts, and ProxySerializableStreamOptions in proxy.ts; the retry.provider.maxRetries setting is also removed from ProviderRetrySettings.

Macroscope summarized 0d2a6f0.

Providers now make exactly one attempt per request and report structured
failures; the agent session auto-retry loop is the only retry mechanism:

- OpenAI/Azure/Anthropic SDK clients are constructed with maxRetries: 0,
  Bedrock with maxAttempts: 1 (Google sets no retryOptions and Mistral
  already used retries: none). Their internal uncapped Retry-After sleeps
  were invisible to the UI and ignored retry.provider.maxRetryDelayMs.
- The Codex hand-rolled retry loop (which also retried deterministic 4xx)
  is deleted; HTTP errors throw a structured CodexApiError instead.
- openai-completions and codex now record provider_stream_failure
  diagnostics like every other provider; codex parses nested streaming
  error payloads (usage_limit_reached with plan/resets_at) into the
  friendly usage-limit message.
- StreamFailureInfo carries retryAfterMs extracted from Retry-After /
  retry-after-ms headers or usage-limit reset info; the session retry
  loop waits at least that long, capped by retry.provider.maxRetryDelayMs
  (fails fast with an informative error beyond the cap, 0 disables).
- Structured invalid_request/refusal failures are no longer retried once
  before being treated as permanent; auth keeps its single-retry hedge.
- The dead maxRetries/maxRetryDelayMs stream-option plumbing and the
  retry.provider.maxRetries setting are removed.
Comment thread packages/coding-agent/src/core/side-question.ts
…ssion loop

Review follow-up: side questions and the one-shot completeSimple consumers
(compaction summaries, branch summarization, refinement, auto-refine review)
never enter the session auto-retry loop, so setting SDK maxRetries to 0 left
them with zero retries anywhere.

New core/provider-retry.ts holds the single policy definition (structured
failure kind readers, permanent-kind rule, Retry-After-aware delay with the
maxRetryDelayMs cap) plus completeWithProviderRetry for one-shot calls; the
AgentSession loop now delegates to the same policy functions. Side questions
retry with the session's settings (drop failed assistant turn, re-run);
the utility completions retry with the default policy.

Intentionally not wrapped: the daemon agent-status summarizer - it is a
periodic best-effort call that naturally re-attempts on the next cycle.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 36c1426. Configure here.

Comment thread packages/coding-agent/src/core/provider-retry.ts
Comment thread packages/coding-agent/src/core/compaction/compaction.ts Outdated
Comment thread packages/coding-agent/src/core/provider-retry.ts Outdated
…ancelled

Review follow-up: completeWithProviderRetry swallowed an abort during the retry sleep and returned the previous provider error, so cancelling compaction/branch-summary/refinement mid-backoff surfaced as a failure instead of an abort.
…letions

Review follow-up: compaction summaries, branch summarization, and
refinement calls used the default retry policy, ignoring configured
retry.enabled/maxRetries/baseDelayMs/maxRetryDelayMs. The AgentSession
call sites now pass providerRetryPolicy(settingsManager) down, matching
side questions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant