Skip to content

feat: add max_retries to the messages handler factories - #214

Open
aknight-ld wants to merge 1 commit into
mainfrom
AIC-3084-max-retries-passthrough
Open

aknight-ld wants to merge 1 commit into
mainfrom
AIC-3084-max-retries-passthrough

Conversation

@aknight-ld

@aknight-ld aknight-ld commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Add in a max retries passthrough argument to let handler factories natively retry.


Note

Overview
Adds an optional max_retries knob on the messages handler factories (Claude, OpenAI, LangChain, LiteLLM) and their convenience wrappers, so callers can raise provider-level retries for bursty workloads (e.g. concurrent eval workers) without touching model.parameters.

Shared validate_max_retries in launchdarkly_ai_server enforces None or a non-negative int at handler construction (including rejecting bool). Omitted values leave each SDK’s default unchanged; model.parameters cannot override the handler setting.

Wiring differs by integration: native Anthropic/OpenAI clients get max_retries only when set; LangChain applies it to auto-built chat models and errors if combined with a custom llm; LiteLLM picks max_retries vs num_retries per route (OpenAI/Azure vs everything else) so retries stay on one layer.

Reviewed by Cursor Bugbot for commit 4bed864. Bugbot is set up for automated code reviews on this repo. Configure here.

@aknight-ld
aknight-ld force-pushed the AIC-3084-max-retries-passthrough branch from 3072cbf to 4bed864 Compare October 27, 2026 14:24
@aknight-ld
aknight-ld marked this pull request as ready for review October 27, 2026 15:17
@aknight-ld
aknight-ld requested a review from a team as a code owner October 27, 2026 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant