feat(routing): EndpointModeAuto for runtime protocol auto-detection - #1137
Open
0x0079 wants to merge 3 commits into
Open
feat(routing): EndpointModeAuto for runtime protocol auto-detection#11370x0079 wants to merge 3 commits into
0x0079 wants to merge 3 commits into
Conversation
4 tasks
0x0079
force-pushed
the
feat/endpoint-mode-auto
branch
5 times, most recently
from
June 11, 2026 13:13
02f1a8c to
3f9ef19
Compare
0x0079
force-pushed
the
feat/endpoint-mode-auto
branch
2 times, most recently
from
June 16, 2026 07:03
3d33661 to
112a94a
Compare
Adds an "auto" OpenAI endpoint mode: for OpenAI-compatible providers the gateway tries the scenario-preferred protocol first, falls back to the alternate on a retryable failure, and caches the winning protocol per provider+model. Gated behind the global "auto_endpoint" experimental flag. Rebased onto main and aligned with main's structure: endpoint code lives in package server (protocol_endpoint.go + endpoint_auto.go/endpoint_cache.go), not a separate internal/endpoint package — following main's housekeeping consolidation. The auto path nests as protocol-fallback (outer) over provider-failover (inner) sharing a single first-chunk gate, via the new dispatchWithPriorityFailoverGated variant. - ai: EndpointModeAuto + IsAutoEndpointMode; non-Codex OAuth defaults to auto - server: resolveAutoTarget (override→cache→scenario), dispatchWithAutoFallback, EndpointCache; wired into chat + responses handlers via runOpenAI*Attempt - protocol_endpoint: ResolveOpenAIEndpoint handles Auto (mirror incoming) as the non-auto fallback - probe: E2E probe honors the endpoint cache - client: IsNonRetryableForProtocolSwitch guards pointless protocol swaps - config: auto_endpoint scenario flag - frontend: experimental toggle + i18n (en/zh) https://claude.ai/code/session_01N9JnUTVt41NcGxYyj3xewf
0x0079
force-pushed
the
feat/endpoint-mode-auto
branch
from
July 1, 2026 16:36
8583e6b to
6a54bbf
Compare
- Extract autoDispatchOrFailover: the ~25-line "auto vs plain failover" decision was duplicated verbatim in openai_chat.go and openai_responses.go, differing only by the IncomingAPIType constant and inner attempt body. Both call sites now share one helper. - Drop the ok bool from autoDispatchFn. Every real call site returned ok=true unconditionally (dispatchWithPriorityFailoverGated never yields a nil provider), so served != nil alone was already the complete success signal; ok was a vestige of an earlier direct-transform design. No behavior change; verified via existing endpoint_auto_test.go suite. https://claude.ai/code/session_01N9JnUTVt41NcGxYyj3xewf
…point endpoint_auto.go, endpoint_cache.go and their tests were split out of the same feature (OpenAI endpoint resolution + auto-detection) that already lives in protocol_endpoint.go. Four files for one cohesive concern was unnecessary fragmentation — folded them back in, organized under section banners (static resolution / cache / runtime auto-detection dispatch). No code changes beyond the move; verified via full endpoint_auto_test.go suite now inlined in protocol_endpoint_test.go. https://claude.ai/code/session_01N9JnUTVt41NcGxYyj3xewf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
autooption forOpenAIEndpointModethat detects per-model protocol support at runtime — no static config needed for aggregator providers (OpenRouter, SiliconFlow, etc.) hosting models with mixed Chat Completions / Responses API supportEndpointModeAuto, using the same cache for consistent behaviorautoas a visible optionDesign
EndpointCache(provider+model → protocol) with 24h TTL, success-only writesfirstChunkGatepattern buffers the response until commit/discard decision — no bytes hit the wire on failed first attemptdispatchWithPriorityFailoverGatedaccepts an external gate to avoid nested gate commit signal issuesKey files
ai/provider.goEndpointModeAutoconstantinternal/server/endpoint_cache.gointernal/server/endpoint_auto.gointernal/server/failover_dispatch.gointernal/server/openai_chat.gointernal/server/openai_responses.gointernal/server/endpoint_resolution.goautocaseinternal/probe/e2e.gofrontend/src/components/ProviderFormDialog.tsxTest plan
endpoint_cache_test.go: Get/Set, TTL expiry, concurrent safetyendpoint_auto_test.go: error classification (401→no retry, 404→retry, 429→no retry, context_length→no retry, unknown 500→retry)endpoint_resolution_test.go: auto mode casesauto, verify chat-only model falls back correctly, cache populated on successGenerated by Claude Code