Repository navigation
fix(executor): keep accounts routable after a transport failure - #437
Merged
caigee-cmd merged 2 commits intoOct 29, 2026
Merged
Conversation
A dial, TLS handshake, or reset failure never reaches the upstream, but it fell through to the generic unavailable branch: 15s cooldown plus failover. On a single-account route — the normal personal setup — that parks the only usable account, and every following request is answered with "all accounts are cooling down" instead of the real transport error. Classify a *url.Error as upstream_unreachable: 502, no cooldown, no failover. Detection is shape-based, so an upstream 502 body that merely quotes "TLS handshake timeout" still cools the account as before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ClassifyErrorto the generic unavailable branch: 15s account cooldown + failover. The account gets parked although its credential, quota, and rate limit were never exercised.429 rate_limit "all accounts are cooling down"(Retry-After = the remainder of the 15s), so the real cause — the transport error — never reaches the client. With a 1-account provider route there is no other account to fail over to, so the account cools itself in a loop: handshake timeout → 502 → 15s cooldown → 429s → cooldown expires → handshake timeout again.*url.Errorasupstream_unreachable: 502, no cooldown, no failover, message kept. Detection is shape-based:http.Clientreturns*url.Erroronly when no response arrived, so an upstream (or proxy) 502 whose body merely quotesnet/http: TLS handshake timeoutkeeps the existing 15s cooldown.Observed upstream shape:
GET /on the same host succeeded seconds later, and other overseas providers showed the same handshake timeouts in the same window — i.e. an intermittent egress problem, not a sick account.Test plan
go test ./...— CI green (go,windows-host,container,frontend,worker,secrets). Untouched packages are unaffected.go vet ./...— CI greencd worker && npm test— untouched, CI greencd frontend && npm run build && npm run lint— untouched, CI greenchangelog/unreleased/transport-failure-no-cooldown.mdTestClassifyErrorTransportFailureKeepsAccountRoutable(asserts the account is still picked byPickRouteafterwards),TestClassifyErrorWrappedTransportFailure, andTestClassifyErrorUpstreamQuotedHandshakeTimeoutStillCools(pins that detection is shape-based, not message-based)Note for local runners: on my Windows box a clean
2da8c45checkout already failsTestRepeatedFailuresBackOffandTestBackoffResetsOnKindChangeplus a few env-dependent packages. Verified by stashing this diff. CI does not reproduce it, so it is local-environment noise, not this PR.Two judgment calls
Failover: false. Every account in a deployment shares one egress path unless it sets its ownproxy_url, so failing over just repeats the same 8–11s handshake per account and ends at the same 502. If you prefer to keep one failover attempt for per-account-proxy setups, flipping this field is the whole change —upstream_emptyalready doesFailover: true, Cooldown: 0for the same "the account is fine" reasoning.ObserveStreamFailure, which has its own documented reason (bytes are already on the wire) and is untouched here.