Repository navigation
fix(providers): raise default stream first-byte deadline to 60s - #439
Merged
Merged
Conversation
Qoder and WorkBuddy abandoned a stream that had not sent its response head within 30s. Thinking-model cold starts regularly take longer than that, so the request failed over (and waited another full deadline on the next account) before the upstream had a chance to answer. 60s is still short of the client's patience, and QODER_STREAM_TTFB_MS / WORKBUDDY_STREAM_TTFB_MS keep the old deadline available per deployment. The worker default moves with the Go side so the two agree.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
internal/providers/qoder/worker.goandworker/src/upstream.mjs, so the worker and the gateway still agree) and WorkBuddy (internal/providers/workbuddy/ttfb.go).QODER_STREAM_TTFB_MSandWORKBUDDY_STREAM_TTFB_MSstill override it, with the same 5s floor.Test plan
go test ./internal/providers/qoder/ ./internal/providers/workbuddy/cd worker && npm test(87 pass)QODER_STREAM_TTFB_MS=30000still restores the old deadline for a deployment that wants it