fix: allow slow CPU transcription through the proxy - #452
Conversation
Non-streaming proxied requests inherited the shared client's 30s read timeout, cutting off CPU transcription. Add a bounded PROXY_TIMEOUT (60s) and surface a read timeout as a 504 with CPU/GPU guidance.
Surface the backend 504 as a clear timeout toast, and guard against a cancelled request's late success repopulating the output box.
|
Review Solid, well-scoped fix -- good docs on the new timeout constant (mirrors the existing A few things worth a look: 1. Duplicated/drifting 504 copy (minor)
These already differ slightly and will keep drifting since nothing ties them together. Consider 2. Reimplements an existing error-parsing helper (code reuse) 3. Core behavior path still untested (acknowledged in PR body) 4. Non-streaming proxy path still does not check upstream status (pre-existing, adjacent) 5. Nit: exception chaining Nothing here blocks merging -- points 1 and 2 are the ones worth actually acting on; 3-5 are minor/follow-up material. |
Lift parseErrorResponse into the shared lib so speech-recognition and text-generation don't diverge; reuse the backend's 504 detail in the timeout toast. Chain the proxy ReadTimeout -> 504 with `from e`.
Summary
PROXY_TIMEOUT(60s read — generous for the short-clip service UI, still bounded) for the non-streaming proxy path, and convert anhttpx.ReadTimeoutinto a 504 with actionable CPU/GPU guidance instead of a generic 500.callSpeechRecognitionInference.controller.signal.abortedbefore writing output.Test plan
PROXY_TIMEOUTvalue locked by a unit test inlib/tests/unit/test_http_client.py.web/src/routes/speech-recognition/lib/requests.test.jscovers the success body, the 504 carrying status + backend detail, and the generic fallback when the error body has no JSON.abort()no longer writes its stale transcript to the output box.Follow-ups
Heavier tests deferred to backlog issues: #449 (proxy
ReadTimeout→ 504 route test) and #450 (container stale-response guard test).