[CP v2.16] Fix WebSocket disconnect during large manual log file uploads (#1062) - #344
Merged
rquidute merged 1 commit intoJul 22, 2026
Conversation
…#343) * Fix WebSocket disconnect during large manual log file uploads Uploading a manual test log with hundreds of thousands of lines (e.g. TC-CADMIN-1.17) called logger.info() once per line. Each call triggered a broadcast over the same main WebSocket used by the CLI and a synchronous DB commit, stalling the event loop long enough for the WebSocket's ping/pong keepalive to time out mid-upload. The connection was dropped with ConnectionClosedError, and the next prompt-response send failed with 'Unexpected error uploading file' even though the upload itself had already succeeded (issue #1062). - Batch uploaded log lines into chunks of 500 before calling logger.info(), instead of one call per line, so a large file produces a handful of log/broadcast/DB-commit operations instead of hundreds of thousands. - Collapse the invalid-UTF-8 warning to a single message per upload instead of one per bad line. - Add app.uvicorn_worker.ExtendedTimeoutUvicornWorker, which raises ws_ping_timeout to 60s (matching the value already used by the dev-only start-reload.sh script), and wire it up as the default worker class in gunicorn/start.sh so production tolerates the same event-loop stalls as local dev. - Add unit tests covering chunked logging, invalid UTF-8 handling, and unsupported content-type rejection. * Address review: strip CRLF instead of LF only when batching log lines .rstrip("\n") left a trailing \r on every line of a Windows-style (CRLF) uploaded log, since the file is read in binary mode and split only on \n. That stray \r would end up embedded in the joined chunk, showing up as ^M characters or unexpected double newlines wherever the chunk is logged or displayed. Use .rstrip("\r\n") so both Unix and Windows line endings are handled correctly. Add a regression test with a CRLF-terminated upload. * Fix CI mypy error: annotate FakeUploadFile.file as BinaryIO mypy's structural check against the UploadFile protocol requires file: BinaryIO exactly; a bare BytesIO() assignment left the attribute inferred as BytesIO, which mypy treats as incompatible with the protocol's BinaryIO annotation despite BytesIO satisfying it at runtime. Annotate the attribute explicitly so handle_uploaded_file(FakeUploadFile(...)) type-checks.
|
Tick the box to add this pull request to the merge queue (same as
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request addresses websocket connection drops during large manual log uploads (GitHub issue #1062). It introduces log chunking (500 lines per chunk) in handle_uploaded_file to prevent event loop starvation, optimizes UTF-8 warning frequency, and implements a custom ExtendedTimeoutUvicornWorker with an increased ws_ping_timeout of 60 seconds. Comprehensive unit tests have also been added to verify these changes. There are no review comments, and I have no feedback to provide.
antonio-amjr
approved these changes
Jul 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cherry-pick of the merged fix for #1062 (WebSocket disconnect during large manual log file uploads) onto
v2.16-develop, for the Matter v1.7 TE2 release line.Original PR: #343 (merged to
v2.15.1-developascb44885).Contents
Single cherry-picked commit (squash-merge of #343, including review fixes for CRLF handling and the mypy
BinaryIOannotation):logger.info(), instead of one call per line, avoiding the event-loop stall that previously tripped the WebSocket's keepalive ping/pong during large uploads.app.uvicorn_worker.ExtendedTimeoutUvicornWorker(raisesws_ping_timeoutto 60s) and wires it up as the default production worker class ingunicorn/start.sh.Testing
Cherry-picked cleanly with no conflicts. Re-ran flake8/mypy/black/isort against the
v2.16-developbase for all touched files — all clean.