Skip to content

1.4.0: interaction preview flow control behind proxies, preview JPEG quality setting - #19

Merged
LeeO86 merged 3 commits into
mainfrom
fix/d354-interact-backpressure
Nov 6, 2026
Merged

LeeO86 merged 3 commits into
mainfrom
fix/d354-interact-backpressure

Conversation

@LeeO86

@LeeO86 LeeO86 commented Nov 6, 2026

Copy link
Copy Markdown
Owner

Problem. On a slow path the interaction preview fell behind and control was lost. 1.3.4 limits what the kernel holds unsent (TCP_NOTSENT_LOWAT), but a reverse proxy between the function and the browser reads the previews at once and keeps them in its own buffers. Then the server sees no back-pressure, the previews get seconds old, and in the end the session is closed as stuck, which ends its control.

Changes

  • Preview flow control (SPEC §7.2, §7.3; docs/interact.schema.json). A client that answers each preview frame with {"type":"preview_ack"} has at most 2 previews unanswered. Until an answer comes, the server skips the session (counted in preview_frames_dropped_total), then it sends the newest picture. A session that answers nothing for 5 s is closed as preview_not_taken. Clients that never send preview_ack work as before. The web UI and the widgets answer every frame. The input limit does not count acks.
  • TCP_NOTSENT_LOWAT is 32 KiB instead of 64 KiB (measured, below).
  • websocket_stuck_close also logs the interaction session (0 on /api/v1/events) and oldest_ms, the age of the oldest waiting message. websocket_stuck_closes_total{reason} is unchanged.
  • BROWSER_PREVIEW_QUALITY (default 75, range 10–100, needs a restart): the JPEG quality of preview.jpg and of the interaction preview.
  • Docs: SPEC, docs/api.md, README (slow-path behaviour in plain words, the new setting), CHANGELOG "Unreleased".

Lab. Setup: plain docker without privileges, tc (netem 40 ms + tbf) in a sidecar that shares the container's network namespace, or in a userspace TCP proxy container. The proxy reads 64 KiB chunks and leaves its kernel send buffer to autotuning, as nginx does. The page sends random pixels, so each preview is a 50 KB JPEG. The client asks for 25 fps. Age = arrival time − TAI of the grain the preview was encoded at (median / p95):

Path 1.3.3 1.3.4 this PR
direct, 2 Mbit/s 2.76 / 2.87 s 0.79 / 0.91 s 0.36 / 0.45 s (with acks), 0.65 s (client without acks)
proxy, 2 Mbit/s – 10.1 / 10.8 s, growing for 15 s 0.22 / 0.37 s
proxy, 500 kbit/s – 7.7 s, up to 14 s, then stuck close outbox_age after 20.6 s (control lost) 1.15 s, no close, control kept for 60 s
direct, 16 Mbit/s 79 ms, 21.4 fps – 54 ms, 16.8 fps (with acks)
LAN – – 2 ms, 21.6 fps (with acks)

The frame rate was the same everywhere (the link limit), except on 16 Mbit/s with a 40 ms round trip. There, 2 frames in flight give 17 instead of 22 fps.

TCP_NOTSENT_LOWAT sweep (1.3.3 image with the sysctl, 2 Mbit/s, 40 ms): 16 KiB 0.65 s, 32 KiB 0.64 s, 64 KiB 0.80 s, 128 KiB 1.07 s. At 6 and 16 Mbit/s every value gave the same frame rate.

Log lines from the integration test against this image:

warn websocket_stuck_close client=172.17.0.1 session=2 reason=outbox_age outbox_bytes=11443 oldest_ms=5077
warn websocket_stuck_close client=172.17.0.1 session=7 reason=preview_not_taken outbox_bytes=0 oldest_ms=0

Tests. Unit tests: the window (per session, an ack before the next send, a frame that was not sent, acks over the input limit), a held-back preview counts towards the stuck close, and the new setting. tests/unit/web/interact.test.mjs: the UI answers every frame, and preview_ack is in the schema. tests/integration/interact_test.py checks that 2 previews stay unanswered, 1 more comes per answer, and the session is closed and counted 5 s after the last answer. Locally: 114/114 unit tests, 14/14 web tests, and interact_test.py against the built image all passed.

Notes

  • The -codecs variant builds from the same sources and needs nothing extra.
  • An ack sent for no frame lets one more preview through for the rest of the session. In the lab, one extra ack before the first frame raised the age at 2 Mbit/s from 0.23 to 0.39 s. So the UI sends acks only for frames it received.

🤖 Generated with Claude Code

https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4

LeeO86 and others added 3 commits November 6, 2026 20:56
…oxy.

A reverse proxy between the function and the browser reads the previews at
once and holds them in its own buffers, so TCP_NOTSENT_LOWAT on our socket
does not help. Lab (50 KB previews, 25 fps asked, 40 ms round trip, a
buffering userspace proxy with a shaped link to the client): 1.3.4 delivered
10-11 s old previews at 2 Mbit/s; at 500 kbit/s the delay grew to 14 s and
the session was closed as stuck (outbox_age) after 20 s, ending control.

- Preview flow control: a client that answers each preview with
  preview_ack has at most 2 previews unanswered; the server skips it until
  an answer comes (counted as dropped) and then sends the newest picture.
  No answer for 5 s closes it as preview_not_taken. Clients that never ack
  are unchanged. The web UI and the widgets ack every frame. Acks are not
  counted against the input limit.
- TCP_NOTSENT_LOWAT 32 KiB instead of 64 KiB: 16/32/64/128 KiB at
  2 Mbit/s 40 ms gave 0.65/0.64/0.80/1.07 s median age; no fps loss at 6
  and 16 Mbit/s.
- websocket_stuck_close logs the interaction session and oldest_ms.
- BROWSER_PREVIEW_QUALITY (75, 10-100, restart) for the preview JPEG.

Median preview age after the change: behind the proxy 0.22 s at 2 Mbit/s
and 1.1 s at 500 kbit/s (no close, control kept for 60 s); direct at
2 Mbit/s 0.36 s (1.3.4: 0.79 s). Cost: 17 instead of 22 fps on a
16 Mbit/s path with 40 ms round trip; none on a LAN.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4
Review of the preview flow control:
- An ack now resets the stuck timer (WebSocket::binaryTaken). Before, only
  a sent frame did, so a client that answered every frame while its window
  was shut by a backlog from before its first ack could be closed as
  preview_not_taken. The counter keeps counting every send from the first
  frame on, so that backlog is drained before new previews go out
  (clamping it on the first ack would leave the backlog in flight for
  good: each answer for a buffered frame would let a new one in).
- A shut window skips a session (and counts a drop towards the stuck
  close) only when there is a new picture; a static page closes no one.
- preview_ack is recognised with a size and substring check before the
  input limit; other messages are parsed after it, as before.
- SPEC: an extra ack is ignored when nothing is unanswered.

Lab, 500 kbit/s behind the buffering proxy, 40 ms, fresh connect: median
age 1.17 s, 60 s without a close, control kept; two reconnects on the same
instance: no close (the first seconds up to 3.9 and 7.6 s while the
backlog drained).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4
@LeeO86 LeeO86 changed the title Keep the interaction preview current on a slow path, also behind a proxy 1.4.0: interaction preview flow control behind proxies, preview JPEG quality setting Nov 6, 2026
@LeeO86
LeeO86 merged commit 78366f7 into main Nov 6, 2026
3 checks passed
@LeeO86
LeeO86 deleted the fix/d354-interact-backpressure branch November 6, 2026 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant