1.3.4: interaction preview sends every new picture, stuck closes logged and counted

.github/workflows/container-codecs.yaml

@@ -1,7 +1,7 @@
 name: Container (H.264/H.265)
 
 # The image variant with a CEF build that plays H.264 and H.265 (docs/proprietary-codecs.md).
-# A tag codecs-<version> (for example codecs-1.3.3) builds that commit and pushes
+# A tag codecs-<version> (for example codecs-1.3.4) builds that commit and pushes
 # <version>-codecs. Normal releases (v*.*.*) are not touched.
 on:
   push:

CHANGELOG.md

@@ -1,6 +1,17 @@
 # Changelog
 
-## Image variant 1.3.3-codecs
+## 1.3.4 (2026-11-05)
+
+- **Interaction preview shows every new picture** (SPEC §7.3). The WebSocket preview waited only 20 ms for a new preview and otherwise sent the previous JPEG again: on a busy page (YouTube on test-all-gfx) about 2 of 9 frames per second were new, so the preview stood still for seconds. It now waits for the preview like `preview.jpg` (up to 100 ms), and a session never gets the same picture twice.
+  - Tests: `tests/integration/interact_test.py` (a moving page: no frame repeats the previous picture).
+- **Stuck closes are logged and counted** (SPEC §7.3, §12). A WebSocket closed because its client did not read (which ends interaction control) logs `websocket_stuck_close` with the client and reason and counts `websocket_stuck_closes_total{reason}` (`outbox_size`, `outbox_age`, `preview_not_taken`; at 0 from the start).
+  - Tests: `tests/unit/test_httpserver.cpp` (each reason reported once), `tests/unit/test_counters.cpp`, `tests/integration/metrics_test.py`, `interact_test.py`.
+- **Previews do not pile up in the kernel** (SPEC §7.3). WebSockets keep at most 64 KiB unsent (`TCP_NOTSENT_LOWAT`); on a slow path the server drops previews early instead of holding seconds of them in the send buffer, so the preview latency no longer grows and shrinks.
+  - **Behaviour change for clients:** a client of `/api/v1/interact` must read what it receives. One that only sends input and reads nothing is now closed after about 5 s (before, the kernel buffered megabytes of previews, so it took much longer); the web UI and widget always read. Tests that drive input read in the background (`tests/integration/pages_test.py`).
+
+## Image variant 1.3.3.2-codecs (2026-11-05)
+
+First image with H.264/H.265, built from 1.3.3; from 1.3.4 on every release also has `<version>-codecs`.
 
 - **H.264 and H.265 (proprietary codecs).** A second image, `<version>-codecs`, uses a CEF build of the same version with H.264 (`<video>`, MSE, WebRTC receive) and H.265 (`<video>`, MSE) decoding in software; the normal image cannot play them, so for example the WebRTC previews of other functions stayed black. The function itself is unchanged. **Licensing is the user's responsibility** ([`docs/proprietary-codecs.md`](docs/proprietary-codecs.md)).
   - `docker/Dockerfile` takes `CEF_URL` (default: the CEF project's build as before); `docker/cef-codecs/` holds the patches and the URL and SHA-1 of the build; a tag `codecs-<version>` runs `.github/workflows/container-codecs.yaml`.

CMakeLists.txt

@@ -1,7 +1,7 @@
 cmake_minimum_required(VERSION 3.24 FATAL_ERROR)
 
 project(mxl-browser-source
-    VERSION 1.3.3
+    VERSION 1.3.4
     DESCRIPTION "MXL media function: a web page rendered offscreen with CEF"
     LANGUAGES CXX C
 )

SPEC.md

@@ -387,8 +387,8 @@ Limits: 200 input messages per second per session, 64 KiB per message; beyond th
 The siblings preview with low-rate JPEG (multiviewer `/preview.jpg` at 5 fps and 480 px, test player thumbnail encoded on request). The browser source keeps that mechanism and adds a push path for interaction:
 
 - `GET /api/v1/preview.jpg`: the newest frame, converted after the request arrives (each request gets a ticket and is answered with a preview made for it or a later one; 1.1.0 answered with the previous request's picture), encoded once per preview.
-- Preview frames over the interaction WebSocket at `BROWSER_PREVIEW_FPS` (default 10, up to 25 while the session controls the page) and `BROWSER_PREVIEW_WIDTH` (default 960). A session asks for its rate with `preview` (a watcher can only lower it); the UI asks for 25 when its session gets control and for the default (`preview` without `fps`) when it loses it. The server also drops a session to at most `BROWSER_PREVIEW_FPS` when it loses control (toggle off, take-over, timeout), so only the controller can have more.
-- A client that stops reading never slows the others. Preview frames are sent without waiting: when a client's socket is full, its frame is dropped (the next one is newer; `preview_frames_dropped_total`), and a frame that has started goes out whole before anything else. Text messages (state, acks, errors, events) keep their order and are never dropped: what does not fit waits in a per-client outbox that the connection's own thread drains, so no sender ever waits for a client (state, cursor and console broadcasts come from the CEF UI thread, where a wait could trip the hang timeout). A client whose outbox holds more than 4 MiB or whose oldest message has waited 5 s, or that takes no preview frame for 5 s, is closed, which ends its control. This holds for `/api/v1/events` too. Polling a JPEG adds a round trip per frame; a push keeps the click-to-picture time short. WebRTC is not used: full-rate monitoring is mxl-webrtc-monitor's job (route the video flow to it).
+- Preview frames over the interaction WebSocket (each a preview made after it was due, waited for up to 100 ms like `preview.jpg`; a session never gets the same picture twice, 1.3.4) at `BROWSER_PREVIEW_FPS` (default 10, up to 25 while the session controls the page) and `BROWSER_PREVIEW_WIDTH` (default 960). A session asks for its rate with `preview` (a watcher can only lower it); the UI asks for 25 when its session gets control and for the default (`preview` without `fps`) when it loses it. The server also drops a session to at most `BROWSER_PREVIEW_FPS` when it loses control (toggle off, take-over, timeout), so only the controller can have more.
+- A client that stops reading never slows the others. Preview frames are sent without waiting: when a client's socket is full, its frame is dropped (the next one is newer; `preview_frames_dropped_total`), and a frame that has started goes out whole before anything else. Text messages (state, acks, errors, events) keep their order and are never dropped: what does not fit waits in a per-client outbox that the connection's own thread drains, so no sender ever waits for a client (state, cursor and console broadcasts come from the CEF UI thread, where a wait could trip the hang timeout). A client whose outbox holds more than 4 MiB or whose oldest message has waited 5 s, or that takes no preview frame for 5 s, is closed, which ends its control; such a close is logged (`websocket_stuck_close` with the client and reason) and counted (`websocket_stuck_closes_total{reason}`). The kernel keeps at most 64 KiB of a WebSocket unsent (`TCP_NOTSENT_LOWAT`), so on a slow path previews are dropped early instead of waiting seconds in the send buffer. This holds for `/api/v1/events` too. Polling a JPEG adds a round trip per frame; a push keeps the click-to-picture time short. WebRTC is not used: full-rate monitoring is mxl-webrtc-monitor's job (route the video flow to it).
 - The preview is made from the BGRA frame (downscaled box filter, composited over a checkerboard in the UI when transparent), so it costs nothing when no client watches.
 
 ## 8. NMOS
@@ -689,6 +689,7 @@ Prefix `mxl_browser_source_`. Durations in seconds with millisecond buckets.
 | `interaction_events_total` | counter | `type`, `result` (`ok`, `rejected`, `dropped`) |
 | `preview_frames_total` | counter | – (preview frames sent; until 1.3.1 also those a blocked client never took) |
 | `preview_frames_dropped_total` | counter | – (preview frames dropped because the client's socket was full, §7.3) |
+| `websocket_stuck_closes_total` | counter | `reason` (`outbox_size`, `outbox_age`, `preview_not_taken`; WebSockets closed because the client did not read, §7.3; 1.3.4) |
 | `devtools_sessions` | gauge | – |
 | `sender_enabled` | gauge | `sender` |
 | `nmos_registered` | gauge | – |

deploy/mxl-browser-source-gpu.yaml

@@ -60,7 +60,7 @@ spec:
           type: RuntimeDefault
       containers:
         - name: browser-source
-          image: ghcr.io/leeo86/mxl-browser-source:1.3.3
+          image: ghcr.io/leeo86/mxl-browser-source:1.3.4
           env:
             - name: NMOS_HOST_ADDRESS
               valueFrom:

deploy/mxl-browser-source.yaml

@@ -53,7 +53,7 @@ spec:
           type: RuntimeDefault
       containers:
         - name: browser-source
-          image: ghcr.io/leeo86/mxl-browser-source:1.3.3
+          image: ghcr.io/leeo86/mxl-browser-source:1.3.4
           env:
             - name: NMOS_HOST_ADDRESS
               valueFrom:

docker/docker-compose.demo.yaml

@@ -21,7 +21,7 @@ services:
       - ./registry.json:/home/registry.json:ro
 
   browser-source:
-    image: ${MBS_IMAGE:-ghcr.io/leeo86/mxl-browser-source:1.3.3}
+    image: ${MBS_IMAGE:-ghcr.io/leeo86/mxl-browser-source:1.3.4}
     network_mode: host
     user: "1000:1000"
     shm_size: 1gb

docker/docker-compose.host.yaml

@@ -4,7 +4,7 @@
 # GPU: add -f docker/docker-compose.gpu.yaml.
 services:
   browser-source:
-    image: ${MBS_IMAGE:-ghcr.io/leeo86/mxl-browser-source:1.3.3}
+    image: ${MBS_IMAGE:-ghcr.io/leeo86/mxl-browser-source:1.3.4}
     network_mode: host
     user: "1000:1000"
     shm_size: 1gb

docs/api.md

@@ -127,7 +127,7 @@ Our own routes:
 
 No token (the page asks for it). SPEC §10.6.
 
-`GET /widgets` → `[{"id":"interaction","title":"Browser interaction","params":{"type":"object","properties":{}},"min_size":{"w":640,"h":400},"version":"1.3.3"},{"id":"ograf","title":"OGraf graphics","params":{"type":"object","properties":{"layers":{"type":"string",…}}},"min_size":{"w":640,"h":480},"version":"1.3.3"}]`, with `Access-Control-Allow-Origin: <Origin>` when `WIDGET_FRAME_ANCESTORS` lists that origin (or `*`) and always `Vary: Origin`. Other methods: 405.
+`GET /widgets` → `[{"id":"interaction","title":"Browser interaction","params":{"type":"object","properties":{}},"min_size":{"w":640,"h":400},"version":"1.3.4"},{"id":"ograf","title":"OGraf graphics","params":{"type":"object","properties":{"layers":{"type":"string",…}}},"min_size":{"w":640,"h":480},"version":"1.3.4"}]`, with `Access-Control-Allow-Origin: <Origin>` when `WIDGET_FRAME_ANCESTORS` lists that origin (or `*`) and always `Vary: Origin`. Other methods: 405.
 
 `GET /widget/interaction[?theme=dark|light|transparent]` and `GET /widget/ograf[?theme=…&layers=1,2]` → the UI page with only the interaction window, or the Graphics tab's preview, layers and form (only the listed layers; no library, upload or delete; `docs/ograf.md` §8), `Content-Security-Policy: frame-ancestors <WIDGET_FRAME_ANCESTORS>`, no `X-Frame-Options`. A bad `theme`: 400; another id: 404. Other query parameters are ignored.
 

src/app/api.cpp

@@ -395,6 +395,7 @@ namespace mbs::app
             return std::nullopt;
         }
         ops::WebSocketHandlers h;
+        h.stuck = [this](char const* reason) { _metrics.inc("websocket_stuck_closes_total", {{"reason", reason}}, 1); };
         if (req.path == "/api/v1/events")
         {
             h.open = [this](std::shared_ptr<ops::WebSocket> const& socket) {

src/app/application.cpp

@@ -1473,9 +1473,12 @@ namespace mbs::app
 
     // Preview frames for the interaction sessions at their rates (SPEC §7.3): one JPEG per new
     // frame, sent to each session that is due. Nothing is converted while nobody watches.
+    // A session never gets the same picture twice: until 1.3.3 a preview that took longer than
+    // 20 ms was answered with the previous JPEG (about 2 new pictures a second on a busy page).
     void Application::previewLoop()
     {
         std::map<int, Clock::time_point> due;
+        std::map<int, std::uint64_t> shown; // per session: the preview sequence it got last
         std::vector<std::uint8_t> rgb;
         std::uint64_t sequence = 0;
         while (!_stopping.load())
@@ -1499,17 +1502,20 @@ namespace mbs::app
                 }
             }
             std::erase_if(due, [&](auto const& entry) { return rates.count(entry.first) == 0; });
+            std::erase_if(shown, [&](auto const& entry) { return rates.count(entry.first) == 0; });
             if (ready.empty())
             {
                 continue;
             }
+            // Waits for a preview made after this request, as preview.jpg does (at most 100 ms).
             auto const ticket = _engine->requestPreview();
-            for (int i = 0; i < 10 && !_engine->preview(ticket, rgb, sequence); ++i)
+            for (int i = 0; i < 50 && !_engine->preview(ticket, rgb, sequence); ++i)
             {
                 std::this_thread::sleep_for(std::chrono::milliseconds(2));
             }
             std::string jpeg;
             std::uint64_t grain = 0;
+            std::uint64_t jpegSequence = 0;
             {
                 std::lock_guard lock{_jpegMutex};
                 if (sequence != _jpegSequence && !rgb.empty())
@@ -1520,6 +1526,7 @@ namespace mbs::app
                 }
                 jpeg = _jpeg;
                 grain = _jpegGrain;
+                jpegSequence = _jpegSequence;
             }
             if (jpeg.empty())
             {
@@ -1536,9 +1543,11 @@ namespace mbs::app
                 std::lock_guard lock{_socketsMutex};
                 for (int id : ready)
                 {
-                    if (auto it = _interactSockets.find(id); it != _interactSockets.end())
+                    auto it = _interactSockets.find(id);
+                    if (it != _interactSockets.end() && shown[id] != jpegSequence)
                     {
                         targets.push_back(it->second);
+                        shown[id] = jpegSequence;
                     }
                 }
             }

src/ops/counters.cpp

@@ -33,6 +33,7 @@ namespace mbs::ops
         zero(m, "console_messages_total", "level", {"debug", "info", "warning", "error"});
         zero(m, "ograf_uploads_total", "result", {"ok", "rejected"});
         zero(m, "ograf_restores_total", "result", {"ok", "error"});
+        zero(m, "websocket_stuck_closes_total", "reason", {"outbox_size", "outbox_age", "preview_not_taken"});
         for (auto const* action : {"load", "play", "stop", "update", "custom", "clear"})
         {
             for (auto const* result : {"ok", "error", "timeout", "unavailable"})

src/ops/httpserver.cpp

@@ -3,6 +3,7 @@
 
 #include <arpa/inet.h>
 #include <netinet/in.h>
+#include <netinet/tcp.h>
 #include <poll.h>
 #include <sys/socket.h>
 #include <unistd.h>
@@ -16,6 +17,7 @@
 #include <cstring>
 #include <thread>
 
+#include "util/logging.hpp"
 #include "util/uuid.hpp"
 
 namespace mbs::ops
@@ -224,10 +226,13 @@ namespace mbs::ops
         return {};
     }
 
-    WebSocket::WebSocket(int fd, std::string peer)
+    WebSocket::WebSocket(int fd, std::string peer, std::function<void(char const* reason)> onStuck)
         : _fd(fd)
         , _peer(std::move(peer))
+        , _onStuck(std::move(onStuck))
     {
+        int const unsent = kUnsentBytes;
+        ::setsockopt(_fd, IPPROTO_TCP, TCP_NOTSENT_LOWAT, &unsent, sizeof(unsent));
     }
 
     WebSocket::~WebSocket() = default;
@@ -244,7 +249,7 @@ namespace mbs::ops
         _outboxBytes += _outbox.back().bytes.size();
         if (_outboxBytes > kOutboxBytes)
         {
-            fail();
+            stuck("outbox_size");
             return false;
         }
         flushLocked(now);
@@ -269,6 +274,16 @@ namespace mbs::ops
         _outboxBytes = 0;
     }
 
+    void WebSocket::stuck(char const* reason)
+    {
+        log::warn("websocket_stuck_close", {{"client", _peer}, {"reason", reason}, {"outbox_bytes", static_cast<std::uint64_t>(_outboxBytes)}});
+        if (_onStuck)
+        {
+            _onStuck(reason);
+        }
+        fail();
+    }
+
     void WebSocket::flushLocked(Clock::time_point now)
     {
         while (!_outbox.empty())
@@ -291,7 +306,7 @@ namespace mbs::ops
         }
         if (!_outbox.empty() && now - _outbox.front().queued >= kStuckTimeout)
         {
-            fail();
+            stuck("outbox_age");
         }
     }
 
@@ -304,7 +319,7 @@ namespace mbs::ops
         }
         else if (now - _droppingSince >= kStuckTimeout)
         {
-            fail();
+            stuck("preview_not_taken");
             return Sent::Closed;
         }
         return Sent::Dropped;
@@ -644,7 +659,7 @@ namespace mbs::ops
             {
                 return;
             }
-            auto socket = std::make_shared<WebSocket>(fd, req.peer);
+            auto socket = std::make_shared<WebSocket>(fd, req.peer, handlers.stuck);
             {
                 std::lock_guard lock{socketsMutex};
                 std::erase_if(sockets, [](auto const& w) { return w.expired(); });

src/ops/httpserver.hpp

@@ -50,17 +50,21 @@ namespace mbs::ops
     /// outbox holds more than kOutboxBytes or its oldest frame is older than kStuckTimeout.
     /// Binary frames (previews) are dropped instead of queued when the socket is full (the next
     /// one is newer); one that has started is finished first. A client that takes no binary
-    /// frame for kStuckTimeout is closed too.
+    /// frame for kStuckTimeout is closed too. Such a "stuck" close is logged and reported to
+    /// `onStuck` with its reason (outbox_size, outbox_age, preview_not_taken).
+    /// The kernel keeps at most kUnsentBytes unsent (TCP_NOTSENT_LOWAT), so a slow path drops
+    /// previews early instead of holding seconds of them in the send buffer.
     class WebSocket
     {
     public:
-        WebSocket(int fd, std::string peer);
+        WebSocket(int fd, std::string peer, std::function<void(char const* reason)> onStuck = {});
         ~WebSocket();
         WebSocket(WebSocket const&) = delete;
         WebSocket& operator=(WebSocket const&) = delete;
 
         static constexpr std::chrono::milliseconds kStuckTimeout{5000};
         static constexpr std::size_t kOutboxBytes = 4 * 1024 * 1024;
+        static constexpr int kUnsentBytes = 64 * 1024;
 
         enum class Sent
         {
@@ -92,6 +96,7 @@ namespace mbs::ops
         bool flush();
         // With _sendMutex held:
         void fail();
+        void stuck(char const* reason); // logs, reports and closes
         void flushLocked(Clock::time_point now);
         Sent dropped(Clock::time_point now); // counts towards kStuckTimeout
 
@@ -101,6 +106,7 @@ namespace mbs::ops
         bool _dropping = false;
         Clock::time_point _droppingSince;
         std::string _peer;
+        std::function<void(char const*)> _onStuck;
         std::mutex _sendMutex;
         std::atomic<bool> _open{true};
     };
@@ -110,6 +116,7 @@ namespace mbs::ops
         std::function<void(std::shared_ptr<WebSocket> const&)> open;
         std::function<void(std::shared_ptr<WebSocket> const&, std::string const& message, bool binary)> message;
         std::function<void(std::shared_ptr<WebSocket> const&)> close;
+        std::function<void(char const* reason)> stuck; // a stuck close (WebSocket)
         std::size_t maxMessage = 64 * 1024; // larger messages close the connection (1009)
     };