fix(web-server) [BRNS-DESK-021]: prune ws subscribers on close instead of writing to closed sockets - #79
Conversation
…d of writing to closed sockets Each client navigation closed the WebSocket while broadcasts were still in flight, and tokio_tungstenite logged "Sending after closing is not allowed" per attempt. The forwarder tasks also outlived the connection: select! only drops the losing JoinHandles, which detaches those tasks rather than aborting them, so they kept their broadcast receivers and their sink clone alive and kept serializing for a dead socket. The broadcaster now hands each connection a Subscriber registration that deregisters on drop or explicit prune, the forwarders skip a fan-out for a pruned subscriber and prune on send failure, and the connection teardown prunes and aborts every task before the sink drops. Deregistration is the only thing logged, once per connection.
nir-ssvlabs
left a comment
There was a problem hiding this comment.
The exactly-once claim holds where it matters: prune() gates its decrement on live.swap(false, AcqRel) returning the previous value, so only the first caller ever decrements and Drop calling prune() cannot double-count. That is what keeps the AtomicUsize from underflowing, which on a usize would have been the real bug rather than a miscount.
The second fix is the larger one and it is wired correctly: subscriber.prune() runs at ws.rs:531, before the four .abort() calls at 535-538, so a forwarder mid-fan-out sees the connection gone rather than racing the sink drop — and the heartbeat now returns on ping failure instead of outliving the socket. Detaching on JoinHandle drop is an easy thing to read past; catching it while reading for something else is the good kind of accident.
- 🔵 The acceptance criterion — 20 reloads at
RUST_LOG=debugwith zeroSending after closing is not allowed— is still unperformed, and the two new tests do not stand in for it: they pin the counter's behaviour under churn and double-prune, not the absence of the log line. Everything structural says it should hold; nothing yet says it does. Worth one manual pass before this is called closed.
Checked: the prune/Drop exactly-once invariant and its memory ordering, teardown order against the aborts, the heartbeat's failure return, and that Rust is green on macOS, Ubuntu and Windows — which covers the compile and the two tests you could not run locally. Not read: the replay-buffer path below the teardown, untouched here.
Merge: ✅ into main.
| // would keep their broadcast receivers and their sink clone alive indefinitely. | ||
| a_forward.abort(); | ||
| b_forward.abort(); | ||
| cmd_loop.abort(); |
There was a problem hiding this comment.
🟡 cmd_loop.abort() cancels an in-flight dispatch_command at an arbitrary await point when another task ends the select! first (e.g. a_forward hits the dead socket while a command is still executing); stop_run_impl has already removed the session handle from state.sessions but its update_status(RunStatus::Stopped) never runs, leaving the run stuck in running state — pre-PR the detached task completed the command. Abort only a_forward/b_forward/heartbeat_task and await cmd_loop (it exits when ws_rx yields None after the socket drops), or run side-effectful dispatches in a spawned task so teardown cannot cancel them mid-write.
Summary
tokio_tungsteniteloggedSending after closing is not allowed~99x per session at DEBUG during ordinary navigation and reloads. Send failure is now the deregistration signal: the broadcaster hands each connection aSubscriberregistration, forwarders skip the fan-out for a pruned subscriber, and a failed send prunes it and exits.tokio::select!only drops the losingJoinHandles, and dropping aJoinHandledetaches the task rather than aborting it — soa_forward,b_forward,cmd_loop, andheartbeat_taskkept their broadcast receivers and theirArcclone of the split sink alive for the rest of the process. Teardown now prunes, then aborts all four.[broadcaster] subscriber gone, pruned (N live)), never per failed send.The lifecycle race
The race cannot be eliminated — a client close is asynchronous to a broadcast already fanning out — so it is handled once instead of being logged per attempt:
Subscriberis registered at connection setup, shared by the forwarder tasks, and deregisters onprune()or onDrop. Both paths decrement exactly once, so an entry cannot leak whichever way the connection dies.is_live()before touching a message, which also drops the wastedserde_jsonserialization for a client that is already gone.prune()before the sink drops, so a forwarder mid-fan-out sees the connection as gone; then it aborts the tasks, which drops the receivers and the sink.In the common case (clean navigation)
cmd_loopobserves the close frame first, so teardown prunes and aborts while the forwarders are still parked onrecv()— no send to a closed socket happens at all. Theprune()-on-send-error path is the belt for the genuinely racing case.Severity, honestly: no functional failure. Nothing was dropped or duplicated, and the log line is DEBUG-only. What this fixes is debug noise that would mask a real WS problem, wasted serialization per closed client, and the detached-task/receiver leak behind it.
Verification
Ran locally:
cargo fmt --manifest-path src-tauri/Cargo.toml(applied) and--check(clean). The pre-commit hook's Rust formatting check also passed.web_servermodule to confirm the mechanism, including that the subscriber "set" is a pair oftokio::sync::broadcastchannels rather than an explicit registry — the ticket's assumed data structure did not exist, hence the newSubscriberseam.Only CI can prove:
cargo testwas NOT run locally — no local Rust build env in this worktree (no cargo target dir), so the two new tests inbroadcaster.rs(subscriber_count_returns_to_baseline_after_churn,explicit_prune_is_idempotent_with_drop) are unrun. They are written to the existing#[cfg(test)] mod tests/ plain#[test]idiom already used inweb_server/mod.rsand need no async runtime. Compilation itself is therefore also CI-verified only.cargo clippyandcargo buildwere not run.RUST_LOG=debugproducing zeroSending after closing is not allowedlines, and events still reaching live clients — has not been performed; it needs a running app.