Skip to content
57 changes: 45 additions & 12 deletions plans/gpu-resident-frames.md
Original file line number Diff line number Diff line change
Expand Up @@ -2924,18 +2924,51 @@ probe. The real findings from that work:

### What is left

- A lane's task queue can grow past the scheduler's cap. On foundmeat's
libx264 run, a node of the foundmeat package in compose's sidecar reached
1023 tasks against a cap of 10, while the reader of the stream it reads
waited for room. How it got past the cap is not known yet. The cap
(`lane.queue.len() < cap`) is checked only where a producer waits for
room, before a reader delivers and before a lane starts a task, so tasks
queued any other way would not be counted. The check predates the
branch's change to the bound, since a6a649c counts only the arrivals
queued at a self-clocked reader. While such a queue stays above the cap,
every reader of the streams that lane reads is held back with it. Next:
find which path queued the tasks, with the `room:` trace and the lane's
intake, and bound it.
- The 1023-task queue and 0.32.0 falling short of real time (2026-10-19).
The pace fix shipped in 0.32.1; the waker fix (the scheduler's waker
set after its workers started, on main too, the cause of the hangs
seen while testing) shipped in 0.32.2; the lane change below, held
back at first since the queue did not keep 0.31.0 from real time,
went in once the CPU profile showed it was the idle broadcast's
cost. The lane was `listen_audio` in
compose's sidecar. Its clock is the programme's sound, an `asplit` copy
that rides one input with compose's own `a`, and the module declares
a window of one sample, so each 1024-sample packet settled 1024 ticks.
An input clock's cut queued every settled tick at once (the cap was
checked only where a reader delivers), and every tick cost a turn of the
scheduler. On top of that, 3dad921's pace check walked the whole quarter
second of ticks (12000 of them) on every tick whenever the clock ran at
real time. The lane fell behind and held back the reader that carries
compose's sound, so the programme ran at 50 fps: 8.3 media seconds per
10 s of wall on 0.32.0 with the main feed alone, against 10.15 on
0.31.0. Bisected in a container on the box. a6a649c and 912bf25
reverted: 8.34. bcce48e: 10.15. 8f6b9d9: 8.34. 0.32.0 with the pace
check off: 10.15. Fixed: the pace keeps running extremes and is asked
only where a port-fed hold uses it; the cut stops at the cap and holds
the next tick back; a one-at-a-time lane takes up to 1024 settled ticks
in a turn; a turn's progress marks leave once; and an audio clock's
reader is held back by packets waiting, not by ticks. With two cams the
query keeps real time (10.15 against 8.02 on 0.32.0). Left for
foundmeat: `listen_audio` could declare a window of a packet or more,
since it reads only the times.
- The Linux route's CPU was measured on 2026-10-20 (perf, strace, gdb on
the box): 1.34 cores idle on 0.32.1, 61% of it the kernel, the four
workers making about 30,000 context switches a second each because a
node clocked a sample a tick (foundmeat's listener on the programme's
sound) was cut one task a sample and every dispatch and completion
woke every worker. The host's waits on the device cost under 1%, and
CUDA's waits already block (libav sets `CU_CTX_SCHED_BLOCKING_SYNC`
itself). The lane change (the cut stops at the cap, ticks taken in
turns) brought it to 0.99 and keeping the host's plane textures to
0.92, the Windows figure; the node's own window of a packet rather
than a sample (foundmeat 0.3.0) brought their idle broadcast to 0.46
on 0.32.2 alone. Two further commits, the decoder's copy and the
encoder's feed submitted on the device after the picture's semaphore
with no host wait (branch `fix/linux-route-cpu`, 91c0689 and
6a87172), measured within noise and rest on wgpu-core's submit
holding its device fence lock for writing, which is not documented;
they stay unmerged until a wgpu release documents that or the saving
is worth the dependence.
- The host waits for the decoder's write before each copy is submitted,
and the decode thread waits with it. A wait on the queue would need a
submit with wait semaphores, which wgpu-core does not have; a raw submit
Expand Down
22 changes: 22 additions & 0 deletions sidecar/NODE-CLI.md
Original file line number Diff line number Diff line change
Expand Up @@ -369,6 +369,28 @@ tick that held them.
the clock reads: an `arrival` input's pts may sit on any origin (a feed
at the epoch while the clock counts from 0), so they never hold the
clock.
- **Queues.** A node's ticks wait in a queue of at most `2 * jobs + 2`
until a worker runs them, and a reader waits while the queue of
anything reading its stream is full. Every clock is cut no further than
that. One arrival can settle many ticks: an audio clock ticks every
`window` samples, so a node with a window of one sample on a clock
bound to sound is cut 1024 ticks by a packet of 1024 samples, 48000 a
second. Those past the bound stay with the clock and are cut as the
queue drains, and while any are left the node counts as full to its
readers, except to the reader of that audio clock, which waits only
once the bound's worth of packets waits on the clock. A self-clocked node is handed one arrival an input a call, and
the arrivals waiting on its input count against the same bound, so a
decoder ahead of a slower encoder holds its reader back rather than
queue every picture it made. A node that runs one tick at a time takes
the ticks queued behind the one it starts, and those its clock has
settled past the bound, up to 1024, and runs them in order in one turn,
and the progress they send on each port leaves once, as the last of
them: a clock of small ticks costs a node call a tick, not a turn of
the scheduler a tick. A node with ticks held back goes first. A node
slower than its clock holds back the reader of its clock's input, and
with it every stream that input carries (two `asplit` copies of one
sound, one for a listener and one for the programme, ride one input),
so it can slow the whole programme.
- **Hold inputs.** A hold input bound to a stream holds the tick until its
source has arrived past it (its picture, or with none its sound, past
`lead` beyond the tick), or ended, for no longer than its
Expand Down
Loading
Loading