Skip to content

Read every input grain; count skipped grains - #8

Merged
LeeO86 merged 1 commit into
mainfrom
fix/input-grain-skips
Nov 5, 2026
Merged

LeeO86 merged 1 commit into
mainfrom
fix/input-grain-skips

Conversation

@LeeO86

@LeeO86 LeeO86 commented Nov 5, 2026

Copy link
Copy Markdown
Owner

Problem

At 1080p50 with 25 Hz updates, input_grains_total grew at about 47.9/s while the source wrote 50/s. No resyncs were counted, and no other metric showed the missing grains.

Cause

src/mxlio/io.cpp, the input reader loop:

  1. mxlFlowReaderGetGrain with a timeout answers MXL_ERR_OUT_OF_RANGE_TOO_EARLY when the wait runs out. It never answers MXL_ERR_TIMEOUT (MXL 218ddaa, PosixDiscreteFlowReader::getGrainImpl with a deadline). The reader thought TOO_EARLY comes back at once, so it paused with wait(5). That pause really sleeps 10 ms, because wait sleeps in 10 ms steps.
  2. The wait is cut at the next update tick (timeout = tick - now). So every 25 Hz tick ran into this pause. Then came the update (analyse and render, about 10 ms). Together that often took longer than one frame (20 ms). By then the head was 2 or more grains past the cursor, and the reader jumped to the head (want = head when cursor + 1 < head). The jumped-over grain was not counted anywhere.

Jumping to the head is on purpose: in latest mode an update only needs the newest grain. The pause was the bug. Without it, the reader keeps up unless an update takes longer than a frame.

Change

  • No pause after MXL_ERR_OUT_OF_RANGE_TOO_EARLY. It is a wait that ran out, like a timeout, so the loop does not spin. Other unexpected answers still pause.
  • New counter mxl_scopes_input_grains_skipped_total for grains the reader jumps over when it is behind. It is counted once, when the grain after the gap is read. accumulate_dropped_total now counts at the same place. Before, a wait that ran out with the cursor still behind could count the same gap again.
  • Docs: SPECIFICATION §2.2 and the §10 metrics table, CHANGELOG under Unreleased.

Tests

  • Unit tests: 88/88 pass.
  • tests/integration/scope.sh has a new check: a 1080p50 scene writer, RGB parade at 25 Hz, 4 s. It wants at least 180 grains read, and read plus skipped between 194 and 206.
    • Before: 175 read, so it fails.
    • After: 202 read, 0 skipped, and the whole script passes.
  • Lab, 10 s per run, 1080p50, 25 Hz, grains read per second:
Mode / pattern Before After
luma / scene 50.1 50.1
ycbcr-parade / scene 50.1 50.1
rgb-parade / scene 33.4 50.2
rgb-parade / random – 50.1
rgb-overlay / scene 25.1 50.2
  • To check the counter, I built once with the old pause put back. Result: 34.5 read/s + 15.7 skipped/s = 50.2/s.

Note: the new integration check needs one update to take less than a frame on the CI runner (about 10 ms here for the RGB parade). A much slower runner would show skipped grains and fail the 180 bound.

🤖 Generated with Claude Code

https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4

…pped grains

mxlFlowReaderGetGrain answers MXL_ERR_OUT_OF_RANGE_TOO_EARLY (never
MXL_ERR_TIMEOUT) when its wait runs out. The reader treated that as an
answer that comes back at once and paused 10 ms. The wait is cut at the
next update tick, so every tick paused; the update after the pause often
ended after the next grain had come, and the reader jumped to the head
over one grain without counting it.

- No pause after MXL_ERR_OUT_OF_RANGE_TOO_EARLY.
- New metric input_grains_skipped_total: grains the reader jumps over
  when it is behind. accumulate_dropped_total counts such a jump once.
- Integration check: 1080p50 RGB parade at 25 Hz reads about 200 grains
  in 4 s (175 before), and read plus skipped is about 200.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4
@LeeO86
LeeO86 merged commit de6972d into main Nov 5, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant