Skip to content

Watchdog: a frame stored after now is fresh data, not a stall (practice false restart) - #150

Merged
ConWan30 merged 1 commit into
mainfrom
fix/watchdog-no-wrap
Nov 11, 2026
Merged

ConWan30 merged 1 commit into
mainfrom
fix/watchdog-no-wrap

Conversation

@ConWan30

Copy link
Copy Markdown
Owner

Draft. Not for merge without Con's word.

What happened. Practice LIVE match-20261111-081216 on the e8509dc build logged no data from device for 4s at 8:15:16 AM CST (14:15:16.638Z) and restarted capture (recording split into video.part2.mp4). Encoder progress at 14:15:13.97Z showed frames still flowing (frame=10111 fps=59.97 drop=0).

Cause (consistent with the log, not reproduced on the PC). The watchdog in run_capture_once read now = me.ms() and then last_data_ms. When the reader thread stores a frame time between those two reads, last > now, and now - last on u64 wraps in release builds (overflow checks are off), so the gap looks huge and a healthy capture is killed.

Change. src/engine.rs only:

  • New watchdog_stalled(now, last, grace) uses saturating_sub, so last > now counts as fresh data.
  • The threshold (> grace) and grace values (15 s before first data, --watchdog after) are unchanged.
  • 3 unit tests: last > now is not a stall; a gap at or under grace is not a stall; a gap over grace is a stall.

Untouched. Receipts, receipt.v0, contracts/, schema/, capture ffmpeg args, recording bytes. REAL MATCH stays on v0.3.38. Live tap stays off by default.

Local (Linux). cargo test --no-fail-fast: lib 203 passed, 2 failed (clip::tests::* fail because the box's local qortime checkout lacks SELF_ATTESTED; that is environmental and unrelated to this diff), main 1, contracts 2, haptic_replay 1, redteam 2. Clippy reports no new warnings on the changed lines.

CI artifact. The CI on main (from #148) uploads qorcap-<sha7>-windows-x64 for this PR. That artifact contains this fix only.

The capture watchdog computed `now - last` on u64 ms. `last_data_ms` is
written by the reader thread, so a frame stored between the two reads
makes `last > now`; in release builds (no overflow checks) the
subtraction wraps to a huge gap and the watchdog kills a healthy capture.

Practice match-20261111-081216 (e8509dc build) logged "no data from
device for 4s" at 14:15:16.638Z while encoder progress showed frames
flowing (frame=10111, drop=0 at 14:15:13.97Z); the recording split into
video.part2.mp4. The wrap is consistent with that log; it was not
reproduced on the PC.

The check moves into `watchdog_stalled`, which uses saturating_sub.
Threshold (`> grace`) and grace values (15 s first data, --watchdog
after) are unchanged. Unit tests cover last > now, gaps at or under
grace, and gaps over grace.

No change to receipts, receipt.v0, contracts/, schema/, the capture
ffmpeg arguments or the recording bytes.
@ConWan30
ConWan30 marked this pull request as ready for review November 11, 2026 15:29
@ConWan30
ConWan30 merged commit 2293813 into main Nov 11, 2026
2 of 3 checks passed
ConWan30 added a commit that referenced this pull request Nov 11, 2026
…#152)

The linux job failed on main and on #150/#151 after about 63 min in
`sudo apt-get update && sudo apt-get install -y ffmpeg`. The logs show
no prompt and no dpkg lock: on the ubuntu-26.04 image the runner could
not resolve azure.archive.ubuntu.com, archive.ubuntu.com or
security.ubuntu.com ("Temporary failure resolving"). apt waited 20 s
per package per mirror for ~90 packages, then exited 100.
packages.microsoft.com and github.com resolved.

The ffmpeg step now downloads a pinned BtbN/FFmpeg-Builds static build
(autobuild-2026-10-31-12-59, n8.1.3 linux64 gpl, month-end builds are
kept long-term), checks its sha256, and puts ffmpeg/ffprobe on PATH
via GITHUB_PATH. If the download or checksum fails, apt runs with
DEBIAN_FRONTEND=noninteractive, Acquire retries/timeouts, a dpkg lock
timeout and an outer `timeout`. The step has timeout-minutes 15.

Job timeouts: linux 40, contracts 20, windows 45 min.

Only .github/workflows/ci.yml changes. release.yml and product code
are untouched.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant