Skip to content

CI: pinned static ffmpeg on Linux (apt mirror DNS failures), bounded apt fallback, job timeouts - #152

Merged
ConWan30 merged 1 commit into
mainfrom
ci/linux-static-ffmpeg
Nov 11, 2026
Merged

ConWan30 merged 1 commit into
mainfrom
ci/linux-static-ffmpeg

Conversation

@ConWan30

Copy link
Copy Markdown
Owner

Draft, CI only. Not for merge without Con's word. Only .github/workflows/ci.yml changes; release.yml and product code are untouched.

Root cause (from the linux job logs of runs 43042443989, 43048838656 and 43058141889).

  • It was not an interactive prompt (tzdata/needrestart) and not a dpkg lock.
  • On the ubuntu-26.04 image (20261101.186.1), the runner could not resolve azure.archive.ubuntu.com, archive.ubuntu.com or security.ubuntu.com ("Temporary failure resolving").
  • apt-get update took 3 min 20 s. install ffmpeg then waited about 20 s per package per mirror across ~90 packages and exited 100 after about 60 min.
  • packages.microsoft.com and github.com resolved fine.
  • The same image installed ffmpeg in 11 s on the 2026-11-11 03:01Z run, so this is a mirror/DNS problem outside this repo.

Fix.

  • The ffmpeg step downloads a pinned static build from BtbN/FFmpeg-Builds: autobuild-2026-10-31-12-59, ffmpeg-n8.1.3-27-g8dca1586f4-linux64-gpl-8.1.tar.xz. That project keeps month-end builds long-term (the oldest still listed is 2024-12-31).
  • It checks sha256 41e9c565…44e3380 against the digest GitHub publishes for that release asset, then adds bin/ (ffmpeg and ffprobe) to GITHUB_PATH. The tests find ffmpeg with shutil.which("ffmpeg") and through PATH in cargo test.
  • Bounded apt fallback, used only if the download or checksum fails: DEBIAN_FRONTEND=noninteractive, Acquire::Retries=2, 20 s http/https timeouts, Dpkg::Lock::Timeout=60, --no-install-recommends, and an outer timeout (180 s update, 300 s install).
  • The step has timeout-minutes: 15. Job timeouts: linux 40, contracts 20, windows 45.

Local check (Linux box). The step script downloaded and verified the build and wrote PATH. With that ffmpeg on PATH:

  • cargo test --locked: lib 206, main 1, contracts 2, haptic_replay 1, redteam 2, all passing.
  • The ffmpeg sidecar tests (highlights, challenge, hl170123, seq125430): 18 passed.

Version note. apt shipped ffmpeg 8.0.1; this build is 8.1.3. Only CI tests use it. Neither the release nor what Con installs uses it (Windows uses get-ffmpeg.ps1, unchanged).

The linux job failed on main and on #150/#151 after about 63 min in
`sudo apt-get update && sudo apt-get install -y ffmpeg`. The logs show
no prompt and no dpkg lock: on the ubuntu-26.04 image the runner could
not resolve azure.archive.ubuntu.com, archive.ubuntu.com or
security.ubuntu.com ("Temporary failure resolving"). apt waited 20 s
per package per mirror for ~90 packages, then exited 100.
packages.microsoft.com and github.com resolved.

The ffmpeg step now downloads a pinned BtbN/FFmpeg-Builds static build
(autobuild-2026-10-31-12-59, n8.1.3 linux64 gpl, month-end builds are
kept long-term), checks its sha256, and puts ffmpeg/ffprobe on PATH
via GITHUB_PATH. If the download or checksum fails, apt runs with
DEBIAN_FRONTEND=noninteractive, Acquire retries/timeouts, a dpkg lock
timeout and an outer `timeout`. The step has timeout-minutes 15.

Job timeouts: linux 40, contracts 20, windows 45 min.

Only .github/workflows/ci.yml changes. release.yml and product code
are untouched.
@ConWan30
ConWan30 marked this pull request as ready for review November 11, 2026 17:53
@ConWan30
ConWan30 merged commit f10b978 into main Nov 11, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant