Skip to content

ci(eval): install ffmpeg for eval scaffolds that render frames #15482

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kyle-sexton merged 2 commits into from
Nov 12, 2026
+96 −0
Merged

ci(eval): install ffmpeg for eval scaffolds that render frames #15482

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
69d895f
ci(eval): install ffmpeg for eval scaffolds that render frames
kyle-sexton Nov 12, 2026
aae9e6d
ci(eval): install ffmpeg after isolation, only for plugins that need it
kyle-sexton Nov 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 96 additions & 0 deletions .github/workflows/eval-browser-dispatch.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -525,6 +525,38 @@ jobs:
npm ci --prefix /opt/tools
echo "/opt/tools/node_modules/.bin" >> "$GITHUB_PATH"

# Eval cases whose scaffold renders frames (explainer-video, knowledge)
# name ffmpeg as a host tool, and the hosted image carries none. Same
# BtbN static build, tag and SHA-256 as pr-require-checks.yml's demo-video
# leg, into /opt/tools/bin, which every eval run's sandbox can read. After
# isolation, like shfmt below, so the binary never runs with sudo.
- name: Install ffmpeg for eval scaffolds
if: >-
steps.stage.outputs.go == 'true' &&
(contains(steps.plugins.outputs.list, 'plugins/explainer-video') ||
contains(steps.plugins.outputs.list, 'plugins/knowledge'))
timeout-minutes: 3
env:
FFMPEG_RELEASE: autobuild-2026-09-30-13-08
FFMPEG_BUILD: n8.1.3-9-g29e619e767
FFMPEG_SHA256: 97ce978979194b5cf7e06a5e68020dbdaa7a4f3294c5452b6a1bc347100dbb79
run: |
asset="ffmpeg-$FFMPEG_BUILD-linux64-gpl-8.1.tar.xz"
archive="$RUNNER_TEMP/$asset"
mkdir -p /opt/tools/bin
curl -fSL --no-progress-meter --retry 7 --retry-all-errors -o "$archive" \
"https://github.com/BtbN/FFmpeg-Builds/releases/download/$FFMPEG_RELEASE/$asset"
echo "$FFMPEG_SHA256 $archive" | sha256sum -c - ||
{ echo "::error::$asset does not match FFMPEG_SHA256"; exit 1; }
tar -xJf "$archive" -C /opt/tools/bin --strip-components=2 --wildcards '*/bin/ffmpeg' '*/bin/ffprobe'
rm -f "$archive"
echo "/opt/tools/bin" >> "$GITHUB_PATH"
for tool in ffmpeg ffprobe; do
line=$("/opt/tools/bin/$tool" -version | sed -n 1p)
echo "$line"
[[ "$line" == *"$FFMPEG_BUILD"* ]] || { echo "::error::$tool reports '$line', not $FFMPEG_BUILD"; exit 1; }
done
Comment thread
kyle-sexton marked this conversation as resolved.
Resolved
Comment thread
kyle-sexton marked this conversation as resolved.
Resolved

# The guardrails hooks read every Bash command through shfmt and refuse the
# call without it, so a guardrails eval needs it on PATH. Same v3.14.1 pin
# and SHA-256 as pr-require-checks.yml. In /opt/tools, since each eval run
Expand Down Expand Up @@ -1356,6 +1388,38 @@ jobs:
npm ci --prefix /opt/tools
echo "/opt/tools/node_modules/.bin" >> "$GITHUB_PATH"

# Eval cases whose scaffold renders frames (explainer-video, knowledge)
# name ffmpeg as a host tool, and the hosted image carries none. Same
# BtbN static build, tag and SHA-256 as pr-require-checks.yml's demo-video
# leg, into /opt/tools/bin, which every eval run's sandbox can read. After
# isolation, like shfmt below, so the binary never runs with sudo.
- name: Install ffmpeg for eval scaffolds
if: >-
steps.stage.outputs.go == 'true' &&
(contains(steps.plugins.outputs.list, 'plugins/explainer-video') ||
contains(steps.plugins.outputs.list, 'plugins/knowledge'))
timeout-minutes: 3
env:
FFMPEG_RELEASE: autobuild-2026-09-30-13-08
FFMPEG_BUILD: n8.1.3-9-g29e619e767
FFMPEG_SHA256: 97ce978979194b5cf7e06a5e68020dbdaa7a4f3294c5452b6a1bc347100dbb79
run: |
asset="ffmpeg-$FFMPEG_BUILD-linux64-gpl-8.1.tar.xz"
archive="$RUNNER_TEMP/$asset"
mkdir -p /opt/tools/bin
curl -fSL --no-progress-meter --retry 7 --retry-all-errors -o "$archive" \
"https://github.com/BtbN/FFmpeg-Builds/releases/download/$FFMPEG_RELEASE/$asset"
echo "$FFMPEG_SHA256 $archive" | sha256sum -c - ||
{ echo "::error::$asset does not match FFMPEG_SHA256"; exit 1; }
tar -xJf "$archive" -C /opt/tools/bin --strip-components=2 --wildcards '*/bin/ffmpeg' '*/bin/ffprobe'
rm -f "$archive"
echo "/opt/tools/bin" >> "$GITHUB_PATH"
for tool in ffmpeg ffprobe; do
line=$("/opt/tools/bin/$tool" -version | sed -n 1p)
echo "$line"
[[ "$line" == *"$FFMPEG_BUILD"* ]] || { echo "::error::$tool reports '$line', not $FFMPEG_BUILD"; exit 1; }
done

# The guardrails hooks read every Bash command through shfmt and refuse the
# call without it, so a guardrails eval needs it on PATH. Same v3.14.1 pin
# and SHA-256 as pr-require-checks.yml. In /opt/tools, since each eval run
Expand Down Expand Up @@ -2140,6 +2204,38 @@ jobs:
npm ci --prefix /opt/tools
echo "/opt/tools/node_modules/.bin" >> "$GITHUB_PATH"

# Eval cases whose scaffold renders frames (explainer-video, knowledge)
# name ffmpeg as a host tool, and the hosted image carries none. Same
# BtbN static build, tag and SHA-256 as pr-require-checks.yml's demo-video
# leg, into /opt/tools/bin, which every eval run's sandbox can read. After
# isolation, like shfmt below, so the binary never runs with sudo.
- name: Install ffmpeg for eval scaffolds
if: >-
steps.stage.outputs.go == 'true' &&
(contains(steps.plugins.outputs.list, 'plugins/explainer-video') ||
contains(steps.plugins.outputs.list, 'plugins/knowledge'))
timeout-minutes: 3
env:
FFMPEG_RELEASE: autobuild-2026-09-30-13-08
FFMPEG_BUILD: n8.1.3-9-g29e619e767
FFMPEG_SHA256: 97ce978979194b5cf7e06a5e68020dbdaa7a4f3294c5452b6a1bc347100dbb79
run: |
asset="ffmpeg-$FFMPEG_BUILD-linux64-gpl-8.1.tar.xz"
archive="$RUNNER_TEMP/$asset"
mkdir -p /opt/tools/bin
curl -fSL --no-progress-meter --retry 7 --retry-all-errors -o "$archive" \
"https://github.com/BtbN/FFmpeg-Builds/releases/download/$FFMPEG_RELEASE/$asset"
echo "$FFMPEG_SHA256 $archive" | sha256sum -c - ||
{ echo "::error::$asset does not match FFMPEG_SHA256"; exit 1; }
tar -xJf "$archive" -C /opt/tools/bin --strip-components=2 --wildcards '*/bin/ffmpeg' '*/bin/ffprobe'
rm -f "$archive"
echo "/opt/tools/bin" >> "$GITHUB_PATH"
for tool in ffmpeg ffprobe; do
line=$("/opt/tools/bin/$tool" -version | sed -n 1p)
echo "$line"
[[ "$line" == *"$FFMPEG_BUILD"* ]] || { echo "::error::$tool reports '$line', not $FFMPEG_BUILD"; exit 1; }
done

# The guardrails hooks read every Bash command through shfmt and refuse the
# call without it, so a guardrails eval needs it on PATH. Same v3.14.1 pin
# and SHA-256 as pr-require-checks.yml. In /opt/tools, since each eval run
Expand Down
Loading