Skip to content

Commit a740a25

Browse files
localai-botmudler
andauthored
fix(ci): skip the master image rebuild for commits no image can see (#11223)
On 2026-07-30, 12 of the 23 queued runs of this workflow were commits like "add 1 new model to gallery" or a docs fix, each rebuilding all 18 container images. That was roughly 216 queued jobs producing byte-identical output, in a queue holding 1071 jobs with an oldest entry two days old. Verified against the shipped Dockerfile before assuming it: the final stage copies only entrypoint.sh, healthcheck.sh and the local-ai binary, there is no go:embed of gallery/ or docs/, and the gallery is fetched at runtime from github:mudler/LocalAI/gallery/index.yaml@master. A gallery-only commit produces an identical image, and the gallery change reaches users through GitHub whether or not an image is rebuilt, so nothing is delayed by skipping. Add a `changes` job that decides once whether the push can affect an image; the other 11 jobs take `needs: changes` and an `if:` on its output. A job gate rather than paths-ignore on the trigger, for two reasons that both fail silently if got wrong: - paths-ignore on `push` also applies to tag pushes, and a tag created on an existing commit carries an empty commits list. That would skip the release image build with no failure anywhere. The gate short-circuits to build for refs/tags/*, and for a base commit that is missing, zero or unresolvable -- the same run-everything posture the backend matrix filter takes for a truncated diff. - the merge jobs use `if: ${{ !cancelled() && ... }}`, and !cancelled() is true when a dependency is skipped, so they need the gate named explicitly or they would try to merge manifest lists for images never built. Checked the decision logic against real commits from the queue: the two gallery/docs commits resolve to build=false, the two code commits to build=true, and all three fallback paths (tag, zero base, unresolvable base) to build=true. Assisted-by: Claude:opus-5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
1 parent df7c946 commit a740a25

2 files changed

Lines changed: 76 additions & 20 deletions

File tree

.agents/ci-caching.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -194,7 +194,16 @@ Two properties this relies on:
194194
- `paths-ignore` skips a run only when **every** changed file matches, so a PR touching the gallery *and* Go code still runs everything. That is what makes the exclusion safe rather than a hole.
195195
- `master` carries no branch protection and no rulesets, so a skipped workflow reports no status and nothing waits on it. If required status checks are ever introduced, these four entries must be excluded from the required set or PRs will hang on "Expected — Waiting for status to be reported".
196196

197-
Deliberately **not** filtered: `image.yml` on master push still rebuilds all 18 container images for a gallery-only commit. Skipping it would mean the `master` and `latest` tags are not republished for that commit, which is a publishing-semantics decision rather than a pure cost one.
197+
### `image.yml` on master push is gated too, by a job rather than a path filter
198+
199+
The same reasoning applies to master pushes, and the volume is larger there: on 2026-07-30, **12 of the 23 queued `image.yml` runs** were commits like "add 1 new model to gallery" or a docs fix, each rebuilding all 18 container images.
200+
201+
`image.yml` now has a `changes` job that decides once whether the push can affect any image; the other 11 jobs carry `needs: changes` plus an `if:` on its output. Verified against the shipped `Dockerfile`: the final stage copies only `entrypoint.sh`, `healthcheck.sh` and the `local-ai` binary, there is no `go:embed` of `gallery/` or `docs/`, and the gallery is fetched at runtime from `github:mudler/LocalAI/gallery/index.yaml@master`. A gallery-only commit therefore produces byte-identical images, and the gallery change reaches users through GitHub immediately whether or not an image is rebuilt.
202+
203+
Two properties to preserve if you touch it:
204+
205+
- **It is a job gate, not `paths-ignore`.** `paths-ignore` on `push` also applies to tag pushes, and a tag created on an existing commit carries an empty commits list, which would silently skip the release image build. The gate short-circuits to "build" for `refs/tags/*`, and for any push whose base commit is missing, zero, or unresolvable.
206+
- **The merge jobs must name the gate explicitly.** They use `if: ${{ !cancelled() && ... }}`, and `!cancelled()` is true when a dependency is *skipped*, so without the extra condition they would run and try to merge manifest lists for images that were never built.
198207

199208
## The `DEPS_REFRESH` cache-buster (Python backends)
200209

.github/workflows/image.yml

Lines changed: 66 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,53 @@
1313
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
1414

1515
jobs:
16-
hipblas-jobs:
16+
# Decide once whether this push can change any image. Gallery metadata is
17+
# fetched at runtime and never baked into an image, and docs/markdown never
18+
# enter one, so a push confined to those paths produces byte-identical
19+
# images. On 2026-07-30, 12 of the 23 queued runs of this workflow were
20+
# commits like "add 1 new model to gallery" or a docs fix, each rebuilding
21+
# all 18 images.
22+
#
23+
# A job-level gate rather than `paths-ignore` on the trigger: paths-ignore
24+
# would also apply to tag pushes, and a tag created on an existing commit
25+
# carries an empty commits list, which would silently skip the release image
26+
# build. Tags short-circuit to "build" below, as does a push whose base
27+
# commit cannot be resolved -- the same run-everything posture the backend
28+
# matrix filter takes for a truncated diff.
29+
changes:
1730
if: github.repository == 'mudler/LocalAI'
31+
runs-on: ubuntu-latest
32+
outputs:
33+
build: ${{ steps.decide.outputs.build }}
34+
steps:
35+
- uses: actions/checkout@v7
36+
with:
37+
fetch-depth: 0
38+
- id: decide
39+
env:
40+
BEFORE: ${{ github.event.before }}
41+
AFTER: ${{ github.sha }}
42+
run: |
43+
set -euo pipefail
44+
emit() { echo "$2"; echo "build=$1" >> "$GITHUB_OUTPUT"; exit 0; }
45+
case "${GITHUB_REF}" in
46+
refs/tags/*) emit true "tag push: building every image" ;;
47+
esac
48+
if [ -z "${BEFORE:-}" ] || [ "${BEFORE}" = "0000000000000000000000000000000000000000" ] \
49+
|| ! git cat-file -e "${BEFORE}^{commit}" 2>/dev/null; then
50+
emit true "no resolvable base commit: building every image"
51+
fi
52+
files="$(git diff --name-only "${BEFORE}" "${AFTER}")"
53+
echo "changed files:"; echo "${files:-<none>}"
54+
[ -z "${files}" ] && emit true "empty diff: building every image"
55+
if echo "${files}" | grep -qvE '^(gallery/|docs/|examples/)|\.md$'; then
56+
emit true "push touches image-visible content: building"
57+
fi
58+
emit false "only gallery/docs/markdown changed: images identical, skipping"
59+
60+
hipblas-jobs:
61+
needs: changes
62+
if: github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true'
1863
uses: ./.github/workflows/image_build.yml
1964
with:
2065
tag-latest: ${{ matrix.tag-latest }}
@@ -47,7 +92,8 @@
4792
ubuntu-codename: 'noble'
4893

4994
core-image-build:
50-
if: github.repository == 'mudler/LocalAI'
95+
needs: changes
96+
if: github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true'
5197
uses: ./.github/workflows/image_build.yml
5298
with:
5399
tag-latest: ${{ matrix.tag-latest }}
@@ -155,8 +201,8 @@
155201
# merge whenever any matrix cell of the parent build fails or is
156202
# cancelled. Same fix as backend.yml's merge jobs — we still want to
157203
# publish the manifest list for tag-suffixes whose legs all succeeded.
158-
if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
159-
needs: core-image-build
204+
if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
205+
needs: [changes, core-image-build]
160206
uses: ./.github/workflows/image_merge.yml
161207
with:
162208
tag-latest: 'auto'
@@ -168,8 +214,8 @@
168214
quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
169215

170216
gpu-vulkan-image-merge:
171-
if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
172-
needs: core-image-build
217+
if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
218+
needs: [changes, core-image-build]
173219
uses: ./.github/workflows/image_merge.yml
174220
with:
175221
tag-latest: 'auto'
@@ -187,8 +233,8 @@
187233
# Each merge job needs only its parent build matrix and is filtered by
188234
# tag-suffix in image_merge.yml's artifact-download pattern.
189235
gpu-nvidia-cuda-12-image-merge:
190-
if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
191-
needs: core-image-build
236+
if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
237+
needs: [changes, core-image-build]
192238
uses: ./.github/workflows/image_merge.yml
193239
with:
194240
tag-latest: 'auto'
@@ -200,8 +246,8 @@
200246
quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
201247

202248
gpu-nvidia-cuda-13-image-merge:
203-
if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
204-
needs: core-image-build
249+
if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
250+
needs: [changes, core-image-build]
205251
uses: ./.github/workflows/image_merge.yml
206252
with:
207253
tag-latest: 'auto'
@@ -213,8 +259,8 @@
213259
quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
214260

215261
gpu-intel-image-merge:
216-
if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
217-
needs: core-image-build
262+
if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
263+
needs: [changes, core-image-build]
218264
uses: ./.github/workflows/image_merge.yml
219265
with:
220266
tag-latest: 'auto'
@@ -226,8 +272,8 @@
226272
quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
227273

228274
gpu-hipblas-image-merge:
229-
if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
230-
needs: hipblas-jobs
275+
if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
276+
needs: [changes, hipblas-jobs]
231277
uses: ./.github/workflows/image_merge.yml
232278
with:
233279
tag-latest: 'auto'
@@ -239,8 +285,8 @@
239285
quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
240286

241287
nvidia-l4t-arm64-image-merge:
242-
if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
243-
needs: gh-runner
288+
if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
289+
needs: [changes, gh-runner]
244290
uses: ./.github/workflows/image_merge.yml
245291
with:
246292
tag-latest: 'auto'
@@ -252,8 +298,8 @@
252298
quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
253299

254300
nvidia-l4t-arm64-cuda-13-image-merge:
255-
if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
256-
needs: gh-runner
301+
if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
302+
needs: [changes, gh-runner]
257303
uses: ./.github/workflows/image_merge.yml
258304
with:
259305
tag-latest: 'auto'
@@ -265,7 +311,8 @@
265311
quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
266312

267313
gh-runner:
268-
if: github.repository == 'mudler/LocalAI'
314+
needs: changes
315+
if: github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true'
269316
uses: ./.github/workflows/image_build.yml
270317
with:
271318
tag-latest: ${{ matrix.tag-latest }}

0 commit comments

Comments
 (0)