Skip to content

incluster-comp-pr-merged: use registry cache instead of GHA cache#89

Merged
matthyx merged 1 commit into
mainfrom
fix/registry-cache
Jul 20, 2026
Merged

incluster-comp-pr-merged: use registry cache instead of GHA cache#89
matthyx merged 1 commit into
mainfrom
fix/registry-cache

Conversation

@lyuval-armosec

@lyuval-armosec lyuval-armosec commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator

Root cause

error writing layer blob: failed to reserve cache — GitHub started issuing read-only Actions-cache tokens (2026-06-26) to runs that resolve to the default branch, for any triggering event not on a small trusted allowlist (push, workflow_dispatch, repository_dispatch, delete, registry_package, page_build, schedule). pull_request_target isn't on that list.

This workflow's callers (kubevuln and all other consumers) trigger on pull_request_target: types: [closed] — once a PR merges, GITHUB_REF resolves to main (the default branch), so the run gets a read-only cache token, and cache-to: type=gha's first blob-write fails with exactly this error. This is independent of build platform count, job structure, or cache scope.

Confirmed via GitHub Support in docker/build-push-action#1571: docker/build-push-action#1571 (comment)

Fix

Swap cache-from/cache-to from type=gha to type=registry,ref=${IMAGE_NAME}:buildcache. Registry cache authenticates via the existing Quay login and never touches GitHub's Actions Cache Service, so the trigger-trust restriction can't affect it.

Chose this over fixing the trigger itself (e.g. swapping to push on the target branch, as GitHub Support suggests) because maintainers sometimes push directly to main and do not want that to trigger a release — a naive push-based trigger would break that expectation across all 6 consumers. Registry cache fixes the actual bug without touching any consumer's trigger.

Also removes the actions: write permission from the docker-build job — it was added specifically to support type=gha cache writes and is no longer needed.

Test plan

  • Trigger pr-merged on a consumer repo (e.g. kubevuln) pointed at this branch and confirm no "failed to reserve cache" error
  • Confirm cache is reused on a second run
  • Confirm the resulting multi-platform image, attestation, and cosign signing are unaffected

@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The docker-build workflow step now reads from and writes to a registry-backed Docker Buildx cache instead of the GitHub Actions cache.

Changes

Docker Build Cache

Layer / File(s) Summary
Registry-backed Buildx cache
.github/workflows/incluster-comp-pr-merged.yaml
docker-build uses an image-specific registry cache for both cache-from and cache-to, with mode=max for cache export.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Possibly related PRs

Suggested reviewers: matthyx, varadraj75

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: switching the workflow from GitHub Actions cache to registry cache.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/registry-cache

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
.github/workflows/incluster-comp-pr-merged.yaml (1)

180-181: 🚀 Performance & Scalability | 🔵 Trivial

Add retention and storage monitoring for the registry cache.

mode=max stores intermediate layers in Quay; add a cleanup/retention policy and alerting so repeated builds do not accumulate unbounded cache storage.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/incluster-comp-pr-merged.yaml around lines 180 - 181,
Update the workflow’s registry cache configuration for cache-to in the build
step to add an appropriate Quay retention or cleanup policy and storage
monitoring/alerting for the buildcache reference, while preserving cache reuse
and mode=max behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/incluster-comp-pr-merged.yaml:
- Around line 180-181: Remove the actions: write entry from the job permissions
block governing the build steps in the workflow, while preserving registry
authentication and all existing read permissions. The cache-related build
configuration using cache-from and cache-to does not require GitHub Actions
write access.

---

Nitpick comments:
In @.github/workflows/incluster-comp-pr-merged.yaml:
- Around line 180-181: Update the workflow’s registry cache configuration for
cache-to in the build step to add an appropriate Quay retention or cleanup
policy and storage monitoring/alerting for the buildcache reference, while
preserving cache reuse and mode=max behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e34caeba-5660-47c6-b7cb-9bd63437bc68

📥 Commits

Reviewing files that changed from the base of the PR and between ad4904d and 73309b6.

📒 Files selected for processing (1)
  • .github/workflows/incluster-comp-pr-merged.yaml

Comment thread .github/workflows/incluster-comp-pr-merged.yaml
@github-actions

Copy link
Copy Markdown

Summary:

  • License scan: failure
  • Credentials scan: failure
  • Vulnerabilities scan: failure
  • Unit test: success
  • Go linting: failure

1 similar comment
@github-actions

Copy link
Copy Markdown

Summary:

  • License scan: failure
  • Credentials scan: failure
  • Vulnerabilities scan: failure
  • Unit test: success
  • Go linting: failure

lyuval-armosec added a commit that referenced this pull request Jul 20, 2026
Remove actions: write from docker-build job permissions. It was
added specifically to support the type=gha cache backend's write
requirement (commit ad4904d); now that this job uses type=registry
cache exclusively, it no longer touches GitHub's Actions Cache
service at all, so the permission is unnecessary.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Yuval Levy <yuvalle@armosec.io>
Alternative, simpler fix for 'error writing layer blob: failed to
reserve cache' (seen e.g. on kubevuln's pr-merged run:
https://github.com/kubescape/kubevuln/actions/runs/29737569926/job/88346893168).

The failure is caused by GitHub issuing read-only Actions-cache tokens
to pull_request_target-triggered runs that resolve to the default
branch (policy change 2026-06-26, see
https://github.blog/changelog/2026-06-26-read-only-actions-cache-for-untrusted-triggers/
and docker/build-push-action#1571 (comment)).
kubevuln's (and every other consumer's) pr-merged.yaml triggers on
pull_request_target: types: [closed], so once a PR merges into main,
cache-to: type=gha's first blob-write attempt fails with exactly
"failed to reserve cache" -- independent of job/matrix structure.

Switching cache-from/cache-to to type=registry avoids this mechanism
entirely: cache layers become ordinary content-addressed OCI blobs
pushed to Quay (already authenticated via the existing Login to Quay
step) instead of going through GitHub's actions-cache API, so the
trigger-trust restriction can't affect it.

All 6 confirmed consumers of this reusable workflow (synchronizer,
operator, prometheus-exporter, kubevuln, storage, http-request) build
Go binaries with a "--platform=$BUILDPLATFORM" cross-compile pattern
in their Dockerfiles and no RUN steps requiring target-arch execution,
so this keeps the original single docker-build job, single
multi-platform build-push-action call, and QEMU setup entirely
unchanged -- only the cache backend changes.

Also removes the now-unnecessary actions: write permission (added in
a recent main commit specifically to support type=gha cache writes;
no longer needed since this job doesn't touch GitHub's Actions Cache
Service at all).

Cache is stored under "${IMAGE_NAME}:buildcache" in Quay. Unlike GHA
cache, it isn't subject to GitHub's 10GB/repo eviction -- worth
monitoring Quay storage/retention for this tag over time.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Yuval Levy <yuvalle@armosec.io>
@github-actions

Copy link
Copy Markdown

Summary:

  • License scan: failure
  • Credentials scan: failure
  • Vulnerabilities scan: failure
  • Unit test: success
  • Go linting: failure

@github-actions

Copy link
Copy Markdown

Summary:

  • License scan: failure
  • Credentials scan: failure
  • Vulnerabilities scan: failure
  • Unit test: success
  • Go linting: failure

1 similar comment
@github-actions

Copy link
Copy Markdown

Summary:

  • License scan: failure
  • Credentials scan: failure
  • Vulnerabilities scan: failure
  • Unit test: success
  • Go linting: failure

@matthyx
matthyx merged commit 52bcdc0 into main Jul 20, 2026
9 checks passed
@matthyx
matthyx deleted the fix/registry-cache branch July 20, 2026 16:08
@github-actions

Copy link
Copy Markdown

Summary:

  • License scan: failure
  • Credentials scan: failure
  • Vulnerabilities scan: failure
  • Unit test: success
  • Go linting: failure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants