Skip to content

[Automation] - Stop the runner image build writing cache it never reads - #18681

Merged
izaac merged 1 commit into
rancher:masterfrom
izaac:executor-disk-report
Aug 7, 2026
Merged

[Automation] - Stop the runner image build writing cache it never reads#18681
izaac merged 1 commit into
rancher:masterfrom
izaac:executor-disk-report

Conversation

@izaac

@izaac izaac commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary

CI only change, no product code affected.

Occurred changes and/or fixed issues

init.sh built the runner image with --no-cache, but Pre-Clean already removes that tag at the start of every build, so the image was rebuilt regardless. The flag only bought a cold build while making BuildKit write one cache record per layer that the build never reads back.

Also adds cypress/jenkins/disk-report.sh to catch a regression here, since nothing else reports what a build costs a disk shared by sixteen executors and several other jobs.

Technical notes summary

  • Dockerfile.quickstart has nine layer instructions, and a build with the flag added exactly nine cache records and around 870MB. Without it, none.
  • Cached layers stay correct because Docker invalidates on content, and this matches how run.sh builds the same image locally.
  • The report only reads: df, docker info, docker system df, plus a counter under /tmp it removes itself. It never prunes, removes or stops anything, which matters on a daemon shared with other jobs.
  • It runs after Checkout because a script in the repo cannot run before the repo is in the workspace, and the after phase is guarded by fileExists.

Testing done

Two runs on the staging Jenkins job, both SUCCESS with 20/20 tests:

build cache before after net disk
before this change 208 records 217 records ~1.5GB
after this change 20.13GB 20.13GB 266MB

Cache totals came back byte identical, with every record accounted for by ageing rather than creation. The runner image also builds in 9 seconds rather than 75.

Checklist

  • The PR is linked to an issue and the linked issue has a Milestone, or no issue is needed
  • The PR has a Milestone
  • The PR template has been filled out
  • The PR has been self reviewed
  • The PR has a reviewer assigned
  • The PR has automated tests or clear instructions for manual tests and the linked issue has appropriate QA labels, or tests are not needed
  • The PR has reviewed with UX and tested in light and dark mode, or there are no UX changes
  • The PR has been reviewed in terms of Accessibility
  • The PR has considered, and if applicable tested with, the three Global Roles Admin, Standard User and User Base

@izaac izaac added QA/None QA/dev-automation Issues that engineers have written automation around so QA doesn't have look at this labels Aug 6, 2026
@izaac izaac added this to the v2.16.0 milestone Aug 6, 2026
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

🔄 Auto-Retry Failed Run

The Tests workflow failed for the commits below and has been automatically retried.
Follow up on a retried run through its link if needed.

Commit Workflow run Retried at
93eaf53 31069470870 2026-08-06 04:32 UTC

Pre-Clean already removes the runner image tag every build, so --no-cache
only bought a cold build while making BuildKit write one cache record per
layer that the build never reads back.

Measured on the agent: nine layer instructions, nine records and 870MB
added per build with the flag, none without it. The image also builds in
9 seconds rather than 75. Cached layers stay correct because Docker
invalidates on content, and this matches how run.sh builds it locally.

Also adds cypress/jenkins/disk-report.sh to catch a regression here. It
only reads: df, docker info and docker system df, plus a counter under
/tmp that it removes itself. It never prunes, removes or stops anything,
which matters on a daemon shared with other jobs.
@izaac
izaac force-pushed the executor-disk-report branch from efa1ef3 to 91b8b4d Compare August 6, 2026 21:32
@izaac izaac closed this Aug 6, 2026
@izaac izaac reopened this Aug 6, 2026
@izaac
izaac merged commit d243208 into rancher:master Aug 7, 2026
49 checks passed
@izaac
izaac deleted the executor-disk-report branch August 7, 2026 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

QA/dev-automation Issues that engineers have written automation around so QA doesn't have look at this QA/None

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants