ci: size runners to the macOS baseline, self-heal docker cache corruption - #51
Conversation
Every target now aims for the ~160s macOS floor (smallest mac label is 6vcpu): gnu to 8vcpu (172s measured), musl/arm/windows to 16vcpu (estimated ~2.8-3.5 min, closest achievable). 16vcpu labels verified via smoke run. Docker package jobs gain a repair-retry: a corrupted base layer on the builder's sticky disk (three identical rocm digest-commit failures) is evicted via buildx prune and rebuilt once; healthy builds are untouched. actionlint labels pruned to the used set.
|
Warning Review limit reached
Next review available in: 22 minutes Limit details: You’ve used all 3 included reviews currently available. Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
Comment |
There was a problem hiding this comment.
Pull request overview
This PR updates the release and Docker publishing GitHub Actions workflows to (1) align runner sizing across targets to a macOS baseline wall-clock and (2) add a one-time self-healing retry path for Docker image publishing when the Buildx cache is corrupted.
Changes:
- Rebalanced
release.ymlmatrix runner labels to target ~macOS baseline duration (macOS is the fixed floor). - Added a failure-only “prune + retry once” repair path around
build-push-actionindocker.ymlto recover from persistent builder cache corruption. - Pruned
.github/actionlint.yamlrunner labels to match the labels actually used by workflows.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| .github/workflows/release.yml | Updates build matrix runner sizes and clarifies the sizing rationale in comments. |
| .github/workflows/docker.yml | Adds a continue-on-error first publish step, prunes Buildx cache on failure, and retries once. |
| .github/actionlint.yaml | Removes unused Blacksmith labels and adds the newly used 16vcpu labels to keep actionlint aligned. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Runner tuning principle
macOS is the fixed floor: the smallest Blacksmith macOS label is 6vcpu (~160s builds), so the release can never finish faster than ~2.7 min. Every other target is sized to land at that baseline — not faster (wasted vcpus), not slower (gates the release).
The 16vcpu labels were smoke-verified (all three picked up and ran). Sizing rationale with the measured baselines lives in a comment above the matrix.
Docker repair-retry
The rocm publish has now failed 3 times with the identical error:
failed commit on ref "layer-sha256:4c8a4cb..." unexpected commit digest. Same layer ref across the v1 cache-key, the fresh docker-v2 key, and a solo rerun — so it is a corrupted base-image layer (rocm/dev-ubuntu-24.04:6.4.4-complete) sitting on the persistent builder's disk, not a concurrency race and not fixed by rotating the cache-key.build-push-actionnow runs withcontinue-on-error, and on failure the job prunes the builder (docker buildx prune -af, evicting the corrupt layer) and retries once. Healthy builds never pay for this; sick ones self-heal instead of failing the release.This PR is also the vehicle for backfilling
1.0.1-rocm: after merge I will re-dispatch the workflow for tag v1.0.1, which exercises the repair path immediately.Validation
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is enabled.Summary by cubic
Sizes release runners to the macOS baseline and adds a self-healing Docker publish retry. Other targets now finish near the ~160s macOS floor, and Docker publishes recover from corrupted base-image layers instead of failing the release.
gnutoblacksmith-8vcpu-ubuntu-2404(172s measured),musltoblacksmith-16vcpu-ubuntu-2404,aarch64toblacksmith-16vcpu-ubuntu-2404-arm,windowstoblacksmith-16vcpu-windows-2025; macOS unchanged. This removes slow-target gating without overprovisioning faster targets.actionlint.yamllabels pruned to match.useblacksmith/build-push-action@v2run usescontinue-on-error; on failure we rundocker buildx prune -afand retry once. This fixes the recurring “unexpected commit digest” caused by a corrupt layer inrocm/dev-ubuntu-24.04:6.4.4-complete. Healthy builds keep cache and do not pay the prune cost.v1.0.1to backfill1.0.1-rocm. No tag format or artifact changes.Written for commit 3c99e4c. Summary will update on new commits.