Release confidence gate - #42
Conversation
The v0.20.0 binary shipped broken because nothing executes what CI builds: mix test cannot reach the Zig launcher and release.yml never runs its artifacts. bin/checks/release_smoke.sh builds (or accepts) a packaged binary, asserts argv passthrough and daemon boot via EPMD, and tears down on every exit path. The script runs as an ExQuality writer stage in full mix quality, now wired to a new pre-push hook. Commits drop to the quick profile, which skips the smoke stage, dialyzer, and coverage. On a toolchain that cannot build the release (Zig 0.15 vs macOS 26 SDK) the stage skips visibly instead of failing.
Burrito 1.6.0 (PR #225) split the launcher's argv so Elixir's CLI consumes --daemon as a script path and halts with "No file named --daemon" — the v0.20.0 daemon crash-loop. 1.5.0 keeps the fused argv BEAM never parses; v0.19.0 shipped on it and ran fine. Pin CI Zig back to 0.15.2 to match: Burrito's Zig check is exact equality, and the mismatched pair is what broke the v0.20.0 tag build. Local mix release stays broken on macOS 26 (burrito #221, a build-host SDK problem); ubuntu cross-compiles are unaffected.
The v0.20.0 tag was pushed twice against an unproven release: CI failed on the tag once, and the rebuilt binary was broken at runtime. Nothing local exercises the ubuntu build config, so prove the release before the tag exists. check_release_preconditions! now dispatches release.yml via workflow_dispatch, watches the run, downloads the burrito_out artifact, and runs bin/checks/release_smoke.sh against the exact bytes CI built. Any failure — including a missing or unauthenticated gh — refuses to tag. Dispatch runs upload a workflow artifact and are barred from publishing a release; only refs/tags/ pushes hit action-gh-release.
0.20.0 rehearsal (verification step 10)Throwaway branch with Zig re-pinned to 0.16.0 (mismatched with Burrito 1.5.0), dispatched via the same Run 31752758337: failed as expected, at This is the exact v0.20.0 tag-build failure mode. The |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 80e08f07dc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex flagged that release.yml's push:tags trigger rebuilt on ubuntu-latest and published binaries mix tag never smoked (Burrito macOS binaries can't run the smoke script in that job). Drop the tag trigger; mix tag now publishes via gh release create using the exact artifact it already downloaded and smoked. Also filter gh run list by --commit/--event so dispatch polling can't grab a concurrent run's result, and add the missing changelog entry for the release confidence gate.
Release confidence gate
Goal: after
mix tag --minor, the release works — no tag rollbacks, no re-releases. v0.20.0 failed twice after the tag was pushed:workflow_dispatchdry-run of release.yml inside themix taggate.--daemonas a script path and halted withNo file named --daemon. Caught now bybin/checks/release_smoke.sh, which runs in fullmix quality(pre-push) and against the CI artifact at tag time.Changes
bin/checks/release_smoke.sh— builds (or accepts) a packaged binary; asserts argv passthrough (--version) and daemon boot (EPMD registration +sev status); teardown in a trap on every exit path; skips (exit 2) when the developer's daemon is running or the local toolchain can't build the release..quality.exs—release_smokecustom writer stage;quickprofile that skips it (plus dialyzer/coverage).mix quality --profile quick; new pre-push runs fullmix quality. Partly reverts 56232b7: dialyzer still blocks the push, just not every commit..claude/settings.jsoncommit gate matches, and now uses$CLAUDE_PROJECT_DIRinstead of a hardcoded path.mix taggate —check_release_preconditions!now dispatches release.yml, watches the run, downloads the artifact, and smokes the exact bytes CI built. Any failure (including missing/unauthenticatedgh, or a skipped smoke) refuses to tag. Dispatch runs upload a workflow artifact and can never publish a release.Red validation (smoke script vs the shipped v0.20.0 binary)
No stray process after the run; a registered
severanceEPMD node makes the script skip (exit 2) instead of racing the real daemon.Green validation (CI artifact on Burrito 1.5.0)
Dispatched run 31752527839 on this branch, downloaded the
burrito_outartifact, smokedsev_macos_arm64:No stray process, no lingering EPMD registration after the run.
Known local ceiling
MIX_ENV=prod mix release sevstill fails on this machine (burrito-elixir/burrito#221: Zig 0.15.2 libc defs vs macOS 26 SDK — a build-host problem; ubuntu cross-compiles are unaffected). Locally the smoke stage reports a visible skip:It goes green locally when burrito #230 lands.