Repository navigation
bootc-ubuntu-setup: Take the container stack from resolute on 24.04 - #1
Draft
cgwalters-bot wants to merge 3 commits into
Draft
cgwalters-bot wants to merge 3 commits into
cgwalters-bot wants to merge 3 commits into
Conversation
cgwalters-bot
force-pushed
the
bot/ubuntu2404-resolute-podman
branch
from
September 24, 2026 02:18
60d2e9a to
75b7ec7
Compare
cgwalters-bot
force-pushed
the
bot/ubuntu2404-resolute-podman
branch
from
September 29, 2026 23:03
75b7ec7 to
edcd4aa
Compare
cgwalters-bot
marked this pull request as ready for review
September 29, 2026 23:03
Collaborator
Author
|
Per tracker#118 (A), this lands before bootc-dev#60. Rebased onto upstream main (40859f0, which only touches the agent-pipeline files, so no retest), now at edcd4aa and ready for review. Approve or Generated-by: https://github.com/cgwalters/#llms |
cgwalters
previously approved these changes
Oct 2, 2026
cgwalters
left a comment
cgwalters
left a comment
There was a problem hiding this comment.
OK but we should really be bumping consumers to 26.04
cgwalters-bot
dismissed
cgwalters’s stale review
October 2, 2026 19:41
Upstream main moved and this no longer rebases cleanly; needs another look after the conflicts are resolved.
Collaborator
Author
|
Not promoted: |
On ubuntu-24.04 the package install takes libc6 2.41 from plucky (the plucky source is at the default priority), and libc6 Breaks the older base-files, so os-release reports 25.04 from then on. workarounds.sh re-reads os-release and so skips its Resolute QEMU setup, which only runs on 24.04: the Resolute QEMU (bootc-dev#53) has never actually been installed on 24.04 consumers such as bcvk and ostree. This redoes bootc-dev#54, which was merged into the stacked support-ubuntu-2604-signoff branch after bootc-dev#52 had already landed, and so never reached main. install-deps.sh already reads os-release once before installing anything; record what it read as a step output and have workarounds.sh use that. The virtualization test now asserts that QEMU is at least 10.2, since a guest can boot with the old one too. Note this means the Resolute QEMU step now really runs on 24.04 libvirt jobs. Consumers can't trust os-release after this action either, so expose the detected value as the host-os output, on the deprecated bootc-ubuntu-setup entry point too. Generated-by: AI
The renovate runner bump in bootc-dev#61 moved the "24.04" setup job to ubuntu-26.04 too, so the PR workflow stopped exercising the 24.04 package path, which is the one most consumers still use. bootc-dev#60 put the literal ubuntu-24.04 back, which the next such bump would rewrite again. Fold the two per-release jobs into one matrix. In bootc-dev#61 Renovate only rewrote literal runs-on values and left the matrix entries in test-virtualization.yml alone, so a matrix should keep 24.04 covered. 24.04 now also gets the heredoc check that only the 26.04 job had, plus a check that the host-os output names the runner image (also on the deprecated entry point). The bwrap AppArmor check stays 26.04-only: on 24.04 bwrap runs unconfined, so there is no transition to verify. Generated-by: AI
The plucky podman 5.4.1 and buildah 1.39.3 we install on 24.04 are built with go1.24.1, which has a runtime bug (golang/go#73141, fixed in go1.24.3): a thread exiting after LockOSThread can hand back its vDSO getrandom state without a P and segfault in mallocgc. The runners' 6.17 kernels have vDSO getrandom, and podman/buildah deliberately exit locked threads while setting up RUN steps, so some image builds die with exit status 139 and no error output. ostree's ci-bootc hit this in 27 jobs over the last three months; the analysis is in https://gist.github.com/cgwalters-bot/f19957abbef4121e4ee35210c48d0a86 The only stable series with a fixed 5.x build of the whole stack is resolute (go1.25.0): questing's podman 5.4.2 uses go1.24.4, but its buildah and skopeo are go1.24.1/1.24.2, and noble-backports has no podman. Resolute is also exactly what 26.04 runners have natively, and this action already uses it for QEMU. Pin resolute at priority 50 and select it with -t for the container packages only. conmon, netavark and aardvark-dns are named explicitly, since noble's versions (e.g. netavark 1.4) already satisfy podman 5.7's dependencies. Plucky stays enabled at its default priority, so the libvirt stack (libvirt 11, OVMF, and virtiofsd 1.13; noble's 1.10 rejects the options bcvk passes), just, and later consumer installs resolve exactly as before. Recommends are skipped, since with -t they would pull resolute's python3 through criu. The closure is still much more than the container stack: it upgrades libc6 to 2.43 along with libc-bin, libc-dev-bin, libc6-dev, libc6-i386 and locales, dpkg, dpkg-dev and libdpkg-perl to 1.23.7, passwd and uidmap to 4.17 (adding login.defs), base-passwd, make 4.4, libgpg-error and base-files. That has two consequences. needrestart would restart most host services mid-job (rsyslog crashed doing so in a test run), so configure it to only list them. And os-release says 26.04 after this step on every 24.04 job, not only on libvirt ones as with plucky's libc before, so anything later in the job that reads it, such as lsb_release, add-apt-repository or actions/setup-python, sees 26.04; the host-os output has the real value. The QEMU setup in workarounds.sh now reuses this source. The basic-requirements test checks that podman and buildah are not built with an affected Go, and now also runs a rootful container, which goes through netavark. Generated-by: AI
cgwalters-bot
force-pushed
the
bot/ubuntu2404-resolute-podman
branch
from
October 2, 2026 19:49
edcd4aa to
ec35d28
Compare
cgwalters-bot
marked this pull request as draft
October 4, 2026 00:05
Collaborator
Author
|
Converted back to a draft: this repository is a midstream (topic bot-midstream), whose main mirrors upstream and only moves by fast-forward sync, so its PRs are review vehicles and are never merged. Generated-by: https://github.com/cgwalters/#llms |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On ubuntu-24.04, bootc-ubuntu-setup installs podman 5.4.1 and buildah 1.39.3 from plucky, both built with go1.24.1. That Go has a runtime bug (golang/go#73141, fixed in go1.24.3) where a thread exiting after LockOSThread can segfault in mallocgc on kernels with vDSO getrandom, which the runners' 6.17 kernels have. podman/buildah exit locked threads while setting up RUN steps, so some image builds die with exit status 139 and no error output. Root cause analysis: https://gist.github.com/cgwalters-bot/f19957abbef4121e4ee35210c48d0a86
This takes the container stack (crun, buildah, podman, skopeo, conmon, netavark, aardvark-dns, passt, catatonit, just) from resolute on 24.04, which is go1.25 throughout and exactly what 26.04 runners have natively. No other source works: questing's podman is go1.24.4 but its buildah and skopeo are still go1.24.1/1.24.2, and noble-backports and the plucky/questing -updates pockets carry no podman. Resolute is pinned at priority 50 and only selected with
-t resolute, so plucky still supplies the libvirt stack as before.There are three commits:
host-osoutput.Caveats: the
-t resoluteclosure is wider than the container stack. It upgrades libc6 to 2.43 plus dpkg, passwd/uidmap, make and base-files. So needrestart is set to list-only (a test run saw rsyslog crash when it restarted services mid-job), and on every 24.04 job os-release reports 26.04 after this action, where before that only happened with libvirt enabled. Later steps that read os-release (lsb_release, add-apt-repository, actions/setup-python) will see 26.04;host-oshas the real value. This also conflicts with bootc-dev#60 (RHEL 10 support): whichever lands second has to port this into bootc-dev#60's layout (packages/ubuntu-24.04-base, the/pluckysuffix, the resolute pin), and bootc-dev#60's workarounds.sh has the same os-release re-read bug that commit 1 fixes.Which consumers are affected: I scanned failed runs on default branches since 2026-06-24 across the repos that use this action on 24.04, grepping job logs for exit 139/SIGSEGV. Only ostreedev/ostree's bootc.yaml was hit, in 27 jobs over 26 runs. It is still happening: a later scan found 20 failed ci-bootc integration jobs with this signature, including main on 09-14 and 09-17 and the ostree#3611 PR run on 09-23 (https://gist.github.com/cgwalters-bot/e97ca68a8e017b97e8a9848e9d4e5693). Nothing turned up in bcvk (143 failed jobs scanned), bootc (835), infra (365), composefs-rs (349), bootupd (52) or rpm-ostree (109). bootupd pins this action by digest and will pick the fix up on its next bump. rpm-ostree vendors its own copy of the action with plucky podman, so this PR doesn't cover it. Given no hits there, it's left alone for now.
Testing:
cgwalters-forge/actions/bootc-ubuntu-setup@bot/ubuntu2404-resolute-podman, at 60d2e9a), passed all 4 jobs (unit-tests and integration on stream9 and stream10), with podman 5.7.0, buildah 1.42.1 and skopeo 1.21.0~pre1 from resolute: https://github.com/cgwalters-bot/ostree/actions/runs/35943443277. An earlier revision also passed there: https://github.com/cgwalters-bot/ostree/actions/runs/35817157996.Generated-by: https://github.com/cgwalters/#llms
Review draft in cgwalters-forge, not upstream yet. This section is removed when the PR is opened upstream.
bootc-dev/actions, basemainPVTI_lADOE9oHIs4BlJLczg9kEMITo review:
/promoteon a line of its own, to open it upstream, ready for review. Either covers only the commits pushed so far.Signed-off-by: Colin Walters <walters@verbum.org>to the commits lacking it (the bot's and yours; anyone else's only if you ask), with you as committer./draftline (in the same comment or before) to open it upstream as a draft (/readyundoes that).