Skip to content

bootc: install with composefs for containers built for it - #4

Closed
cgwalters-bot wants to merge 5 commits into
mainfrom
bot/bootc-composefs-backend
Closed

cgwalters-bot wants to merge 5 commits into
mainfrom
bot/bootc-composefs-backend

Conversation

@cgwalters-bot

@cgwalters-bot cgwalters-bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Collaborator

Detect bootc containers built for the composefs backend the way bootc install does since bootc-dev/bootc#2500: the container ships /usr/lib/composefs/setup-root-conf.toml and has no ostree prepare-root.conf in /usr/lib/ostree or /etc/ostree (bootc-installation(7), "The storage backend"). For those, install with composefs like for UKI images: pass the composefs option to bootc install, skip the post-install stages that mount the ostree deployment (they fail with "Could not find deployment"), and report the dropped customizations as a manifest warning, as for UKI images. Kernel arguments are still passed to bootc install. The pxe-tar-xz image type, whose live root comes from the ostree deployment, is refused for these images.

Stacked on #2; the last three commits are new. The first of them adds the review marker that the repository's AGENTS.md requires at the top of README.md; only the human author removes it.

Testing, on a 16-core RHEL 10 devspace (cgwalters-devspace-37017367119; logs):

  • go test -count=1 -race ./... in a privileged Fedora 43 container (as the unit test CI job): the same 16 cmd/image-builder tests fail as on the base commit; everything else passes, including TestComposefsBackend, TestManifestComposefsCustomizationsWarn, TestManifestPXEComposefsUnsupported and TestRawBootcImageSerializeComposefs. go vet and gofmt clean.
  • tools/gen-manifest-checksums.sh on each commit: no checksum changes.
  • End to end: a composefs image built from bootc main (7d6df9a0) with BOOTC_variant=composefs on fedora-bootc:44, which has the marker, no prepare-root.conf and no composefs setting in its install configuration, made into a qcow2 by this branch's image-builder container with a kernel.append blueprint. It boots (UEFI, qemu-kvm) to a composefs (BLS, grub) deployment with a verity=require composefs root and the karg on the command line. A customizations.user blueprint fails with the bootc container is built for the composefs backend, which does not support: customizations.user. With the previous head of this PR, which read the composefs-backend install config key, the same image fails in org.osbuild.ostree.deployment with "Could not find deployment".
  • Stock quay.io/fedora/fedora-bootc:44 (ostree), with kargs, a user, a group and files: builds and boots an ostree deployment with all of them.

Generated-by: https://github.com/cgwalters/#llms


Review draft in cgwalters-forge, not upstream yet. This section is removed when the PR is opened upstream.

  • Upstream: osbuild/image-builder, base main
  • Board item: PVTI_lADOE9oHIs4BlJLczg9kEmI
  • Fork CI: off; the devspace testing described above is this PR's CI, and upstream CI runs once it is opened there

To review:

  • Approve, or comment /promote on a line of its own, to open it upstream, ready for review. Either covers only the commits pushed so far.
  • If upstream requires DCO, approving also signs off: promote adds Signed-off-by: Colin Walters <walters@verbum.org> to the commits lacking it (the bot's and yours; anyone else's only if you ask), with you as committer.
  • Add a /draft line (in the same comment or before) to open it upstream as a draft (/ready undoes that).
  • Close to drop it.
  • Edit the title and description freely: they become the upstream PR's. Review comments are squashed into the commits they concern, with a reply here.

bootc-image-builder prints manifest warnings instead of failing, but
they were written without a prefix or trailing newline, so the next
progress message ran into the warning text ("...not supported: fooDone
generating manifest") and it was easy to miss. Print each warning as a
"WARNING:" line like the other warnings of the CLI. This also changes
the output of "image-builder manifest/build --ignore-warnings", which
uses the same code.

Generated-by: AI
For a bootc image with a unified kernel, the raw_bootc pipeline skips
every customization stage after "bootc install" and drops the kernel
arguments, since the command line is baked into the UKI. Users, files,
kargs and extra mountpoints given in the blueprint were therefore
silently ignored, and the build succeeded with an image that lacks
them (e.g. no login user).

Report them the same way as other unsupported blueprint options: as a
manifest warning, which the image-builder CLI turns into an error
unless --ignore-warnings is given and bootc-image-builder prints. The
container's own partition table and the default kargs from our image
definitions are not user input and are not reported.

Supporting the /etc and /var customizations is tracked in osbuild#2560.

Generated-by: AI
Per AGENTS.md, for the human author to remove once reviewed.

Generated-by: AI
bootc installs a container with the composefs backend, even without a
UKI, when it ships /usr/lib/composefs/setup-root-conf.toml and has no
ostree prepare-root.conf (see "The storage backend" in
bootc-installation(7)). Record that so the disk image code can take it
into account.

Generated-by: AI
osbuild's bootc install-to-filesystem stage runs bootc from the
container, which installs a container that ships setup-root-conf.toml
and no ostree prepare-root.conf with composefs even though it has no
UKI. The customization stages after it then fail to mount the ostree
deployment ("Could not find deployment").

Treat such containers like UKI images: pass the composefs option, skip
the stages that work on the ostree deployment and report the
customizations that are dropped. Kernel arguments are still passed to
bootc install, which puts them in the BLS entries. The PXE tar image
type builds its live root from the ostree deployment, so it is refused
for these containers.

Generated-by: AI
@cgwalters-bot
cgwalters-bot force-pushed the bot/bootc-composefs-backend branch from 0660d34 to 97fe5d1 Compare October 2, 2026 14:46
@cgwalters-bot cgwalters-bot changed the title bootc: install with composefs when the install configuration selects it bootc: install with composefs for containers built for it Oct 2, 2026
@cgwalters-bot

Copy link
Copy Markdown
Collaborator Author

Reworked for bootc-dev/bootc#2500, which dropped the composefs-backend install config key: image-builder now detects the composefs backend from /usr/lib/composefs/setup-root-conf.toml with no ostree prepare-root.conf, as bootc install does. New head 97fe5d1: d529ba2 was replaced by 39935a2, 0660d34 was reworded into 97fe5d1 (the error text now says "built for the composefs backend"), and 08bfe0f adds the README.md review marker that AGENTS.md requires (yours to remove). Retested on a devspace, including a control showing the old head fails on such an image with "Could not find deployment"; see the description.

Generated-by: https://github.com/cgwalters/#llms

@cgwalters

Copy link
Copy Markdown

Are you sure we need this with the latest bootc shouldn't image builder just call bootc install to-filesystem which should automatically detect the state of this?

Or is this a --source-imgref problem? But at least with bootc-image-builder today we always have the image fetched, is that the case for /usr/bin/image-builder too?

@cgwalters-bot

Copy link
Copy Markdown
Collaborator Author

You are right about bootc. On main it picks composefs by itself, so the --composefs-backend that #4 passes for non-UKI images is redundant in the default case. What image-builder needs the backend for is its own stages after bootc runs, and it decides those when it generates the manifest.

How /usr/bin/image-builder does it today:

  • Yes, the image is always local. By default the container source is Local (bootc_imagetype.go#L235-L245), and image-builder podman runs the image to resolve it first. osbuild then runs bootc install to-filesystem --source-imgref containers-storage:… --skip-fetch-check from the build root (stage). Unless --bootc-build-ref is given, the build root is the target image (bootc.go#L135).
  • bootc reads setup-root-conf.toml and prepare-root.conf from the running root / (install.rs#L1646, #L1773), so in the default case it chooses composefs on its own. Two gaps: with --source-imgref, target_rootfs is None (#L1685), so bootc never detects a UKI and image-builder has to keep passing the flag for UKIs. And with a separate build container, image-builder copies only /usr/lib/bootc/install/ and /usr/lib/ostree/ from the target (bootc_disk.go#L70), not /usr/lib/composefs/.
  • The actual failure is in image-builder. For every non-UKI image, the manifest adds post-install stages that mount org.osbuild.ostree.deployment (raw_bootc.go#L199-L212). Because of the default MOUNT_CONFIGURATION_UNITS (bootc_disk.go#L40), there is always at least the mount-unit stage, even with an empty blueprint. On a composefs install that stage fails with "Could not find deployment". bootc: install with composefs for containers built for it #4 fixes that by skipping those stages, not by telling bootc which backend to use.

A cleaner fix that doesn't duplicate bootc's detection: bootc install mount handles both backends. So image-builder could use #6's org.osbuild.bootc.deployment mount for all post-install stages whenever the build root's bootc has install mount, and keep ostree.deployment otherwise. A bootc too old for install mount (bootc-dev/bootc#2483) is also too old for the auto-detection (bootc-dev/bootc#2500), so it installs ostree for non-UKI images anyway. After that, image-builder only needs to know about UKIs (already detected) and the pxe-tar-xz case, whose live root comes from the ostree deployment.

Proposal: close #4, and rework #6 into that single PR. It would check the bootc capability instead of the backend, pass --composefs-backend only for UKIs, and add /usr/lib/composefs/ to the files copied into a separate build container. The cost is that ostree images built with a new bootc also move to the new mount, which needs an ostree end-to-end test and upstream buy-in, and none of this works until bootc releases osbuild#2483 and osbuild#2500 (neither is in v1.16.14). If you would rather keep the narrow fix that works with released osbuild, #4 can stay as it is. Tell me which one you want.

Generated-by: https://github.com/cgwalters/#llms

@cgwalters

Copy link
Copy Markdown

A cleaner fix that doesn't duplicate bootc's detection: bootc install mount handles both backends. So image-builder could use #6 org.osbuild.bootc.deployment mount for all post-install stages whenever the build root's bootc has install mount, and keep ostree.deployment otherwise. A bootc too old for install mount (bootc-dev/bootc#2483) is also too old for the auto-detection (bootc-dev/bootc#2500), so it installs ostree for non-UKI images anyway. After that, image-builder only needs to know about UKIs (already detected) and the pxe-tar-xz case, whose live root comes from the ostree deployment.

Yes make a new draft for that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

2 participants