Skip to content

ephemeral: Mask bootloader-update.service - #376

Merged
cgwalters merged 1 commit into
bootc-dev:mainfrom
cgwalters-forge:bot/mask-bootloader-update
Sep 28, 2026
Merged

cgwalters merged 1 commit into
bootc-dev:mainfrom
cgwalters-forge:bot/mask-bootloader-update

Conversation

@cgwalters-bot

Copy link
Copy Markdown
Contributor

Decision needed before this goes anywhere: the same problem can be fixed in bootupd instead, and masking the unit here breaks bootupd's own proposed CI check (see "Alternatives" below).

bootupd's bootloader-update.service looks for the block device backing /boot or /sysroot. In an ephemeral VM the root is virtiofs, so it fails with "Failed to find block device from /boot or /sysroot" and the system comes up degraded. That breaks any test that boots an image with bcvk ephemeral and checks systemctl is-system-running.

There is never a bootloader to update in an ephemeral VM, so this masks the unit by default on the kernel command line, like systemd-journal-flush.service already is. Masking a unit that doesn't exist is harmless, so images without bootupd are unaffected. Images already in use carry the current bootupd, so a fix there only helps once they're rebuilt with it.

test_run_ephemeral_ssh_system_command used to run systemctl is-system-running || true, which checked nothing. It now waits for boot to finish and asserts the state is running (printing the failed units otherwise), so a unit that fails in every ephemeral boot is caught here.

Alternatives, and a conflict

  • update: Skip bootloader update when no block devices back the root coreos/bootupd#1072 (open) proposes making bootupctl update skip the update when no block device backs the root.
  • Smaller: the unit already has an ExecCondition that skips live environments ([[ ! $(findmnt -n -o FSTYPE /sysroot) =~ ^(erofs|squashfs)$ ]], see the unit). Adding virtiofs to that regex would make the unit skip cleanly instead of failing.
  • bootupd#1072 also adds a CI smoke test that runs systemctl is-active bootloader-update.service under bcvk ephemeral run-ssh. With this mask the unit never runs, so that assertion would fail, and this PR adds no kernel argument or flag to opt out of the mask.

So the options are: fix it in bootupd (#1072 or the ExecCondition change) and drop this PR, or keep the bcvk mask and add an opt-out (e.g. a flag that skips the default masks).

Testing

On a 16-core RHEL 10.2 devspace with KVM:

  • make validate and cargo test pass (82 + 22 unit tests).
  • The tightened integration test fails with bcvk from main (degraded, bootloader-update.service loaded failed) and passes with this change (twice in a row, ~19s each) on centos-bootc:stream10.
  • By hand on fedora-bootc:44: degraded with bootloader-update.service failed on main, running with this change.

One caveat: an earlier run of the new test, while two other VM tests and a toolbox experiment were running on the same machine, timed out waiting for the VM to come up (240s). It passed quickly when rerun, so I believe that was contention, not this change.

CI note (2026-09-24): the integration-tests (3) failure is unrelated to this change: test_run_ephemeral_dns_resolution failed with "Connection timed out during banner exchange" from ephemeral run-ssh, the ConnectTimeout flake targeted by cgwalters-forge#8. Reran the failed jobs once.

The Signed-off-by: Colin Walters <walters@verbum.org> on these commits was added on cgwalters's approval of the review draft: cgwalters-forge#5 (review)

Generated-by: https://github.com/cgwalters/#llms

bootupd's bootloader-update.service looks for the block device backing
/boot or /sysroot, and in an ephemeral VM the root is virtiofs, so it
fails with "Failed to find block device from /boot or /sysroot". The
system then comes up "degraded", which breaks every test that boots an
image with bcvk and checks systemctl is-system-running, forcing each
consumer (e.g. bootc-dev/infra, bootupd CI) to mask it themselves.

There is never a bootloader to update in an ephemeral VM, so mask it
by default like systemd-journal-flush.service. coreos/bootupd#1072
makes bootupd skip this case itself, but existing images will carry
older bootupd for a long time.

The ephemeral system-command integration test now asserts the VM
reaches "running", so a unit failing in every ephemeral boot is caught
here rather than by bcvk's consumers.

Generated-by: AI
Signed-off-by: Colin Walters <walters@verbum.org>

@cgwalters cgwalters left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A little hacky but will let us support systems with older bootupd cheaply enough so let's do it

@cgwalters
cgwalters merged commit d857ff7 into bootc-dev:main Sep 28, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants