bootc: install to-existing-root fails when / or /run is MS_SHARED (#2455) #312

Activity

1. cgwalters-bot

   added

   target:bootc-dev

   P1

   on Oct 3, 2026
2. cgwalters-bot

   added this to Workstreamon Oct 3, 2026
3. cgwalters-bot

   added

   dispatchOpt in to the dispatcher's auto-dispatch of a devspace runOpt in to the dispatcher's auto-dispatch of a devspace run

   and removed

   dispatchOpt in to the dispatcher's auto-dispatch of a devspace runOpt in to the dispatcher's auto-dispatch of a devspace run

   on Oct 3, 2026
4. cgwalters-bot

   added a parent issue on Oct 3, 2026

   • Epic: bootc and ostree family (parked) #333
5. cgwalters-bot

   added

   P2

   and removed

   P1

   on Oct 3, 2026
6. cgwalters-forge

   locked as resolved and limited conversation to collaborators on Oct 18, 2026
7. cgwalters-bot commented on Oct 22, 2026

   

   cgwalters-bot

   on Oct 22, 2026

   Last edited by cgwalters-bot

   CollaboratorAuthor

   More actions

   ✅ dispatch ended well. Result: #312 (comment)
8. cgwalters-bot commented on Oct 22, 2026

   

   cgwalters-bot

   on Oct 22, 2026

   CollaboratorAuthor

   More actions

   Triage (as of 21 Oct 2026): still wanted. The bug is valid and has a clear lead, but nobody has reproduced it in this pipeline yet. bootc-dev/bootc#2455 is still open, with no comments and no PR that links to it. The code it describes is unchanged on bootc main (de7ae55).

   What I read:

   • bind_storage_roots() (podstorage.rs#L95-L143) runs unshare(CLONE_NEWNS) and then bind-mounts onto /run/bootc/storage. It never changes propagation in the new namespace. Compare chroot.rs, which runs mount_change("/", PRIVATE|REC) right after its unshare (chroot.rs#L138-L148). If /run is shared, this bind mount leaks into the peer group, and so into the host.
   • This is the likely reason the reporter sees the bug and the tested paths don't: install skips ensure_self_unshared_mount_namespace (unshare -m --propagation=slave) whenever --source-imgref is given (install.rs#L1691, #L1875-L1879). That skip came from eb620cce ("install: Don't unshare if we have an external source"), which was meant for osbuild, where bwrap already supplies a private namespace. Run on a bare host, as in the issue's repro, bootc stays in the host's shared namespace. There it also move-mounts a recursive copy of host / onto /target, which sits under a shared / (install.rs#L2697-L2709). In the usual podman run --privileged --pid=host path, bootc does unshare, and the container's mounts are private.
   • The exact ENOENT is not explained yet. CStorage::create checks for podman first (podstorage.rs#L304-L309), so the error most likely comes from a syscall in the pre_exec hook during spawn, probably the mount_bind. Finding out needs a real Fedora 44 host or VM; the hosted runner can't do it.

   Open questions:

   1. Which fix, or both? (a) Make the namespace in bind_storage_roots rslave/private right after unshare. This is small and local. The existing comment says "making the mounts recursively private didn't help", but that was about a different EINVAL, so the change has to be checked by reproducing. (b) Unshare even with --source-imgref when bootc is not already in its own mount namespace. ensure_self_unshared_mount_namespace already returns early in that case, so osbuild should be unaffected. (b) is broader. The plan below starts with (a) and adds (b) only if the repro shows (a) is not enough.
   2. Is running bootc install to-existing-root directly on a host, outside a container, supported? The docs only describe the container form. The fix is cheap either way, so I'd treat it as a bug.

   {
     "schema": "agent-board/job-plan/v1",
     "repo": "cgwalters-forge/tracker",
     "item": 312,
     "kind": "implement",
     "runner": "hosted",
     "brief": "Fix https://github.com/bootc-dev/bootc/issues/2455 in bootc-dev/bootc. Running `bootc install to-existing-root --source-imgref ...` directly on a host whose / and /run have shared propagation (the systemd default) fails with `Creating imgstorage: Initializing images: No such file or directory`. Running `mount --make-private / /run` first works around it.\n\nWhat triage found: `bind_storage_roots()` in crates/lib/src/podstorage.rs runs fchdir(storage_root); unshare(CLONE_NEWNS); mount_bind(\".\", STORAGE_ALIAS_DIR) in a pre_exec hook, and never changes propagation in the new namespace. Compare crates/utils/src/chroot.rs, which calls mount_change(\"/\", PRIVATE|REC) right after its unshare. With --source-imgref, install also skips ensure_self_unshared_mount_namespace (crates/lib/src/install.rs, the `!external_source && BOOTC_SKIP_UNSHARE` check, added in commit eb620cce for osbuild). So on a bare host, bootc runs in the host's shared namespace.\n\nSteps:\n1. In the pre_exec hook of bind_storage_roots, after the unshare and before the bind, make the namespace recursively slave or private: rustix::mount::mount_change(c\"/\", MountPropagationFlags::SLAVE|REC) or PRIVATE|REC. Pick whichever keeps host mounts visible; SLAVE matches what ensure_self_unshared_mount_namespace does. Keep the fchdir before the unshare. Update the long comment there to explain why. Only make async-signal-safe calls between fork and exec.\n2. Reproduce the bug and confirm the fix if the runner allows it, in a privileged container or a VM: set up a shared tmpfs standing in for /run, show that the bind mount leaks into the parent namespace before the fix, and show that it does not after. If the hosted runner cannot create mount namespaces, say so and do not claim a reproduction. In the commit message, explain the root cause as far as you established it, and mark what is still a hypothesis, such as the exact source of the ENOENT.\n3. Leave the external_source unshare skip alone in this PR unless step 2 shows that step 1 is not enough. If it is not enough, describe that in the PR body instead of widening the change.\n\nTests: add a unit or integration test if practical. Gate it on root/CAP_SYS_ADMIN and skip it otherwise, like the existing privileged tests. It should spawn a command through bind_storage_roots in a namespace where / is shared, then check that no new mount appeared at /run/bootc/storage in the parent. Run `cargo fmt --check`, `cargo clippy --all-targets` and `cargo test` for the touched crates, and report the real exit codes. Commit in the bootc style (`podstorage: ...`) with a `Generated-by: AI` trailer and no Signed-off-by. Open one draft PR."
   }
   

   Generated by dispatch · ◷