Upgrade drops to emergency shell on LVM systems: rd.lvm.lv= args inherited by the upgrade boot entry prevent activation of non-root LVs in the upgrade initramfs

[yuravk]

Actual behavior
After running leapp upgrade and rebooting into the upgrade initramfs (RHEL-Upgrade-Initramfs), the system drops into the emergency shell. All logical volumes that back fstab mount points other than / and swap are never activated:

dracut-initqueue[1293]: Scanning devices md126 for LVM logical volumes vg00raid1/lv_root vg00raid1/lv_swap
systemd[1]: Timed out waiting for device dev-mapper-vg00raid1\x2dlv_home.device - /dev/mapper/vg00raid1-lv_home.
systemd[1]: Dependency failed for sysroot-home.mount - /sysroot/home.
systemd[1]: Dependency failed for local-fs.target - Local File Systems.
systemd[1]: Timed out waiting for device dev-mapper-vg00raid1\x2dlv_var.device - /dev/mapper/vg00raid1-lv_var.
systemd[1]: Dependency failed for sysroot-var.mount - /sysroot/var.
systemd[1]: Timed out waiting for device dev-mapper-vg00raid1\x2dlv_data.device - /dev/mapper/vg00raid1-lv_data.
systemd[1]: Dependency failed for sysroot-data.mount - /sysroot/data.

In the emergency shell, lvm lvs -o lv_name,lv_active shows only lv_root and lv_swap active; lv_var, lv_home, lv_data are inactive. Running lvm vgchange -ay and pressing Ctrl-D lets the upgrade continue and finish successfully - which confirms the missing piece is LV activation only.

To Reproduce

1. Install RHEL 9 with LVM: LVs for /, swap, and at least one more fstab mount (e.g. /var or /home). mine is:

NAME                    MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
sr0                      11:0    1  14.5G  0 rom   
nvme0n1                 259:0    0    20G  0 disk  
├─nvme0n1p1             259:1    0     1G  0 part  
│ └─md127                 9:127  0  1023M  0 raid1 /boot
├─nvme0n1p2             259:2    0   611M  0 part  
│ └─md125                 9:125  0 610.9M  0 raid1 /boot/efi
└─nvme0n1p3             259:3    0  18.4G  0 part  
  └─md126                 9:126  0  18.4G  0 raid1 
    ├─vg00raid1-lv_root 253:0    0     8G  0 lvm   /
    ├─vg00raid1-lv_swap 253:1    0     2G  0 lvm   [SWAP]
    ├─vg00raid1-lv_data 253:2    0   1.4G  0 lvm   /data
    ├─vg00raid1-lv_var  253:3    0     6G  0 lvm   /var
    └─vg00raid1-lv_home 253:4    0     1G  0 lvm   /home
nvme0n2                 259:4    0    20G  0 disk  
├─nvme0n2p1             259:5    0     1G  0 part  
│ └─md127                 9:127  0  1023M  0 raid1 /boot
├─nvme0n2p2             259:6    0   611M  0 part  
│ └─md125                 9:125  0 610.9M  0 raid1 /boot/efi
└─nvme0n2p3             259:7    0  18.4G  0 part  
  └─md126                 9:126  0  18.4G  0 raid1 
    ├─vg00raid1-lv_root 253:0    0     8G  0 lvm   /
    ├─vg00raid1-lv_swap 253:1    0     2G  0 lvm   [SWAP]
    ├─vg00raid1-lv_data 253:2    0   1.4G  0 lvm   /data
    ├─vg00raid1-lv_var  253:3    0     6G  0 lvm   /var
    └─vg00raid1-lv_home 253:4    0     1G  0 lvm   /home

The installer puts rd.lvm.lv=<vg>/root rd.lvm.lv=<vg>/swap on the kernel cmdline.
2. leapp preupgrade (no inhibitors), leapp upgrade, reboot.
3. The boot of the upgrade initramfs times out on every dev-mapper-<vg>-<other-lv>.device and enters emergency mode.

Expected behavior
All LVs required to mount the fstab entries under /sysroot are activated in the upgrade initramfs and the upgrade proceeds without manual intervention.

System information:

• Source OS: RHEL 9.8 (kernel 5.14.0-687.42.1.el9_8), target: RHEL 10.2, x86_64 (UEFI)
• Leapp:

leapp-upgrade-el9toel10-deps-0.24.0-1.el9.noarch
libdb-utils-5.3.28-57.el9_6.x86_64
leapp-upgrade-el9toel10-0.24.0-1.el9.noarch
leapp-0.21.0-1.el9.noarch
python3-leapp-0.21.0-1.el9.noarch
leapp-deps-0.21.0-1.el9.noarch
systemd-container-252-67.el9_8.4.x86_64

Root cause
add_upgrade_boot_entry creates the upgrade entry with grubby --copy-default, so it inherits the rd.lvm.lv=<vg>/<lv> arguments of the default entry:

/usr/sbin/grubby --add-kernel /boot/vmlinuz-upgrade.x86_64 --initrd /boot/initramfs-upgrade.x86_64.img \
  --title RHEL-Upgrade-Initramfs --copy-default --make-default --args '...'

With rd.lvm.lv= present (and no rd.lvm.vg=), dracut's 90lvm/lvm_scan.sh activates only the listed LVs via lvchange and skips the whole-VG vgchange -ay branch (it runs only when $LVS is empty or rd.lvm.vg= is given). On a normal boot that is fine because the booted system activates the remaining LVs afterwards, but the upgrade initramfs never switches root - it mounts all fstab entries under /sysroot inside the initramfs, so nothing ever activates the other LVs.

Notably, collect_undesired_args() in add_upgrade_boot_entry already removes the inherited rd.lvm.lv args from the upgrade entry - but only when live mode is enabled. The regular upgrade initramfs has exactly the same problem.

Workaround
In the emergency shell: lvm vgchange -ay (plus mdadm --assemble --scan on 0.24.0-1 if md arrays are missing).

Proposed fix
Remove the inherited rd.lvm.lv args from the upgrade boot entry in regular mode as well, exactly as already done for live mode, so lvm_scan falls through to vgchange -ay and activates all VGs. Verified on the system above (and additionally on AlmaLinux 9.8 -> 10.2 with the same layout): with the args removed, the upgrade boots straight through.

leapp-logs.tar.gz