Warm migration wedges NBD device: Proxmox host auto-activates the guest LVM VG (teardown does not deactivate holders)

[adminsyspro]

Summary

During a warm migration (VDDK/CBT) of a Linux guest that uses LVM, the target Proxmox node auto-activates the guest's LVM volume group on the attached NBD device. The activated logical volumes pin /dev/nbdN, so the reader teardown (nbd-client -d) can no longer release it. The device ends up wedged: /sys/block/nbdN/pid keeps a stale pid whose process is already dead, nbd-client -d returns 0 without freeing it, and rmmod nbd refuses ("Module nbd is in use"). Only a reboot reclaims the slot.

Root cause

1. The nbd kernel module is loaded with max_part > 0 on the node (the cold UEFI-injection path loads it with max_part=16 in frontend/src/lib/migration/pipeline.ts, and the module param is fixed at load time). The warm path's modprobe nbd max_part=0 is then a no-op.
2. With partition scanning on, attaching the guest disk creates nbdNp1/p2/p3, udev fires pvscan --cache -aay, and LVM auto-activates the guest VG (e.g. rootvg) found on a partition.
3. The active device-mapper LVs hold /dev/nbdN open. Nothing in the warm teardown deactivates them, so nbd-client -d cannot release the device.

The reader runs nbdkit read-only (-r), so the source is never at risk; the problem is purely a wedged host device.

Evidence

Reported on #521: lsblk /dev/nbd0 on the target node showed rootvg-{root,var,home,swap,tmp,opt,usr} (dm 252:x) active on nbd0p3, ps/kill confirmed the pid in /sys/block/nbd0/pid was dead, lsof was empty, and rmmod nbd reported "in use".

Current state

PR #534 (dynamic free-NBD-device allocation) mitigates this: it skips any device with a live/stale pid file (so a wedged device no longer blocks a migration) and never nbd-client -d a device it did not attach. But it does not prevent the wedging: each warm migration of a Linux/LVM guest wedges another /dev/nbdN until the node is rebooted, and re-migrating the same guest triggers LVM "duplicate PV" noise (same VG UUID on two nbd partitions).

Proposed fix (needs care)

In the warm reader teardown (buildReaderTeardownCmd / stopVddkReader, frontend/src/lib/migration/warm/vddk-reader.ts), before nbd-client -d, deactivate the holders that sit only on the device we own, then disconnect. This MUST be scoped strictly to /dev/nbdN:

• Data-safety hazard: a mis-scoped vgchange -an could deactivate a host VG and take the node down. The guest VG name can even collide with a host VG (e.g. both pve), so selection must be by PV path on our device, never by VG name alone.
• Likely shape: find the dm holders under /sys/block/nbdN/, deactivate the VG(s) whose PVs are exactly on /dev/nbdN* (e.g. vgchange -an --select "pv_name=~^/dev/nbdN"), then nbd-client -d.
• We cannot force max_part=0 (module param is global; the cold path needs 16), so the fix must assume partitions may exist.

Interim node-side workaround (documented, not code)

On the target Proxmox node, make LVM ignore NBD devices so it stops auto-activating guest VGs. In /etc/lvm/lvm.conf, devices { }:

global_filter = [ "r|/dev/nbd.*|", "a|.*|" ]

then reboot (the reboot also clears any already-wedged device). This is safe on Proxmox: its own LVM/LVM-thin storage lives on real disks, never on /dev/nbd*, and warm migration reads the raw device via dd, not via LVM.

Testing

Needs a Linux guest with an LVM root as a warm-migration reproducer.

Refs #521, PR #534.