Repository navigation
devspace: sealed e2e VM aborts QEMU on AMD EPYC 7763 hosts (SMM assertion) #235
Description
Found while retesting cgwalters-forge/rhel-bootc-examples#4 (gist https://gist.github.com/cgwalters-bot/6877dd16797e13e25c88cb85c0b2ebc0, rbe4-base-qemu-smm-assert.log).
On devspace cgwalters-devspace-36603350717 (AMD EPYC 7763, qemu-kvm-10.1.0-16.el10_2.5, edk2-ovmf-20251114-5.el10_2.2), just e2e sealed boots the image-builder disk with -machine q35,smm=on,accel=kvm -cpu host, cfi.pflash01 secure=on and OVMF_CODE.secboot.fd. QEMU aborts during firmware boot, twice in a row:
qemu-kvm: ../hw/core/cpu-system.c:90: int cpu_asidx_from_attrs(CPUState *, MemTxAttrs): Assertion `ret < cpu->num_ases && ret >= 0' failed.
The unsealed VM (non-secboot OVMF, same machine flags) boots fine on that host, and the same sealed test passed on an AMD EPYC 9V45 devspace (36613010646) and on the two earlier devspaces of 2026-09-29. So the sealed e2e result depends on which runner hardware a devspace lands on.
Next steps: check whether it reproduces with a minimal qemu-kvm -machine q35,smm=on -cpu host + secboot OVMF on that CPU model (then it's a qemu-kvm bug to report), and whether -cpu EPYC (or dropping -cpu host) avoids it; the e2e test could retry on a QEMU abort, or bot-devspace could say which CPU a devspace got.
Generated-by: https://github.com/cgwalters/#llms
Activity
cgwalters-bot commentedon Oct 2, 2026
It didn't reproduce on a second EPYC 7763. On devspace cgwalters-devspace-37023785367 (7763, same qemu-kvm-10.1.0-16.el10_2.5, edk2-ovmf-20251114-5.el10_2.2 and kernel 6.12.0-211.16.1.el10_2), the full
e2e sealedat rbe#4 5d680ce passed, and four more boots of its sealed disk with the e2e's QEMU flags were clean. A minimalqemu-kvm -machine q35,smm=on -cpu hostwith secboot OVMF (stock and test-enrolled vars) also ran clean to the BDS, and so did a CentOS Stream 10 guest doing 200 EFI variable writes, there and on a 9V45. So it's the particular host, not the CPU model. The planned-cpu/TCG/flag bisect needs a failing host, so I didn't do it.The original log shows two failures in a row. The real one is
KVM: entry failed, hardware error 0xffffffffat SMM entry (SMM=1 EIP=8000, after an SMI through port 0xB2): Hyper-V, under which devspaces run KVM nested, rejected the VMRUN. The assertion comes after that, while QEMU dumps the state. It's a known QEMU bug, fixed upstream in 0516f4b7 (v10.2.0, stable 10.0.5/10.1.1), and the RHEL 10.2 qemu-kvm doesn't have the backport. The entry failure itself looks like microsoft/WSL#41709: KVM keeps CR3[63:32] on SMM entry, and Hyper-V's nested VMRUN refuses that state. A KVM patch is on the list (https://lore.kernel.org/r/20260928-kvm-smm-cr3-upper-bits-v1-1-138f52dd531e@gmail.com). The dump'sCR3=0022a002doesn't rule that out, because QEMU 10.1 truncates CR3 to 32 bits outside long mode, and-m 4096on q35 puts half the guest RAM above 4 GiB. That's likely but not confirmed. Why only some hosts reject the state is unknown, probably their Hyper-V build.Findings, a draft bug report (not filed: a backport request for 0516f4b7 in RHEL qemu-kvm), and the reproducer scripts and logs: https://gist.github.com/cgwalters-bot/e12a58dc856bc87c951cda266510dfd2
Mitigation:
bot-devspacenow prints each devspace's CPU model and lists it, in cgwalters-bot/homegit#93. I didn't open an e2e PR, since nothing was verified on a failing host. The candidate is-m 2560inboot()of rbe'stests/composefs_e2e.py, which keeps all guest RAM below 4 GiB. The alternative is to capture QEMU's stderr and reportKVM: entry failedas a host problem.Generated-by: https://github.com/cgwalters/#llms
cgwalters-bot commentedon Oct 22, 2026
✅ dispatch ended well. Result: #235 (comment)
cgwalters-bot commentedon Oct 22, 2026
Triage (22 Oct 2026): this needs cgwalters to decide. The investigation is finished, and nothing left is a task an implement run can verify.
KVM: entry failed, hardware error 0xffffffffon SMM entry: Hyper-V refuses the nested VMRUN, which looks like Nested KVM on AMD: SMM entry fails (VMRUN invalid, 0xffffffff) for Windows 11 Secure Boot guests when guest CR3 is above 4GiB microsoft/WSL#41709 plus the KVM patch on the list. Thecpu_asidx_from_attrsassertion is a later QEMU bug, fixed upstream in 0516f4b7 (10.2.0 / 10.1.1) but not backported to RHEL 10.2 qemu-kvm. Neither can be fixed in our repositories.bot-devspacereport each devspace's CPU. homegit is now frozen (docs/homegit-freeze.md), so whether anything still runs that code is unclear.boot()intests/composefs_e2e.py(lines 258-267) still passes-machine q35,smm=on,accel=kvm -cpu host -m 4096.Decision. Pick one:
(a) Close as not planned. The bug is external and hasn't come back. Reopen if a failing host turns up.
(b) File the RHEL qemu-kvm backport request for 0516f4b7. The draft is in the gist. Filing it is an outward-facing report, so it needs your account and your OK.
(c) Change rbe#4's e2e to capture QEMU's stderr and report
KVM: entry failedas a host problem instead of a test failure. That needs no failing host to check. The other idea,-m 2560to keep guest RAM below 4 GiB, can't be verified without one.My recommendation: (a). Add (c) to rbe#4 only if that PR is going ahead, and do (b) only if you want the RHEL fix.
Generated-by: https://github.com/cgwalters/#llms