virtio-gpu: KMS + virgl 3D driver for QEMU/UTM/libvirt guests - #499
virtio-gpu: KMS + virgl 3D driver for QEMU/UTM/libvirt guests#499borovikovd wants to merge 12 commits into
Conversation
a22175c to
419b934
Compare
756adf1 to
fbf4b08
Compare
Build drm_gem_shmem_helper.c, drm_fbdev_shmem.c and
drm_format_helper.c into drm.ko, for KMS drivers whose objects live
in anonymous memory rather than device VRAM (virtio-gpu; bochs/qxl
would use the same stack), and import drm_simple_kms_helper.c
(GPL-2.0-or-later) for the virtio module to build.
drm_gem_shmem_helper.{c,h} is the Linux 6.12 file under its
GPL-2.0-only license, following existing practice - drm.ko already
builds GPL-2.0 drm_writeback.c and drm_panel_backlight_quirks.c, and
dmabuf.ko builds GPL-2.0-only sync_file.c. The FreeBSD diff is 47
lines under #ifdef: gfp masks, VM-object locking around the fault
handler's pfn insert (as ttm_bo_vm does), and elision of the
device-level DMA mapping, which virtio-gpu does not use (the device
addresses guest memory physically).
drm_gem.c grows FreeBSD drm_gem_get_pages()/drm_gem_put_pages(): the
Linux 6.x originals walk folio batches and the unevictable LRU, which
LinuxKPI does not model, so these are the pre-folio page-at-a-time
shape over shmem_read_mapping_page(). drm_fbdev_shmem.c compiles
with its deferred-I/O paths under #ifdef __linux__ (LinuxKPI has no
fb_deferred_io; the helper works without it). LinuxKPI additions:
dma_fence_match_context() and the trace_dma_fence_emit() stub, both
used by virtio-gpu's fence code.
drivers/gpu/drm/virtio/ and include/uapi/linux/virtio_gpu.h from Linux v6.12, byte-identical, including files the FreeBSD build will not compile (Kconfig, the Linux Makefile, virtgpu_vram.c). No FreeBSD content in this commit, so future re-syncs against newer Linux are a clean re-import; all local changes come as separate commits on top.
The complete diff against Linux v6.12: two files, +36 lines, zero deletions, every line inside #ifdef __FreeBSD__ (or the __linux__ side of an #elif that preserves the original). Compiled with __linux__ these files are byte-equivalent to upstream. virtgpu_drv.c: a hw.virtio_gpu sysctl node for LinuxKPI module parameters, and a virtio_gpu_freebsd_driver() accessor so the newbus glue can reach probe/config_changed without the Linux virtio bus (virtio_gpu_driver is static, and module_virtio_driver() expands to nothing useful here). virtgpu_object.c: MODULE_PARM_DESC for virglhack (LinuxKPI's module_param_named() requires the description symbol), and a page-aligned allocation for the ents array - vmalloc_to_sgt() in virtgpu_vq.c requires it, and LinuxKPI's kvmalloc is malloc(9), which only guarantees page alignment for page-multiple sizes.
Base LinuxKPI has no virtio compat, and both kernels define struct
virtqueue, so the port is two TUs that never share virtio types:
virtgpu_freebsd.c - a native newbus virtio child (BUS_PROBE_VENDOR,
outranking vtgpu(4)) that drives the virtio 1.x status ladder for
kldload-time attach, allocates the control/cursor queues, and hands
off to the upstream driver's probe. Two workarounds, both documented
in README.FreeBSD.md: the notify window is mapped with pmap_mapdev()
because stores through vtpci's notify sub-map fault on QEMU/hvf
aarch64, and each virtqueue gets an explicit virtqueue_enable_intr()
because FreeBSD virtqueues are born with interrupts disabled while
Linux vrings start enabled and the upstream driver relies on that.
The driver is purely interrupt-driven (shared INTx on this platform).
VIRTIO_RING_F_INDIRECT_DESC is negotiated, as Linux always does:
without it, ATTACH_BACKING of a large framebuffer (one segment per
ents page) can demand more ring slots than exist and the upstream
ring-space precheck hangs. Detach and kldunload are refused while
the DRM device is registered (drm-kmod devfs teardown panics
otherwise); diagnostics via dev.virtio_gpu_drm.0.{reaps,queue_state}.
linux_virtio.c - the Linux virtio API over virtqueue(9): sg-chain
flattening that never merges across the readable/writable boundary
(upstream places inline responses right after the command),
virtqueue_add_sgs/get_buf/kick, errno translation, and the interrupt
trampoline. freebsd/ holds the shim headers and the cross-TU
contract (struct fbsd_vq opacity).
virtio/Makefile builds virtio_gpu_drm.ko (all supported
architectures); the 2D KMS configuration only, no virgl. All
imported files build under their upstream licenses, including
GPL-2.0-only virtgpu_vram.c, whose RESOURCE_BLOB/HOST_VISIBLE
features are rejected at negotiation.
drivers/gpu/drm/virtio/README.FreeBSD.md - what the port is (2D KMS, no virgl), how to build and load it, the wlroots/sway recipe (WLR_RENDERER=pixman), UTM notes including HiDPI, platform quirks with their root causes, and diagnostics.
Generated headers, interface-directory symlinks (machine/i386/x86), export_syms and .kld intermediates slipped into the tree; all of them are build products that bsd.kmod.mk (re)creates and .gitignore already covers. The committed machine symlink pointed at arm64 includes and silently broke amd64 builds from a fresh checkout - found in amd64 review testing on PR freebsd#499.
Warn against loading via loader.conf _load entries (DRM core is not initialized at loader time; probe fails with -19 - use kld_list), and document -vga virtio for plain QEMU, where a bare -device virtio-gpu-pci leaves QEMU's default std-VGA in place and X picks the wrong card as primary. Both found in amd64 testing on PR freebsd#499.
Request and propagate VIRTIO_GPU_F_VIRGL. The imported driver is already fully wired for classic virgl (EXECBUFFER, capsets, 3D transfers, per-file contexts, render node); its sync_file and dma_fence_unwrap dependencies resolve against the bundled dmabuf module, so unlocking 3D is feature negotiation only. Blob resources and host-visible memory stay off (virtio_get_shm_region() reports absence), and on hosts without virgl the bit does not negotiate and behavior is unchanged - the same binary serves both. With a virtio-gpu-gl-pci host (UTM GPU acceleration, virglrenderer over ANGLE/Metal on Apple silicon): Mesa 26 virgl probes clean (eglinfo: "virgl (ANGLE (Apple, Apple M4 Pro, OpenGL 4.1 Metal))"), kmscube renders 280-630 fps, and sway runs on its default GLES2 renderer - no WLR_RENDERER=pixman needed. Verified on stock and WITNESS+INVARIANTS kernels: full 2D ladder still green with 3D active (kmstest, 6K/8K dumb-buffer stress, grim), no witness or assertion output. The GL device exposes a 256-slot control ring; the existing MAX(ring, VIRTGPU_MAX_INDIRECT) sglist sizing covers it. amd64 cross-compile clean.
Document the classic-virgl 3D path: host device requirements (UTM GPU-accelerated display / virtio-gpu-gl-pci), Mesa virgl on the guest, sway on the default GLES2 renderer, and the clean 2D fallback on hosts without virgl. Scope line updated; WLR_RENDERER=pixman is now documented as needed only on non-GL host devices.
The initial import was Linux v6.12.0; the tree's DRM core is 6.12.85.
The only virtio delta in the stable series up to that tag is the
per-plane-state fence fix ("drm/virtio: New fence for every plane
update"): the pageflip fence moves from the reusable framebuffer
object to duplicated plane state, fixing fence lifetime when the same
FB is flipped repeatedly. virtgpu_plane.c and virtgpu_drv.h are
byte-identical to v6.12.85; the two files carrying FreeBSD hunks
(virtgpu_drv.c, virtgpu_object.c) are unchanged upstream between
v6.12.0 and v6.12.85.
Add drivers/gpu/drm/virtio/ and include/uapi/linux/virtio_gpu.h to the format-patch path list so future Linux stable syncs cover the driver the same way as amd/i915/radeon. The FreeBSD-only files (freebsd/, virtgpu_freebsd.c, linux_virtio.c, README.FreeBSD.md) do not exist in Linux, so generated patches never touch them, and the FreeBSD hunks in virtgpu_drv.c/virtgpu_object.c are additive #ifdef blocks that git am --3way merges around.
Verified: with the three modules in /boot/modules, kld_list="virtio_gpu_drm" is enough - dmabuf.ko and drm.ko resolve via MODULE_DEPEND. This is the config a drm-kmod package install would use; absolute paths remain for out-of-module-path setups.
|
Tested this on amd64 under QEMU/KVM (the PR's existing testing was aarch64/UTM/QEMU-hvf on Apple Silicon) — wanted to close that gap. Up front: this confirms Xorg/X11 specifically. I could not get any Wayland compositor working on this driver — not a Sway-specific issue, every Wayland path I tried hit the same wall (details below). Summary follows; full writeup and a reproduction script available if useful. Bug found and fixed:
|
|
Good catch @ut316ab on the symlink - it was actually worse: 26 build artifacts (generated headers, machine/i386/x86 symlinks, .kld files) got committed by a stray git add -A on my side. Fixed and verified with a fresh clone on aarch64 + amd64 cross build. Thanks for the X11 confirmation. Added the -vga virtio note to the README - UTM sets up the device itself so I never hit the std-VGA stacking. On Wayland: your trace is right, but that wall is exactly why the README says to run sway with WLR_RENDERER=pixman (and LIBSEAT_BACKEND=seatd). Pixman skips Mesa/EGL/GBM entirely and allocates dumb buffers on the master fd, so the render-node CREATE_DUMB restriction never comes into play. That's how sway runs on my two aarch64 VMs daily. Worth a retry on your setup - I'd expect it to just work. Agreed that a real accelerated path means VIRTGPU_BLOB and is out of scope here. loader.conf: correct, DRM core isn't up at loader time. README now warns to use kld_list (same commit). Yes to the reproduction package, the QEMU launch script especially. UPDATE: the accelerated path turned out to be 8 lines away - classic virgl is now in the PR (last two commits; the imported driver's 3D code was already compiled in). With a GL host device sway runs its default GLES2 renderer, no pixman needed. Verified on aarch64/UTM: kmscube ~600 fps, glmark2-es2 score 689, WITNESS soak clean. On your setup try -device virtio-gpu-gl-pci. The render-node wall now only applies to non-GL devices. |
borovikovd
changed the title
2 participants
FreeBSD guests with a
virtio-gpu-pcidisplay device have no DRM/KMS path, which rules out wlroots/sway and Wayland generally. This ports Linux'sdrivers/gpu/drm/virtio(v6.12, matching this tree): KMS, dumb buffers, fbdev console — plus classic virgl 3D when the host device offers it (GPU-accelerated GLES via Mesa virgl; sway then runs its default GLES2 renderer with no env overrides). No blob resources / venus.Working on FreeBSD 15.1 aarch64 under UTM/QEMU-hvf, and confirmed on amd64 under QEMU-KVM (Xorg/XFCE) in review testing:
/dev/dri/card0, vt(4) console viavt_drmfb, sway + foot + grim (WLR_RENDERER=pixmanon non-GL hosts, default GLES2 renderer on virgl hosts), persistent across reboots. Setup, architecture and platform quirks indrivers/gpu/drm/virtio/README.FreeBSD.md.Commits (review fixes are appended, not squashed):
#ifdef __FreeBSD__Licensing: all imported files keep their upstream licenses, nothing is relicensed. That includes GPL-2.0-only
drm_gem_shmem_helper.candvirtgpu_vram.c— following existing practice, since drm.ko already builds GPL-2.0drm_writeback.canddrm_panel_backlight_quirks.c, dmabuf.ko builds GPL-2.0-onlysync_file.c, and the kmod ports declareBSD2CLAUSE MIT GPLv2. The FreeBSD-specific glue and shim are new code under MIT.Prior art: #119, #62, #356. reviews.freebsd.org/D55012 is an independent virtio-gpu effort for the base system — complementary (this provides the DRM/KMS path), happy to coordinate.
Disclaimer: this code was developed with heavy AI assistance. Knowing the failure modes of AI-generated code, it went through adversarial review and testing rather than just "it works":
Root causes for the platform workarounds (notify-window fault, interrupt arming, indirect descriptors) are documented in the commit messages and README.