nv-vgpu-driver-hybridizer

Hybridize NVIDIA vGPU Host and Guest drivers to retain the Host's CUDA capability.

Because NVIDIA's official vGPU KVM Host driver do not include CUDA and its required binary libraries, the Host cannot use CUDA — CUDA is only available inside VMs with the Guest driver installed.

In special cases, to retain CUDA capability on both the Host and the Guest, the two drivers can be "hybridized". Starting from the Guest driver, add, replace, or patch the necessary source code, scripts, and binary library files from the original Host driver to turn it into a Host driver that provides CUDA capability.

Note 1: In newer drivers the final (patch) version numbers may differ between Host and Guest, which can cause conflicts in some modules and driver versions. We need to patch the version strings embedded in the binaries of those modules so they match the driver version.

Note 2: Only perform hybridization on Host and Guest drivers of the same release; hybridizing mismatched versions may lead to unknown consequences. For example:

vGPU Version	Host Driver Version	Guest Driver Version	Is Verified
19.0	580.65.05	580.65.06	✅️
19.1	580.82.02	580.82.07	❓
19.2	580.95.02	580.95.05	❓

Usage

1. Download the NVIDIA vGPU Host and Guest drivers.

2. Unpack both drivers with command-line argument -x.

3. Run this scripts to hybridize the drivers:

   uv sync
   uv run main.py --host /path/to/host-driver --guest /path/to/guest-driver -o /path/to/output

4. cd to /path/to/output, run ./nvidia-installer -m kernel to install the hybridized host driver.

5. Run nvidia-smi to verify the hybridization. Driver Version should be the same as the Guest driver version, and CUDA Version should NOT be N/A.

   $ nvidia-smi
   Thu Oct 30 09:53:32 2025
   +-----------------------------------------------------------------------------------------+
   | NVIDIA-SMI 580.65.06              Driver Version: 580.65.06      CUDA Version: 13.0     |
   +-----------------------------------------+------------------------+----------------------+
   | GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
   | Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
   |                                         |                        |               MIG M. |
   |=========================================+========================+======================|
   |   0  NVIDIA ...                     Off |   00000000:00:08.0 Off |                    0 |
   |  0%   ..C    P0             ... /  .... |   .....MiB /  .....MiB |      0%      Default |
   |                                         |                        |                  N/A |
   +-----------------------------------------+------------------------+----------------------+

6. Configure vGPU license according to NVIDIA official document.

7. Reboot.

8. Run nvidia-smi -q to verify the status of GPU Virtualization Mode and vGPU Software Licensed Product. e.g.:

   $ nvidia-smi -q
   ...
   GPU Virtualization Mode
       Virtualization Mode               : Host VGPU
       Host VGPU Mode                    : SR-IOV
       vGPU Heterogeneous Mode           : Disabled
   vGPU Software Licensed Product
       Product Name                      : NVIDIA RTX Virtual Workstation
       License Status                    : Licensed
   ...

Troubleshooting

Cannot create vGPU

1. Make sure the host GPU is running in proper display mode. Some GPUs may not support vGPU in graphic mode.

./displaymodeselector --listgpumodes
./displaymodeselector --gpumode compute

2. When using hybridized driver, some processes (like nvidia-persistenced) may occupies the GPU and cause the vGPU creation failed to acquire unbindLock. Try to following the steps:

   1. Use lsof /dev/nvidia* to find processes that are occupying the GPU.
   2. Terminate those processes.
   3. Run cd /proc/driver/nvidia/gpus/0000\:xx\:00.0 — note that xx should be replaced with the actual PCI address.
   4. Run echo 1 > unbindLock to write to unbindLock.
   5. Run cat unbindLock to confirm the value is 1.
   6. Repeat the above steps for GPUs at other PCI addresses to unlock unbindLock.

   References: Preparing a GPU Configured for vGPU for Use in Pass-Through Mode

3. We only tested vGPU creation via Vendor-Specific VFIO Framework by using hybridized driver on Ubuntu 24.04. See: Creating an NVIDIA vGPU on a Linux with KVM Hypervisor that Uses a Vendor-Specific VFIO Framework.

License

This project is licensed under the MIT License - see the LICENSE file for details.