perf: speed up CUDA virtual package detection

[baszalmstra]

Description

Detecting __cuda and __cuda_arch takes about 1.5 seconds on Windows when the GPU has been idle, which is the normal case for a command line tool. Measured with CUDA 13.0 and compute capability 8.9, each in a fresh process:

driver state	detection
idle, or first use after boot	1.5 s
used seconds earlier	460 ms
through the cache	1.0 ms

Nearly all of it is one step. Loading NVML takes 0.5 ms and enumerating devices adds 24 ms. The rest is the driver version query, which loads the CUDA driver library and starts the user-mode driver to read the version out of it. If the driver powered down in the meantime it has to come back up first, and that is where the 1.5 seconds goes. No detection API avoids this, so the only fix is to not do it again.

So this caches the result on disk. A cache hit never touches the driver, which also leaves an idle GPU asleep instead of waking it on every invocation.

The cache is keyed on the boot session, a fingerprint of the installed driver and one of the visible GPUs, so reboots, driver updates and plugging in an eGPU all invalidate it. A TTL covers whatever the fingerprints miss. Callers pass the cache directory through a new cache_dir argument, and None disables it. That is a breaking change to the detection API. The Python bindings take the same argument.

It also fixes three things in the detection itself:

• Detection uses NVML instead of libcuda, so CUDA_VISIBLE_DEVICES no longer affects the result. Under a job scheduler or in CI, where that variable is often set, __cuda and __cuda_arch could come out wrong or disappear entirely.
• __cuda_arch now works on musl, where it was always absent.
• __cuda is detected even when the driver fails to initialize, since the version query no longer needs initialization.

Fixes prefix-dev/pixi#6775

How Has This Been Tested?

• Unit tests for the cache roundtrip and for invalidation on reboot, driver change and TTL, alongside the existing detection tests.
• Benchmarks per detection path, as ignored tests. The numbers above come from running them on Windows with an NVIDIA GPU.
• cargo clippy and cargo fmt pass, the crate builds for x86_64-unknown-linux-musl and both Windows targets, and cargo check passes for py-rattler.
• Checked the nvidia-smi fallback against NVML on hardware: both report CUDA 13.0 and compute capability 8.9.

AI Disclosure

• This PR contains AI-generated content.
  • I have tested any AI-generated content in my PR.
  • I take responsibility for any AI-generated content in my PR.

Tools: Claude Code

Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added sufficient tests to cover my changes.

[wolfv]

CUDA detection logic looks good to me overall 👍

I can’t fully validate the DLL/loading behavior across all platforms, but the general approach makes sense. The boot cache also looks great, especially for Windows startup time.

One small thing I was wondering about: for the fallback paths, can we make sure we consistently try the nvidia-smi-based detection where possible? From the discussion it sounds like the current MUSL/fallback behavior is similar to what cuda-oxide does, which seems reasonable.

I also briefly wondered whether we could extract some constants directly from the driver with something like goblin, even on MUSL systems, but for __cuda_arch we need to load/query the CUDA side anyway, so that probably doesn’t buy us much here.

So from my side: looks good, modulo the fallback consistency check.

[baszalmstra]

Could you maybe recheck, I added a lot of robustness checks

[hunger]

On Linux you will get problems with this:

Containers share a kernel and with that their boot id. You might want to hash that together with the machine ID (if available). At least on systemd-based systems you should be able to tell different containers apart from each other.

[hunger]

Actually: Thinking about that, the graphics card is managed by the kernel, which is the same for all containers anyway. So this should be fine (together with the device-node check for the graphics cards).

[github-actions]

cargo-semver-checks detected API breaking changes compared with the pull request's base revision.

Details

--- failure function_parameter_count_changed: pub fn parameter count changed ---

Description:
A publicly-visible function now takes a different number of parameters.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#fn-change-arity
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.48.0/src/lints/function_parameter_count_changed.ron

Failed in:
  rattler_virtual_packages::cuda::cuda_arch now takes 1 parameters instead of 0, in /home/runner/work/rattler/rattler/crates/rattler_virtual_packages/src/cuda.rs:293
  rattler_virtual_packages::cuda::cuda_info now takes 1 parameters instead of 0, in /home/runner/work/rattler/rattler/crates/rattler_virtual_packages/src/cuda.rs:188
  rattler_virtual_packages::cuda::cuda_version now takes 1 parameters instead of 0, in /home/runner/work/rattler/rattler/crates/rattler_virtual_packages/src/cuda.rs:277

--- failure method_parameter_count_changed: pub method parameter count changed ---

Description:
A publicly-visible method now takes a different number of parameters, not counting the receiver (self) parameter.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#fn-change-arity
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.48.0/src/lints/method_parameter_count_changed.ron

Failed in:
  rattler_virtual_packages::CudaArch::current takes 0 parameters in /tmp/tmp.t24Ez3ORx3/baseline/crates/rattler_virtual_packages/src/lib.rs:780, but now takes 1 parameters in /home/runner/work/rattler/rattler/crates/rattler_virtual_packages/src/lib.rs:849
  rattler_virtual_packages::VirtualPackages::detect takes 1 parameters in /tmp/tmp.t24Ez3ORx3/baseline/crates/rattler_virtual_packages/src/lib.rs:277, but now takes 2 parameters in /home/runner/work/rattler/rattler/crates/rattler_virtual_packages/src/lib.rs:308
  rattler_virtual_packages::VirtualPackages::detect_for_platform takes 2 parameters in /tmp/tmp.t24Ez3ORx3/baseline/crates/rattler_virtual_packages/src/lib.rs:322, but now takes 3 parameters in /home/runner/work/rattler/rattler/crates/rattler_virtual_packages/src/lib.rs:370
  rattler_virtual_packages::Cuda::current takes 0 parameters in /tmp/tmp.t24Ez3ORx3/baseline/crates/rattler_virtual_packages/src/lib.rs:709, but now takes 1 parameters in /home/runner/work/rattler/rattler/crates/rattler_virtual_packages/src/lib.rs:763
  rattler_virtual_packages::VirtualPackage::detect takes 1 parameters in /tmp/tmp.t24Ez3ORx3/baseline/crates/rattler_virtual_packages/src/lib.rs:494, but now takes 2 parameters in /home/runner/work/rattler/rattler/crates/rattler_virtual_packages/src/lib.rs:545
    Building rattler_virtual_packages v4.1.0 (current)
       Built [  22.864s] (current)
     Parsing rattler_virtual_packages v4.1.0 (current)
      Parsed [   0.024s] (current)
    Building rattler_virtual_packages v4.1.0 (baseline)
       Built [  22.619s] (baseline)
     Parsing rattler_virtual_packages v4.1.0 (baseline)
      Parsed [   0.020s] (baseline)
    Checking rattler_virtual_packages v4.1.0 -> v4.1.0 (assume minor change)
     Checked [   0.150s] 196 checks: 194 pass, 2 fail, 0 warn, 57 skip

     Summary semver requires new major version: 2 major and 0 minor checks failed
    Finished [  47.619s] rattler_virtual_packages

[wolfv]

Tested on Fedora with CUDA GPU: works both with MUSL and regular glibc build! Code looks good, too.