Skip to content

feat: enable CUDA backend support on Linux#200

Open
DjagbleyEmmanuel wants to merge 3 commits into
AtomicBot-ai:mainfrom
DjagbleyEmmanuel:feat/cuda-linux-support
Open

feat: enable CUDA backend support on Linux#200
DjagbleyEmmanuel wants to merge 3 commits into
AtomicBot-ai:mainfrom
DjagbleyEmmanuel:feat/cuda-linux-support

Conversation

@DjagbleyEmmanuel

Copy link
Copy Markdown

This PR enables CUDA backend support on Linux by preserving user-imported CUDA builds instead of silently collapsing them to CPU. Previously, any CUDA archive imported via "Install Backend from File" on Linux would be downgraded to the CPU backend. Now CUDA backends remain selectable and runnable when the host driver supports them.

What changed

Rust plugin (tauri-plugin-llamacpp-upstream/src/backend.rs)

  • map_old_backend_to_new now passes through ubuntu-cuda-* names as linux-cuda-* instead of collapsing them to CPU. User-imported CUDA builds (linux-cuda-*, cu12, cu13, cu11) are preserved as-is.
  • determine_supported_backends adds linux-cuda-12-x64 and linux-cuda-13-x64 to the Linux matrix when the host driver supports CUDA 12 or 13. Previously only CPU and Vulkan were listed for Linux.
  • get_backend_category gained generic cuda-13/cuda-12/cuda-11 family matching so locally imported Linux CUDA backends are correctly categorized for priority ranking instead of falling through to a plain "x64" bucket.
  • Updated tests to expect CUDA backends in the Linux supported matrix.

TypeScript extension (backend.ts, index.ts)

  • backendTypeEquivalents now generates generic linux-* to ubuntu-* aliases (and vice versa) so custom CUDA archives installed from file are discoverable regardless of prefix.
  • installBackend preserves ubuntu-cuda-* archive names as linux-cuda-* on Linux instead of collapsing them to CPU.

Why

ggml-org does not publish prebuilt CUDA-Linux binaries. The only way to get CUDA on Linux is to compile llama.cpp from source and import the archive manually. Before this change that workflow was broken: the imported backend would silently become a CPU backend. Now it stays as CUDA and shows up in the backend list.

I tested this on a GTX 1050 with driver 535.309.01 (supports CUDA 12). Users with similar hardware can now use their GPU for inference on Linux.

Testing

  • All 8 determine_supported_backends Rust tests pass
  • Manual verification: CUDA backend appears in the backend list and can load a model
  • The ubuntu-cuda-12.0-x64 archive name correctly maps to linux-cuda-12.0-x64

- Preserve user-imported ubuntu-cuda-* archives as linux-cuda-* in map_old_backend_to_new, installBackend, and backendTypeEquivalents
- Add linux-cuda-12-x64 and linux-cuda-13-x64 to the Linux supported backends matrix
- Add generic cuda-12/13/11 family matching in get_backend_category
- Update tests to expect CUDA in the Linux backend matrix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant