Vulkan backend fails with "Unable to find group render" — no device group_add for non-CUDA/ROCm/Intel images

[gilvandev]

Summary

When manually overriding LLAMA_IMAGE to point at a generic llama.cpp:server-vulkan image on an AMD host, model load fails with:
Internal Server Error ("Unable to find group render: no matching entries in group file")

Switching the same instance back to the ROCm image (server-rocm) loads the model successfully with no changes to host permissions.

Environment

• Host: TrueNAS SCALE (Docker, not k3s)
• GPU: 2x AMD RX 9060 XT (RDNA4 / gfx1200), ROCm-capable
• Host GIDs: render = 107, video = 44 (confirmed via getent group)
• /dev/dri/renderD128 and renderD129 owned by render group on host
• llamaMan spawns llama-server as a sibling container over the Docker socket, as documented

What I believe is happening

Per the README, GPU vendor auto-detection (GPU_TYPE) only recognizes cuda, rocm, and intel. There's no first-class vulkan vendor path. On my AMD host, the ROCm device-mapping path is used (attaches /dev/kfd + /dev/dri), but does not group_add the container to video/render — presumably because that's not required for the ROCm runtime itself.

However, the generic Vulkan/RADV image needs the container process to be a member of the render (and typically video) group to open the render node, per llama.cpp's own Docker guidance for AMD/RADV. Since LLAMA_IMAGE can be overridden independently of GPU_TYPE, selecting a Vulkan image while GPU_TYPE resolves to rocm results in a container with the device node mounted but no matching group membership, hence the render group lookup failure inside the container.

I found that manually forcing GPU_TYPE=intel (which does document requiring /dev/dri + video/render group access) works around the issue, since that appears to be the only vendor path that performs group_add. This isn't a real fix though — it repurposes a vendor path with unrelated semantics (e.g. GPU Devices selection is documented as unsupported under Intel Arc) just to get the group mapping.

Reproduction

1. AMD ROCm host, llamaMan configured normally with GPU_TYPE auto-detected as rocm.
2. Override LLAMA_IMAGE to a llama.cpp:server-vulkan-style image.
3. Launch/load any model.
4. Observe: Internal Server Error ("Unable to find group render: no matching entries in group file").
5. Revert LLAMA_IMAGE to server-rocm (leave GPU_TYPE untouched) → loads fine.
6. Workaround: set GPU_TYPE=intel explicitly while keeping the Vulkan LLAMA_IMAGE → loads fine.

Request

Could Vulkan be supported as a proper first-class vendor path (e.g. GPU_TYPE=vulkan) that attaches /dev/dri and performs group_add for video/render (mirroring what's already done for Intel Arc), independent of whether the host GPU is AMD, Intel, or otherwise Vulkan-capable? This would let users on ROCm-unstable setups (e.g. current RDNA4/gfx1200 driver issues) fall back to Vulkan without hijacking the Intel Arc detection path.

[nullata]

Hi, thanks for your report. I'll take a look at this issue in depth in the upcoming days. Please keep in mind that this would be a best effort approach as I do not own AMD hardware and reproducing it might not be straight forward. If you are eager to get this working asap, feel free to open PR with your proposed changes. Thanks for your feedback!

[gilvandev]

No problem. Unfortunately, I won't be able to create a PR because my environment is a bit limited too. But I can run tests with the changes you make and help you with logs as needed.

[nullata]

Fix is in main:

• Added GPU_TYPE=vulkan as a first-class opt-in vendor path. It attaches /dev/dri (no /dev/kfd) and defaults LLAMA_IMAGE to ghcr.io/ggml-org/llama.cpp:server-vulkan. Auto-detect is unchanged (an AMD host still resolves to rocm) because from a PCI vendor ID alone we can't tell whether you want Vulkan over ROCm.
• Fixed the underlying group_add bug that produced the Unable to find group render error. llamaMan now resolves the render/video GIDs numerically by stat'ing /dev/dri/renderD* and card* on the host and passes those numbers to group_add, so it works against any image regardless of whether it happens to define those group names in its /etc/group. The fix applies to the rocm and intel branches too, not just the new vulkan one.

If llamaMan itself runs in a container, make sure /dev/dri is bind-mounted into it so the stat can see the host GIDs (107/44 in your case); otherwise it falls back to the old name-based path. A new nullata/llamaman:latest image with this change will be on Docker Hub near the end of this week. Would appreciate a report either way once you try it, since I don't have AMD hardware here.