Cuda driver improvements/fixes

[Zelberor]

Description

Makes sure that no cuda driver binaries are installed into the container
-> the binaries from the host have to be used.

Might fix some driver incompatibility issues that started occurring in arlab.

Type of change

• Bug fix (non-breaking change which fixes an issue)

Does this PR introduce a breaking change?

No

Draft status

• Currently untested on the lab pcs.

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works (might be obsolete with CI later on)
• New and existing unit tests pass locally with my changes (might be obsolete with CI later on)

Summary by CodeRabbit

• Chores

  • Improved Docker build and runtime setup for CUDA/GPU-enabled containers by switching to a CUDA toolkit-based installation and adding required utility tooling.
  • Adjusted container setup to avoid driver conflicts and streamline GPU support.

• Bug Fixes

  • Fixed GPU capability declaration formatting and added an explicit bind mount for the GPU driver ICD to ensure consistent runtime behavior.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

Normalize GPU capability syntax in compose files, add a Vulkan ICD bind-mount, and switch Dockerfile GPU setup from Vulkan/driver installs to installing the CUDA toolkit via the cuda-keyring package; also add wget and curl to base images.

Changes

Cohort / File(s)	Summary
Docker Compose GPU config build/docker-compose.carla.cuda.yaml, build/docker-compose.dev.cuda.yml	Normalized capabilities: [ gpu ] formatting and added a volumes entry to bind-mount /usr/share/vulkan/icd.d/nvidia_icd.json under service deployment reservations with an explanatory comment.
Dockerfile CUDA toolkit install build/docker/agent-ros2/Dockerfile, build/docker/carla/Dockerfile	Added wget and curl; replaced prior Vulkan/driver install steps with CUDA toolkit installation using a downloaded cuda-keyring_*.deb, removed the temp deb, and install cuda-toolkit + libxext6 instead of libnvidia-gl-${NVIDIA_DRIVER_VERSION}/nvidia-cuda-toolkit; comments updated accordingly.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

• Pay attention to:
  • Correctness of the cuda-keyring filename and repository steps.
  • Runtime implications of replacing libnvidia-gl with cuda-toolkit + libxext6.
  • Whether the added Vulkan ICD bind-mount covers expected runtime layouts and edge cases.

Poem

🐰 I fetched a tiny keyring bright,
wget, curl through day and night.
CUDA settled, ICD in place,
containers hop with nimble grace.
🥕✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'Cuda driver improvements/fixes' is directly related to the main objective of the changeset, which focuses on CUDA driver handling and preventing driver binaries from being installed in containers.
Description check	✅ Passed	The description covers the core purpose, type of change, breaking change status, and includes a completed checklist. However, it lacks the 'Most important changes' section specifying which files reviewers should focus on.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch cuda-driver-improvements

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 604b500 and c6bc9bb.

📒 Files selected for processing (2)

• build/docker/agent-ros2/Dockerfile (2 hunks)
• build/docker/carla/Dockerfile (2 hunks)

🚧 Files skipped from review as they are similar to previous changes (2)

• build/docker/agent-ros2/Dockerfile
• build/docker/carla/Dockerfile

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: Build and push image

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

♻️ Duplicate comments (2)

build/docker-compose.dev.cuda.yml (1)

25-27: Vulkan ICD mount mirrors unresolved concern from related file.

This Vulkan ICD mount is identical to the one added in build/docker-compose.carla.cuda.yaml (lines 16–18). Please address the clarification needed there, as it applies here as well.

build/docker/agent-ros2/Dockerfile (1)

35-47: Address the same CUDA keyring hardcoding and error-handling concerns as carla/Dockerfile.

This block is identical to build/docker/carla/Dockerfile lines 28–40, which has unresolved concerns:

• Hardcoded ubuntu2404 in URL (line 41); if BASE_IMAGE changes, this breaks.
• Hardcoded keyring version 1.1-1 without version management.
• No wget error handling; silent failures may occur.
• libxext6 necessity unclear (see related comment in carla/Dockerfile review).

Apply the same solutions and error handling suggested for carla/Dockerfile to maintain consistency and robustness.

🧹 Nitpick comments (1)

build/docker-compose.dev.cuda.yml (1)

24-24: GPU capabilities formatting inconsistency risk.

This file applies the same [ gpu ] formatting as build/docker-compose.carla.cuda.yaml. If this is a linting requirement, ensure the change is also applied to any other Compose files with GPU definitions to maintain uniformity across the codebase.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between feb6359 and b176bd2.

📒 Files selected for processing (4)

• build/docker-compose.carla.cuda.yaml (1 hunks)
• build/docker-compose.dev.cuda.yml (1 hunks)
• build/docker/agent-ros2/Dockerfile (2 hunks)
• build/docker/carla/Dockerfile (2 hunks)

🔇 Additional comments (6)

build/docker-compose.carla.cuda.yaml (2)

14-14: Clarify the GPU capabilities formatting change.

The change from [gpu] to [ gpu ] is syntactically valid YAML in both forms. If this formatting is required for tooling compliance, please confirm the linting rule or tool version enforcing it; otherwise, it risks inconsistency if not applied uniformly across all Compose files.

---

16-18: The review mischaracterizes the technical relationship between Vulkan ICD and CUDA driver isolation.

The claim that mounting /usr/share/vulkan/icd.d/nvidia_icd.json "directly contradicts the isolation goal" is technically incorrect. The Vulkan ICD JSON is a configuration file that names the vendor library; the nvidia-container-toolkit mounts the host driver library so the ICD can point to a working vendor library from the host. CUDA binaries remain from the image and rely on host driver compatibility, while the toolkit ensures driver libraries are visible in the container.

This is the standard, supported design for exposing host NVIDIA hardware while keeping CUDA binaries inside the container. There is no contradiction—the Vulkan ICD mount is a legitimate part of the host driver injection strategy.

However, the code comment itself already notes "This should not be necessary" with a reference to the toolkit's GitHub issue, suggesting the modern toolkit may auto-inject this. That's a separate, valid question about redundancy, but it's not an isolation violation.

Likely an incorrect or invalid review comment.

build/docker/carla/Dockerfile (2)

13-13: ✓ LGTM: wget and curl addition supports CUDA keyring download.

The tools are appropriately added to support the external CUDA keyring installation that follows.

---

40-40: Clarify the necessity and scope of libxext6 installation.

Line 40 introduces libxext6 (X11 client library), which appears to be a new dependency for the CUDA toolkit setup. Confirm whether this is:

1. A required dependency of cuda-toolkit that's being explicitly declared for clarity.
2. A workaround for a missing dependency in the CUDA tooling.
3. Intended for GPU graphics rendering support.

Additionally, verify that this library is only installed for the cuda flavour and not unintentionally pulled into the gpu or rocm builds.

build/docker/agent-ros2/Dockerfile (2)

17-17: ✓ LGTM: wget and curl addition mirrors carla/Dockerfile.

The tools are appropriately added to support the external CUDA keyring installation.

---

33-33: Verify NVIDIA_REQUIRE_DRIVER constraint against carla/Dockerfile.

This file specifies ENV NVIDIA_REQUIRE_DRIVER="cuda>=12.0 driver>=${NVIDIA_DRIVER_VERSION}" (line 33), whereas carla/Dockerfile specifies only ENV NVIDIA_REQUIRE_DRIVER="driver>=${NVIDIA_DRIVER_VERSION}" (line 26).

Verify that the cuda>=12.0 constraint is intentional and aligns with the cuda-toolkit package selected. Ensure consistency or document the difference.

[ll7]

@samuel-ibele could you quickly describe what you tested in a comment here? Thank you.

[samuel-ibele]

Tested the CUDA driver fixes on several computers with working and non-working setups.
Testing was successful and even the setups that previously did not work now start without errors.