Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 28 additions & 26 deletions confidential-containers/confidential-containers-deploy.rst
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ After completing the installation, you can :doc:`Run a Sample Workload <run-samp
Label Nodes for Confidential Containers Components
**************************************************

The GPU Operator reads labels to determine what software components to deploy to a node.
The GPU Operator reads labels to determine what software components to deploy to a node.
To configure a node for Confidential Container workloads, you label the node with the ``nvidia.com/gpu.workload.config=vm-passthrough`` label.
Then, when the GPU Operator is installed in a subsequent step, it will deploy the software components needed to run Confidential Containers to the node.

Expand Down Expand Up @@ -144,24 +144,27 @@ Install the Kata Containers Helm Chart
Install Kata Containers using the ``kata-deploy`` Helm chart.
The ``kata-deploy`` chart installs all required components from the Kata Containers project including the Kata Containers runtime binary, runtime configuration, UVM kernel, and images that NVIDIA uses for Confidential Containers and native Kata containers.

The minimum required version is 3.29.0.
The minimum required version is ${kata_version}.

#. Set the chart version and registry path:

.. code-block:: console

$ export VERSION="3.29.0"
$ export VERSION="${kata_version}"
$ export CHART="oci://ghcr.io/kata-containers/kata-deploy-charts/kata-deploy"

#. Create a values file, such as ``kata-nvidia-gpu-values.yaml``, to configure the ``kata-deploy`` chart for NVIDIA Confidential Containers:

#. Install the kata-deploy Helm chart:
.. literalinclude:: ./samples/kata-nvidia-gpu-values.yaml
:language: yaml

#. Install the kata-deploy Helm chart with the values file:

.. code-block:: console

$ helm install kata-deploy "${CHART}" \
--namespace kata-system --create-namespace \
--set nfd.enabled=false \
--wait --timeout 10m \
-f kata-nvidia-gpu-values.yaml \
--version "${VERSION}"

*Example Output immediately after running the command:*
Expand All @@ -173,7 +176,7 @@ The minimum required version is 3.29.0.

The ``--wait`` flag in the install command instructs Helm to wait until the release is deployed before returning.
It can take a 2-3 minutes to return more output.

*Example Output when the release is deployed:*

.. code-block:: output
Expand Down Expand Up @@ -201,12 +204,11 @@ The minimum required version is 3.29.0.
The install command includes ``--set nfd.enabled=false`` to prevent ``kata-deploy`` from deploying NFD.
The GPU Operator will deploy and manage NFD in the next step.


#. Verify that the ``kata-deploy`` pod is running:

.. code-block:: console

$ kubectl get pods -n kata-system | grep kata-deploy
$ kubectl -n kata-system | grep kata-deploy

*Example Output:*

Expand Down Expand Up @@ -234,10 +236,10 @@ The minimum required version is 3.29.0.
kata-qemu-nvidia-gpu-tdx kata-qemu-nvidia-gpu-tdx 40s

Several runtimes are installed by the ``kata-deploy`` chart.
The ``kata-qemu-nvidia-gpu`` runtime class is used with Kata
The ``kata-qemu-nvidia-gpu`` runtime class is used with Kata
Containers, in a non-Confidential Containers scenario.
The ``kata-qemu-nvidia-gpu-snp`` for AMD-based systems or
``kata-qemu-nvidia-gpu-tdx`` for Intel-based systems runtime
The ``kata-qemu-nvidia-gpu-snp`` for AMD-based systems or
``kata-qemu-nvidia-gpu-tdx`` for Intel-based systems runtime
classes are used to deploy Confidential Containers workloads.

The ``kata-deploy`` chart typically creates these runtime classes within 1-2 minutes after the ``kata-deploy`` pod reaches ``Running``.
Expand Down Expand Up @@ -300,7 +302,7 @@ For more details on each of the GPU Operator components, refer to the :ref:`GPU
--set sandboxWorkloads.mode=kata \
--set nfd.enabled=true \
--set nfd.nodefeaturerules=true \
--version=v26.3.1
--version=${gpu_operator_version}

*Example Output:*

Expand All @@ -316,7 +318,7 @@ For more details on each of the GPU Operator components, refer to the :ref:`GPU
``STATUS: deployed`` confirms the Helm release succeeded.
The ``--wait`` flag instructs Helm to wait until the release is deployed before returning.
It may take 3-5 minutes for the Helm command to complete.

Use the following steps to confirm the GPU Operator components are deployed and configured correctly.

#. Verify that all GPU Operator pods, especially the Confidential Computing Manager, Kata Device Plugin and VFIO Manager operands, are running:
Expand Down Expand Up @@ -436,17 +438,17 @@ The following example installs the GPU Operator with both ``P_GPU_ALIAS`` and ``
.. code-block:: console

$ helm install --wait --timeout 10m --generate-name \
-n gpu-operator --create-namespace \
nvidia/gpu-operator \
--set sandboxWorkloads.enabled=true \
--set sandboxWorkloads.mode=kata \
--set nfd.enabled=true \
--set nfd.nodefeaturerules=true \
--set kataSandboxDevicePlugin.env[0].name=P_GPU_ALIAS \
--set kataSandboxDevicePlugin.env[0].value="" \
--set kataSandboxDevicePlugin.env[1].name=NVSWITCH_ALIAS \
--set kataSandboxDevicePlugin.env[1].value="" \
--version=v26.3.1
-n gpu-operator --create-namespace \
nvidia/gpu-operator \
--set sandboxWorkloads.enabled=true \
--set sandboxWorkloads.mode=kata \
--set nfd.enabled=true \
--set nfd.nodefeaturerules=true \
--set kataSandboxDevicePlugin.env[0].name=P_GPU_ALIAS \
--set kataSandboxDevicePlugin.env[0].value="" \
--set kataSandboxDevicePlugin.env[1].name=NVSWITCH_ALIAS \
--set kataSandboxDevicePlugin.env[1].value="" \
--version=${gpu_operator_version}

After installing the GPU Operator, you can view the GPU or NVSwitch resource types available on a node by running the following command:

Expand Down Expand Up @@ -474,7 +476,7 @@ Next Steps

.. note::

You now have a working Confidential Containers runtime.
You now have a working Confidential Containers runtime.

Attestation is what cryptographically verifies the TEE and releases secrets to a
production workload. For attestation concepts and a local
Expand Down
55 changes: 17 additions & 38 deletions confidential-containers/prerequisites.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,46 +27,45 @@ As a :ref:`Kubernetes Cluster Administrator <coco-persona-kubernetes-cluster-adm
You perform most steps in this section.
If you do not have access to host firmware, coordinate with your :ref:`Hardware IT Administrator <coco-persona-hardware-it-administrator>` or :ref:`Host OS Administrator <coco-persona-host-os-administrator>` to confirm or implement hardware prerequisites.

For validated hardware and software versions, refer to :doc:`Supported Platforms <supported-platforms>`.
Use the checklists below for an at-a-glance summary, then follow each linked section for verification steps.
Use the following checklists as at-a-glance summary and supplement to the :doc:`Supported Platforms <supported-platforms>` page.

**Hardware prerequisites**
**Hardware Configuration Requirements**

For validated hardware, refer to :doc:`Supported Platforms <supported-platforms>`.

.. list-table::
:header-rows: 1
:widths: 30 70

* - Prerequisite
* - Configuration Requirement
- Details
* - :ref:`Use a supported platform <coco-prereq-supported-platform>`
- CPU, GPU, and host OS match :doc:`Supported Platforms <supported-platforms>`
* - :ref:`Hardware virtualization and ACS enabled <coco-prereq-hw-virtualization>`
- Hardware virtualization and ACS enabled in host BIOS
* - :ref:`IOMMU enabled <coco-prereq-iommu>`
- IOMMU enabled on each host through the kernel command line (``amd_iommu=on`` or ``intel_iommu=on``)
* - :ref:`No host NVIDIA GPU drivers <coco-prereq-no-host-drivers>`
- No NVIDIA GPU drivers installed or loaded on worker hosts.

**Cluster prerequisites**
**Cluster Configuration Requirements**

For validated software components, refer to :doc:`Supported Platforms <supported-platforms>`.

.. list-table::
:header-rows: 1
:widths: 30 70

* - Prerequisite
* - Configuration Requirement
- Details
* - :ref:`A Kubernetes cluster and cluster administrator access <coco-prereq-cluster-admin>`
- Cluster administrator access to a Kubernetes cluster running a supported version (refer to :ref:`Supported Software Components <coco-supported-software-components>`)
* - :ref:`containerd 2.2.2 installed <coco-prereq-containerd>`
- containerd 2.2.2 installed on each GPU worker node
- Cluster administrator access to a Kubernetes cluster running a supported version.
* - :ref:`Helm installed <coco-prereq-helm>`
- Helm installed on your cluster administration system
* - :ref:`Kubelet configured <coco-prereq-kubelet>`
- Enable ``KubeletPodResourcesGet`` (required before Kubernetes v1.34) and ``RuntimeClassInImageCriApi`` feature gates; set ``runtimeRequestTimeout: 20m`` on GPU worker nodes

*****************
Hardware and BIOS
*****************
********************************************
Hardware and BIOS Configuration Requirements
********************************************

.. _coco-prereq-supported-platform:

Expand Down Expand Up @@ -172,9 +171,9 @@ In this architecture, the NVIDIA GPU Operator handles GPU driver installation an

Refer to `Removing the Driver <https://docs.nvidia.com/datacenter/tesla/driver-installation-guide/removing-the-driver.html>`_ in the NVIDIA Driver Installation Guide.

******************
Kubernetes Cluster
******************
*********************************************
Kubernetes Cluster Configuration Requirements
*********************************************

The following sections describe requirements for worker nodes and for the system you use for cluster administration.

Expand All @@ -186,26 +185,6 @@ Kubernetes Cluster and Cluster Administrator Access
You must have cluster administrator access to a Kubernetes cluster running a supported Kubernetes version.
Refer to the :ref:`Supported Software Components <coco-supported-software-components>` section in :doc:`Supported Platforms <supported-platforms>` for supported Kubernetes and component versions.

.. _coco-prereq-containerd:

containerd 2.2.2
================

Verify the installed version on each GPU worker node:

.. code-block:: console

$ containerd --version

*Example Output:*

.. code-block:: output

containerd containerd.io 2.2.2 ...

Your actual output may vary, but the reported version must be ``2.2.2``.
If you are running a different version on any worker node, refer to the `containerd Getting Started guide <https://containerd.io/docs/2.2/getting-started/>`_ for installation instructions.

.. _coco-prereq-helm:

Helm
Expand Down Expand Up @@ -256,7 +235,7 @@ Apply these settings as follows:
#. Open the kubelet configuration file:

.. code-block:: console

$ sudo nano /var/lib/kubelet/config.yaml

This is typically located at ``/var/lib/kubelet/config.yaml``, but your configuration file may be in a different location.
Expand Down
34 changes: 34 additions & 0 deletions confidential-containers/release-notes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,40 @@ This document describes the new features and known issues for the NVIDIA Confide

----

.. _coco-v1.1.0:

1.1.0
=====

This release expands hardware coverage and updates the validated software stack.

New Features
------------

* Added support for the NVIDIA HGX B300 platform with both single-GPU and multi-GPU passthrough.

* Added support for Ubuntu 26.04 as a host operating system.

* Added support for the following software components:

* Kata Containers 3.31.0
* containerd 2.3.x


Docs Changelog
--------------

The :ref:`coco-install-kata-chart` procedure was updated for this release.
Changes include:

* Installs ``kata-deploy`` with a values file instead of inline ``--set`` flags.

* Includes a new sample values file, :file:`samples/kata-nvidia-gpu-values.yaml`, that configures the ``kata-deploy`` Helm chart for the NVIDIA Confidential Containers reference architecture (NVIDIA GPU shims only, NFD disabled, ``nydus`` snapshotter, and per-shim runtime class node selectors).

* Adds a readiness verification step using ``kubectl rollout status ds/kata-deploy``. This step relies on the readiness reporting added in Kata Containers 3.31.0 and lets you confirm that ``kata-deploy`` has finished extracting artifacts and restarting containerd on every node before continuing.

----

.. _coco-v1.0.0:

*****
Expand Down
107 changes: 107 additions & 0 deletions confidential-containers/samples/kata-nvidia-gpu-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# Example values file to enable NVIDIA GPU shims for the NVIDIA
# Confidential Containers Reference Architecture.

# Set to true for verbose kata-deploy and Kata runtime logging.
debug: false

# Disable Node Feature Discovery (NFD) deployment by kata-deploy.
# Both kata-deploy and the GPU Operator deploy NFD by default. This
# reference architecture relies on the NFD instance that the GPU Operator
# deploys and manages, so the kata-deploy NFD is turned off to avoid a
# duplicate, conflicting deployment.
nfd:
enabled: false

# Install the nydus snapshotter on each node alongside containerd.
# The confidential -snp and -tdx shims below use nydus to pull container
# images directly into the confidential VM (guest pull), which keeps image
# contents inside the trusted execution environment (TEE).
snapshotter:
setup: ["nydus"]

# Disable the chart's default hypervisor/TEE shims and opt in only to
# the NVIDIA GPU shims supported by this reference architecture.
shims:
disableAll: true

# Non-confidential NVIDIA GPU passthrough shim used when Confidential
# Computing mode is off on the node. The runtime class is restricted to
# nodes where the GPU Operator's Confidential Computing Manager has
# reported nvidia.com/cc.ready.state=false, so it will not schedule on
# CC-ready nodes. The empty containerd snapshotter falls back to the
# default (overlayfs); guest pull is not used for this non-confidential
# path.
qemu-nvidia-gpu:
enabled: true
supportedArches:
- amd64
allowedHypervisorAnnotations: []
containerd:
snapshotter: ""
runtimeClass:
# This label is automatically added by the GPU Operator.
nodeSelector:
nvidia.com/cc.ready.state: "false"

# Confidential NVIDIA GPU passthrough for AMD SEV-SNP nodes.
# Scheduled where the GPU Operator reports CC mode is on AND NFD
# reports SEV-SNP support. Set agent.httpsProxy / agent.noProxy if
# the guest needs a proxy to reach the registry.
qemu-nvidia-gpu-snp:
enabled: true
supportedArches:
- amd64
allowedHypervisorAnnotations: []
containerd:
snapshotter: "nydus"
forceGuestPull: false
crio:
guestPull: true
agent:
httpsProxy: ""
noProxy: ""
runtimeClass:
# These labels are automatically added by the GPU Operator and NFD
# respectively.
nodeSelector:
nvidia.com/cc.ready.state: "true"
amd.feature.node.kubernetes.io/snp: "true"

# Confidential NVIDIA GPU passthrough for Intel TDX nodes.
# Same selectors and snapshotter behavior as the SNP shim above,
# but pinned to TDX-capable hosts.
qemu-nvidia-gpu-tdx:
enabled: true
supportedArches:
- amd64
allowedHypervisorAnnotations: []
containerd:
snapshotter: "nydus"
forceGuestPull: false
crio:
guestPull: true
agent:
httpsProxy: ""
noProxy: ""
runtimeClass:
# These labels are automatically added by the GPU Operator and NFD
# respectively.
nodeSelector:
nvidia.com/cc.ready.state: "true"
intel.feature.node.kubernetes.io/tdx: "true"

# Default shim when a pod does not request a runtime class. Set to the
# non-confidential shim so pods only run in a confidential VM when
# they explicitly request the -snp or -tdx runtime class.
defaultShim:
amd64: qemu-nvidia-gpu # Can be changed to qemu-nvidia-gpu-snp or qemu-nvidia-gpu-tdx if preferred

# Create one Kubernetes RuntimeClass per enabled shim above
# (kata-qemu-nvidia-gpu, kata-qemu-nvidia-gpu-snp, kata-qemu-nvidia-gpu-tdx).
# createDefault: false suppresses the generic "kata" RuntimeClass since
# you should always reference a specific NVIDIA shim
# by name in pod specs.
runtimeClasses:
enabled: true
createDefault: false
defaultName: "kata"
Loading