talos-extensions/nvidia-gpu/nvidia-container-toolkit

NVIDIA Container toolkit extension

Usage

Enable the extension in the machine configuration before installing Talos:

machine:
  install:
    extensions:
      - image: ghcr.io/siderolabs/nvidia-container-toolkit:<VERSION>

The following NVIDIA modules needs to be loaded, so add this to the talos config:

machine:
  kernel:
    modules:
      - name: nvidia
      - name: nvidia_uvm
      - name: nvidia_drm
      - name: nvidia_modeset

nvidia-container-cli loads BPF programs and requires relaxed KSPP setting for bpf_jit_harden, so Talos default setting should be overridden:

machine:
  sysctls:
    net.core.bpf_jit_harden: 1

Warning! This disables KSPP best practices setting.

Testing

Apply the following manifest to create a runtime class that uses the extension:

---
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: nvidia
handler: nvidia

Install the NVIDIA device plugin:

helm repo add nvdp https://nvidia.github.io/k8s-device-plugin
helm repo update
helm install nvidia-device-plugin nvdp/nvidia-device-plugin --version=0.11.0 --set=runtimeClassName=nvidia

Apply the following manifest to run CUDA pod via nvidia runtime:

---
apiVersion: v1
kind: Pod
metadata:
  name: gpu-operator-test
spec:
  restartPolicy: OnFailure
  runtimeClassName: nvidia
  containers:
  - name: cuda-vector-add
    image: "nvidia/samples:vectoradd-cuda11.6.0"
    resources:
      limits:
         nvidia.com/gpu: 1

The status can be viewed by running:

❯ kubectl get pods
NAME                READY   STATUS      RESTARTS   AGE
gpu-operator-test   0/1     Completed   0          13s

❯ kubectl logs gpu-operator-test
[Vector addition of 50000 elements]
Copy input data from the host memory to the CUDA device
CUDA kernel launch with 196 blocks of 256 threads
Copy output data from the CUDA device to the host memory
Test PASSED
Done