Skip to main content

DGX Spark — GPU + AI Serving

Overview​

The NVIDIA DGX Spark (10.0.0.251) is the AI inference compute in the enclave. It joins the RKE2 management cluster as an agent node, making its GPUs schedulable as Kubernetes resources. Once joined, the NVIDIA GPU Operator installs the driver stack via DaemonSet, and vLLM serves language models delivered through Harbor as OCI artifacts.

:::caution arm64 — not amd64 The DGX Spark is arm64 (aarch64). Every artifact that touches this node — RKE2 agent tarball, GPU Operator images, vLLM image, and model delivery tooling — must have an arm64 build. Hauler manifests for these components must include linux/arm64 platform entries. Verify before syncing: an amd64-only tarball will fail silently at image pull time. :::

:::info Coming soon The DGX Spark join procedure and AI serving stack will be documented here once this component is deployed. The stubs below describe the planned approach. :::

Step 1 — RKE2 Agent Join​

The DGX Spark joins the existing RKE2 cluster as an agent node. The 40_bootstrap-rke2.sh script will be invoked in agent mode, targeting the arm64 RKE2 tarball from the Hauler store.

Planned procedure:

  1. Confirm the arm64 RKE2 tarball is present in the Hauler store and Harbor
  2. Transfer the agent install tarball to the DGX Spark (or mount from NFS)
  3. Run 40_bootstrap-rke2.sh in agent mode on the DGX Spark, pointing at the RKE2 server VIP 10.0.0.30 for the join token
  4. Verify the node appears Ready in the RKE2 cluster with role <none> (agent)
# Verify DGX joined — run from nuc-00 with RKE2 kubeconfig
kubectl --kubeconfig ~/.kube/carbide-enclave-rke2.kubeconfig get nodes -o wide
# DGX Spark should appear with ARCH=arm64
note

The DGX Spark will be tainted so that only GPU workloads are scheduled on it. General cluster workloads remain on the amd64 VM nodes.

Step 2 — NVIDIA GPU Operator​

The NVIDIA GPU Operator is deployed via Helm from the Hauler-served Harbor registry. It installs and manages the full NVIDIA driver stack (drivers, container toolkit, device plugin, DCGM exporter) as DaemonSets on nodes with the nvidia.com/gpu.present=true label.

Planned procedure:

  1. Label the DGX Spark node: kubectl label node <dgx-node> nvidia.com/gpu.present=true
  2. Install GPU Operator via Helm, pointing at Harbor for images
  3. Verify the GPU resource is visible:
kubectl --kubeconfig ~/.kube/carbide-enclave-rke2.kubeconfig \
  get nodes <dgx-node> -o jsonpath='{.status.allocatable}' | jq .
# Should include "nvidia.com/gpu": "N"

:::info Coming soon Helm values and exact GPU Operator version will be documented here. Version is pinned at v24.3.0 in scripts/env.d/carbide-enclave.sh. :::

Step 3 — AI Serving (vLLM / Ollama)​

Language model inference runs inside the RKE2 cluster on the DGX Spark node. vLLM is the primary serving runtime. Ollama is available as a fallback for models that do not have optimized vLLM support.

Model weights are delivered as OCI artifacts pushed to Harbor — they do not come from Hugging Face or any external source. This keeps the entire model delivery pipeline inside the airgap boundary.

Planned procedure:

  1. Push model weights to Harbor as an OCI artifact (via oras push or Hauler FileContent)
  2. Deploy the vLLM Deployment with a nodeSelector targeting the DGX Spark and a resourceRequest for nvidia.com/gpu
  3. An initContainer pulls the model from Harbor OCI into a shared volume before the vLLM process starts
  4. Expose the OpenAI-compatible endpoint via a Kubernetes Service and ingress
# Conceptual structure — not final
resources:
  requests:
    nvidia.com/gpu: "1"
nodeSelector:
  nvidia.com/gpu.present: "true"

:::info Coming soon Final deployment manifests, model packaging procedure, and Harbor OCI artifact push commands will be documented here once the DGX Spark is joined and the GPU Operator is verified. :::

References​

ResourceURL
NVIDIA GPU Operator docshttps://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/
NVIDIA GPU Operator Helm charthttps://helm.ngc.nvidia.com/nvidia/charts/
vLLM documentationhttps://docs.vllm.ai
vLLM Docker imageshttps://hub.docker.com/r/vllm/vllm-openai
Harbor OCI artifact push (oras)https://oras.land/docs/
Hauler FileContent spechttps://hauler.dev/docs/reference/manifests/#files