Skip to main content

Network Planning

The network plan is one of the most important Day 0 outputs. IPs, DNS names, and subnet choices ripple through every config file, every TLS certificate SAN, and every Kubernetes service endpoint. Getting this wrong after the cluster is running is expensive.

Subnet: 10.0.0.0/22 Domain: carbide-enclave.kubernerdes.com


IP Assignment Table​

IPHostnameRole
10.0.0.1gatewayDefault gateway / router
10.0.0.10nuc-00Bastion / admin host (DNS, DHCP, NTP, Web, TFTP, step-ca, Hauler)
10.0.0.11nasNAS / NFS server
10.0.0.30rancher (VIP)Rancher VIP — RKE2 API :6443, supervisor :9345, Rancher UI :443
10.0.0.31rancher-01RKE2 control plane node 1
10.0.0.32rancher-02RKE2 control plane node 2
10.0.0.33rancher-03RKE2 control plane node 3
10.0.0.40observability (LB)Observability stack load balancer VIP
10.0.0.50apps (LB)Application load balancer VIP
10.0.0.98keycloak (VIP)Keycloak OIDC service
10.0.0.99harbor (VIP)Harbor OCI registry + UI
10.0.0.100harvester (VIP)Harvester API + UI virtual IP
10.0.0.101nuc-01Harvester node 1 (management NIC)
10.0.0.102nuc-02Harvester node 2 (management NIC)
10.0.0.103nuc-03Harvester node 3 (management NIC)
10.0.0.111nuc-01-kvmnuc-01 IP-KVM management interface
10.0.0.112nuc-02-kvmnuc-02 IP-KVM management interface
10.0.0.113nuc-03-kvmnuc-03 IP-KVM management interface
10.0.0.251sparkNVIDIA DGX Spark (arm64 AI inference node)
10.0.0.172–254—DHCP dynamic pool
tip

Leave a gap between .113 and .172 for future static assignments. The DHCP pool starting at .172 gives you plenty of headroom.


Service VIPs​

The "VIP" addresses (.30, .98, .99, .100) are virtual IPs managed by Harvester's built-in load balancer (backed by kube-vip or MetalLB). They float across healthy nodes — you point DNS at the VIP, and the LB handles failover.

VIPServicePorts
10.0.0.30Rancher / RKE2 API443 (Rancher UI), 6443 (Kubernetes API), 9345 (RKE2 supervisor)
10.0.0.98Keycloak443
10.0.0.99Harbor443 (UI + registry API)
10.0.0.100Harvester443 (UI + API)

These VIPs are configured in the Harvester post-install bootstrap step, not during bare-metal install.


DNS Design​

DNS is served by BIND (named) on nuc-00. It is authoritative for carbide-enclave.kubernerdes.com and the reverse zone 0.0.10.in-addr.arpa.

:::note OpenSUSE Leap 15.6 BIND quirks

  • The root hints file is root.hint (not named.ca)
  • Zone files live at /var/lib/named/ root, not in a master/ subdirectory
  • These paths are already correct in the committed named configs :::

Key DNS Records​

NameTypeValueNotes
nuc-00A10.0.0.10Bastion
caCNAMEnuc-00step-ca internal CA shortname
nuc-01A10.0.0.101
nuc-02A10.0.0.102
nuc-03A10.0.0.103
harvesterA10.0.0.100Harvester VIP
rancherA10.0.0.30Rancher VIP
rancher-01A10.0.0.31RKE2 node 1
rancher-02A10.0.0.32RKE2 node 2
rancher-03A10.0.0.33RKE2 node 3
harborA10.0.0.99Harbor VIP
*.harborA10.0.0.99Harbor notary / sub-projects
keycloakA10.0.0.98Keycloak VIP
sparkA10.0.0.251DGX Spark
nasA10.0.0.11NAS/NFS

All names are under carbide-enclave.kubernerdes.com — e.g., the full Rancher hostname is rancher.carbide-enclave.kubernerdes.com.


DHCP​

ISC dhcpd runs on nuc-00. Configuration approach:

  • Static leases by MAC address for all named nodes (nuc-01/02/03, KVM interfaces, spark, nas)
  • Dynamic pool 10.0.0.172–10.0.0.254 for anything else (installer environments, test VMs, etc.)
  • next-server and filename options set for PXE/iPXE boot on the management subnet
note

On OpenSUSE Leap 15.6, there is no dhcpd group. Use the numeric GID from id -g dhcpd when setting file ownership. This is already handled in the bootstrap script.


iPXE Boot Chain​

When a Harvester node powers on cold, it boots over the network using this chain:

node powers on
→ DHCP request (nuc-00:67)
→ nuc-00 assigns static IP by MAC
→ returns: next-server=10.0.0.10, filename=ipxe.efi
→ TFTP (nuc-00:69)
→ node downloads ipxe.efi
→ iPXE firmware initializes, sends second DHCP request (user-class=iPXE)
→ nuc-00 matches user-class, returns: filename=http://10.0.0.10/harvester/harvester/ipxe-menu
→ iPXE loads the boot menu over HTTP (nuc-00:80)
→ menu entry matches node hostname/IP
→ node-specific Harvester install config loaded
→ Harvester installer runs unattended

The per-node DHCP host entries in dhcpd.conf are what route each NUC to the right Harvester configuration automatically — no manual intervention needed at the console.


Kubernetes Internal Networks​

The RKE2 cluster uses separate CIDRs for pod and service networks. These must not overlap with the physical subnet (10.0.0.0/22) or with each other.

NetworkCIDRPurpose
Pod network10.42.0.0/16RKE2 default; one /24 per node
Service network10.43.0.0/16RKE2 default; ClusterIP allocations
Physical subnet10.0.0.0/22Physical hosts and VMs
caution

Do not change the pod or service CIDRs after the cluster is running — it requires a full cluster rebuild. The RKE2 defaults (10.42.0.0/16 and 10.43.0.0/16) do not overlap with the enclave's physical subnet (10.0.0.0/22), so the defaults are safe to use as-is.


Pre-Deployment DNS Checklist​

Before starting Day 1, verify:

  • Forward DNS resolves all hostnames in the table above to the correct IPs
  • Reverse DNS resolves all static IPs back to their hostnames
  • dig +short rancher.carbide-enclave.kubernerdes.com returns 10.0.0.30
  • dig +short harbor.carbide-enclave.kubernerdes.com returns 10.0.0.99
  • dig +short -x 10.0.0.99 returns harbor.carbide-enclave.kubernerdes.com
  • dig +short ca.carbide-enclave.kubernerdes.com returns 10.0.0.10
  • All wildcard entries (*.harbor) resolve correctly
  • nuc-00 is set as the DNS resolver for all nodes (check /etc/resolv.conf or NetworkManager config)