Skip to main content

Network Planning

The network plan is one of the most important Day 0 outputs. IPs, DNS names, and subnet choices ripple through every config file, every TLS certificate SAN, and every Kubernetes service endpoint. Getting this wrong after the cluster is running is expensive.

Subnet: 10.0.0.0/22 Domain: carbide-enclave.kubernerdes.com


IP Assignment Table

IPHostnameRole
10.0.0.1gatewayDefault gateway / router
10.0.0.10nuc-00Bastion / admin host (DNS, DHCP, NTP, Web, TFTP, step-ca, Hauler)
10.0.0.11nasNAS / NFS server
10.0.0.30rancher (VIP)Rancher VIP — RKE2 API :6443, supervisor :9345, Rancher UI :443
10.0.0.31rancher-01RKE2 control plane node 1
10.0.0.32rancher-02RKE2 control plane node 2
10.0.0.33rancher-03RKE2 control plane node 3
10.0.0.40observability (LB)Observability stack load balancer VIP
10.0.0.50apps (LB)Application load balancer VIP
10.0.0.98keycloak (VIP)Keycloak OIDC service
10.0.0.99harbor (VIP)Harbor OCI registry + UI
10.0.0.100harvester (VIP)Harvester API + UI virtual IP
10.0.0.101nuc-01Harvester node 1 (management NIC)
10.0.0.102nuc-02Harvester node 2 (management NIC)
10.0.0.103nuc-03Harvester node 3 (management NIC)
10.0.0.111nuc-01-kvmnuc-01 IP-KVM management interface
10.0.0.112nuc-02-kvmnuc-02 IP-KVM management interface
10.0.0.113nuc-03-kvmnuc-03 IP-KVM management interface
10.0.0.251sparkNVIDIA DGX Spark (arm64 AI inference node)
10.0.0.172–254DHCP dynamic pool
tip

Leave a gap between .113 and .172 for future static assignments. The DHCP pool starting at .172 gives you plenty of headroom.


Service VIPs

The "VIP" addresses (.30, .98, .99, .100) are virtual IPs managed by Harvester's built-in load balancer (backed by kube-vip or MetalLB). They float across healthy nodes — you point DNS at the VIP, and the LB handles failover.

VIPServicePorts
10.0.0.30Rancher / RKE2 API443 (Rancher UI), 6443 (Kubernetes API), 9345 (RKE2 supervisor)
10.0.0.98Keycloak443
10.0.0.99Harbor443 (UI + registry API)
10.0.0.100Harvester443 (UI + API)

These VIPs are configured in the Harvester post-install bootstrap step, not during bare-metal install.


DNS Design

DNS is served by BIND (named) on nuc-00. It is authoritative for carbide-enclave.kubernerdes.com and the reverse zone 0.0.10.in-addr.arpa.

:::note OpenSUSE Leap 15.6 BIND quirks

  • The root hints file is root.hint (not named.ca)
  • Zone files live at /var/lib/named/ root, not in a master/ subdirectory
  • These paths are already correct in the committed named configs :::

Key DNS Records

NameTypeValueNotes
nuc-00A10.0.0.10Bastion
caCNAMEnuc-00step-ca internal CA shortname
nuc-01A10.0.0.101
nuc-02A10.0.0.102
nuc-03A10.0.0.103
harvesterA10.0.0.100Harvester VIP
rancherA10.0.0.30Rancher VIP
rancher-01A10.0.0.31RKE2 node 1
rancher-02A10.0.0.32RKE2 node 2
rancher-03A10.0.0.33RKE2 node 3
harborA10.0.0.99Harbor VIP
*.harborA10.0.0.99Harbor notary / sub-projects
keycloakA10.0.0.98Keycloak VIP
sparkA10.0.0.251DGX Spark
nasA10.0.0.11NAS/NFS

All names are under carbide-enclave.kubernerdes.com — e.g., the full Rancher hostname is rancher.carbide-enclave.kubernerdes.com.


DHCP

ISC dhcpd runs on nuc-00. Configuration approach:

  • Static leases by MAC address for all named nodes (nuc-01/02/03, KVM interfaces, spark, nas)
  • Dynamic pool 10.0.0.172–10.0.0.254 for anything else (installer environments, test VMs, etc.)
  • next-server and filename options set for PXE/iPXE boot on the management subnet
note

On OpenSUSE Leap 15.6, there is no dhcpd group. Use the numeric GID from id -g dhcpd when setting file ownership. This is already handled in the bootstrap script.


iPXE Boot Chain

When a Harvester node powers on cold, it boots over the network using this chain:

node powers on
→ DHCP request (nuc-00:67)
→ nuc-00 assigns static IP by MAC
→ returns: next-server=10.0.0.10, filename=ipxe.efi
→ TFTP (nuc-00:69)
→ node downloads ipxe.efi
→ iPXE firmware initializes, sends second DHCP request (user-class=iPXE)
→ nuc-00 matches user-class, returns: filename=http://10.0.0.10/harvester/harvester/ipxe-menu
→ iPXE loads the boot menu over HTTP (nuc-00:80)
→ menu entry matches node hostname/IP
→ node-specific Harvester install config loaded
→ Harvester installer runs unattended

The per-node DHCP host entries in dhcpd.conf are what route each NUC to the right Harvester configuration automatically — no manual intervention needed at the console.


Kubernetes Internal Networks

The RKE2 cluster uses separate CIDRs for pod and service networks. These must not overlap with the physical subnet (10.0.0.0/22) or with each other.

NetworkCIDRPurpose
Pod network10.42.0.0/16RKE2 default; one /24 per node
Service network10.43.0.0/16RKE2 default; ClusterIP allocations
Physical subnet10.0.0.0/22Physical hosts and VMs
caution

Do not change the pod or service CIDRs after the cluster is running — it requires a full cluster rebuild. The RKE2 defaults (10.42.0.0/16 and 10.43.0.0/16) do not overlap with the enclave's physical subnet (10.0.0.0/22), so the defaults are safe to use as-is.


Pre-Deployment DNS Checklist

Before starting Day 1, verify:

  • Forward DNS resolves all hostnames in the table above to the correct IPs
  • Reverse DNS resolves all static IPs back to their hostnames
  • dig +short rancher.carbide-enclave.kubernerdes.com returns 10.0.0.30
  • dig +short harbor.carbide-enclave.kubernerdes.com returns 10.0.0.99
  • dig +short -x 10.0.0.99 returns harbor.carbide-enclave.kubernerdes.com
  • dig +short ca.carbide-enclave.kubernerdes.com returns 10.0.0.10
  • All wildcard entries (*.harbor) resolve correctly
  • nuc-00 is set as the DNS resolver for all nodes (check /etc/resolv.conf or NetworkManager config)