Networking & CNI¶

Every CAPVCO cluster needs a CNI (the pod network) and two IP ranges that must not collide with anything. This guide explains what ships, what you choose, and the one mistake that breaks a cluster before it starts.

The CNI add-on¶

CAPVCO installs a CNI for you as a ClusterResourceSet add-on, selected by a label on the Cluster:

metadata:
  labels:
    capvco.infrastructure.cluster.x-k8s.io/cni: calico

The stock templates set this — you don't add it by hand. There is one value, calico, and it pins veth_mtu to 1350 for every cluster.

Why 1350 everywhere

The site-to-site VPN that meshes geo cloudspaces has a measured path MTU of ~1400. With the default MTU, packets over the tunnel are silently dropped and you get maddening intermittent pod connectivity. Calico sets veth_mtu: 1350 (1400 − 50 for VXLAN overhead), which is the knob that actually works — a FelixConfiguration MTU override does not take effect here. The reduced MTU is safe on ordinary single-cloudspace networks too, so calico uses it unconditionally: a cluster that starts in one location can later expand into more cloudspaces/locations over the tunnel without any CNI or MTU change.

Pod and Service CIDRs¶

Two ranges, set on the Cluster (defaults shown):

Range	Default	Set by
Pod CIDR	`10.244.0.0/16`	`POD_CIDR` → `Cluster.spec.clusterNetwork.pods`
Service CIDR	`10.96.0.0/12`	`SERVICE_CIDR` → `Cluster.spec.clusterNetwork.services`

The overlap rule (read this)¶

The pod and service CIDRs must not overlap the cloudspace's private network. A cloudspace is a real routed L3 network; if your pod CIDR overlaps the node LAN, Calico will hijack traffic meant for the LAN (including the API server and etcd peers) and the cluster tears itself apart in confusing ways.

CAPVCO refuses to provision on a detected overlap rather than let you find out the hard way:

VCOCluster … FailureReason=CIDROverlap

Rules of thumb:

Keep the defaults unless you have a reason not to — 10.244.0.0/16 (pods) and 10.96.0.0/12 (services) don't overlap the cloudspace ranges whitesky.cloud hands out.
If you set a custom cloudspace private network (VCOCluster.spec.cloudspace.privateNetwork), pick pod/service CIDRs disjoint from it.
For geo clusters, every cloudspace's private network must be mutually non-overlapping and disjoint from the pod/service CIDRs. Leave spec.geo.privateNetworks unset and the controller auto-assigns disjoint /24s for you.

If you change the pod CIDR, keep Calico in sync

On the stock templates the Calico install reads the pod CIDR, so overriding POD_CIDR is enough. If you customise the CNI manifests yourself, make sure CALICO_IPV4POOL_CIDR matches POD_CIDR — a mismatch gives you pods that can't reach each other.

Cloudspace networking¶

Each cluster gets its own cloudspace — an isolated virtual datacenter with a private network and a gateway to an external network. You rarely touch this directly, but two knobs exist on VCOCluster.spec.cloudspace:

Field	Purpose
`privateNetwork`	Override the auto-assigned private CIDR (must not overlap pod/service CIDRs).
`externalNetworkId`	Pin a specific external network (the routable/public side). Defaults to the location's default external network.
`quotas`	Cap `vcpu` / `memoryMiB` / `diskGiB` / `publicIp` for the whole cloudspace.
`private`	Create the cloudspace with no external network (no egress, no public endpoint). See the caveat below.

cloudspace.private: true is not usable yet

A fully private cloudspace (no external network) is part of the air-gapped roadmap but is currently rejected by the controller (FailureReason=UnsupportedConfiguration) because the control-plane endpoint has nowhere public to live and the management cluster can't reach an internal-only endpoint yet. For zero-egress today, use an air-gapped cluster on a normal cloudspace. This is tracked in known limitations.

Next: give the control plane some redundancy → HA control plane.