Skip to content

Pre-baked node images

By default, stock-image nodes install containerd + Kubernetes on first boot — simple, but slow and it needs internet egress. A pre-baked image has all of that already installed, so nodes boot in a fraction of the time, deterministically, and without egress (the prerequisite for air-gapped clusters).

This guide covers building an image, publishing it into your whitesky.cloud location(s), and pointing a cluster at it.

When to use pre-baked

Stock image Pre-baked image
First-boot time Minutes (installs on boot) Fast (nothing to install)
Needs egress on boot Yes (pkgs.k8s.io) No
Determinism Depends on upstream apt at boot time Pinned at build time
Use for Learning, quick tests Production, geo, air-gapped

The base template defaults to a stock image; cluster-template-prebaked.yaml, -ccm, -backup, and -geo use pre-baked ones.

1. Build an image

Images are built with kubernetes-sigs/image-builder (QEMU/qcow2 target) — the recipe and wrapper live in image-builder/. Building needs a host with KVM (/dev/kvm).

  • In a release: every tag automatically builds a node image for each version in KUBERNETES_VERSIONS and attaches them to the GitLab Release under downloads/images/ — you usually don't build these by hand.
  • Ad-hoc in CI: trigger the build-node-image job manually on a branch pipeline; it runs on the KVM runner and produces dist/*.qcow2 artifacts (no release attachment).
  • Locally: run image-builder/build-node-image.sh on a KVM box (it verifies the toolchain and builds). See image-builder/README.md.

Each image is specific to one (OS, Kubernetes version) — e.g. Ubuntu 24.04 + Kubernetes v1.36.4.

2. Publish it to your location(s)

A qcow2 file isn't usable until it's imported into whitesky.cloud as a vm-image in a specific location. Use the capvco-imagepublish CLI (shipped as a release binary):

whitesky.cloud imports directly from a download URL — e.g. the GitLab release/artifact URL of the qcow2. This is the simplest path:

export VCO_PORTAL_URL=https://<your-whitesky.cloud-portal>
export VCO_IAM_REFRESH_URL=https://<your-iam-host>/v1/oauth/jwt/refresh
export VCO_JWT=<operator-jwt>  VCO_CUSTOMER_ID=<customer-id>

capvco-imagepublish \
  -url "https://gitlab.whitesky.cloud/api/v4/projects/730/packages/generic/capvco-images/node-images/capvco-ubuntu-2404-cloudimg-v1.36.4.qcow2" \
  -name "k8s-v1.36.4" \
  -locations <loc-a>,<loc-b>

If the image isn't hosted anywhere, upload it to objectspace/S3 and let the platform import the presigned URL:

export S3_ENDPOINT=… S3_BUCKET=… S3_ACCESS_KEY_ID=… S3_SECRET_ACCESS_KEY=…
capvco-imagepublish -file ./dist/node-v1.36.4.qcow2 -name "k8s-v1.36.4" -locations <loc-a>

capvco-imagepublish is idempotent — an already-published image of the same name in a location is adopted, not duplicated — and publishes into every -locations entry from one source. Full flag reference: CLI tools.

Publishing is operator-privileged

Creating a vm-image is a platform-operator privilege — a per-cluster customer credential gets a 403. Use an operator credential (VCO_JWT) for capvco-imagepublish. If you only have a customer credential, ask your whitesky.cloud operator to publish the image (or upload it out-of-band), then use the resulting image id via the override path below.

3. Point a cluster at the image

Each location reports its own image id — note them from the publish output (or the portal).

VCO_LOCATION=<loc-a> VCO_IMAGE_ID=<image-id-in-loc-a> \
clusterctl generate cluster my-cluster \
  --kubernetes-version v1.36.4 --infrastructure vco --flavor prebaked \
  --worker-machine-count 1 > my-cluster.yaml

VCO_IMAGE_ID has no default in any template on purpose — image ids are location-specific, and a wrong id would silently boot the wrong image, so you always supply it.

Images are location-scoped and not portable, so a geo cluster needs an id per site — the template takes one per site via GEO_SITEn_IMAGE_ID (which populate spec.geo.locationImages). Sites sharing a location use the same id:

GEO_SITE1_IMAGE_ID=<id-in-loc-a> \
GEO_SITE2_IMAGE_ID=<id-in-loc-b> \
GEO_SITE3_IMAGE_ID=<id-in-loc-c> \
# …plus GEO_SITEn_LOCATION / GEO_SITEn_EXTNET_ID and GEO_DNS_ZONE

which render as:

spec:
  geo:
    locationImages:
      - {location: <loc-a>, imageId: <id-in-a>}
      - {location: <loc-b>, imageId: <id-in-b>}
      - {location: <loc-c>, imageId: <id-in-c>}

See Geo-redundant clusters.

Boot disk ≥ 40 GiB

Pre-baked images require at least a 40 GiB boot disk (bootDiskGiB). The templates default to that — don't shrink it below the image's minimum.

Upgrades = new image + rolling replace

Kubernetes upgrades are never in-place. To move to a new version:

  1. build + publish a pre-baked image for the new version to each location,
  2. reference it (VCO_IMAGE_ID / spec.geo.locationImages, or a version-keyed spec.geo.images entry), and
  3. bump KubeadmControlPlane.spec.version and the MachineDeployment version — KCP rolls new machines on the new image, one at a time. See HA control plane.

For a geo cluster, spec.geo.images can resolve the right image id per (version, location) automatically, publishing on demand from an S3 source if you give it an operator credential (spec.geo.imagePublishCredentialSecretRef).

Next: run with no egress at all → Air-gapped clusters.