News

Business insights and articles written by our team of  professionals

VKS – Kubernetes on Sovereign Ground

If you’ve deployed VCF 9 in an air-gapped environment, you’ve already addressed the platform-level challenges of software distribution and lifecycle management. vSphere Kubernetes Service (VKS) introduces a second layer of complexity: container workloads that assume connectivity to public registries, package repositories, and update channels.

This article examines the VKS-specific obstacles in disconnected environments and the mechanisms VCF 9 provides to address them. For the underlying VCF 9 deployment considerations, see our companion article on VCF 9 in air-gapped environments.

VKS Packages and Supervisor Services

In VCF 9, VKS is delivered as a Supervisor Service—a collection of Carvel packages that bundle container images with deployment configuration. This architecture decouples VKS from the vSphere Supervisor release cycle, enabling independent upgrades.

For air-gapped environments, VKS supports three registration methods. Synchronous registration occurs automatically with vCenter updates. Asynchronous public registration allows manual registration from VMware’s public registry. Asynchronous private registration enables manual registration after relocating packages to an internal registry.

For isolated networks, the private registry workflow uses the imgpkg utility to relocate packages: download the VKS definition YAML from the public repository, extract the imgpkgBundle image path, create a tar archive with cosign signatures preserved, then upload to your private registry. Update the YAML to reference the new location and register with the Supervisor. Once registered, the system uses its local registry for subsequent upgrades without requiring internet connectivity.

The same approach applies to VKS Standard Packages—the open source applications bundled in Carvel format for deployment on VKS clusters. These support air-gapped operation with a local registry, including registries that require authentication.

vSphere Kubernetes Release Management

VKS provisions clusters using vSphere Kubernetes Releases (VKr). Each release bundles a specific Kubernetes version with compatible container images, VM templates (OVAs), and configuration metadata. In connected environments, these synchronise automatically via subscribed Content Libraries. Dark sites require manual distribution through local Content Libraries.

The offline workflow involves creating a local Content Library in vCenter (subscribed libraries require internet connectivity), downloading VKr bundles from Broadcom’s distribution endpoint at wp-content.vmware.com/v2/latest/, transferring to the isolated environment, and importing into the library.

Each VKr requires four files: the OVF descriptor, VMDK disk image, certificate file, and manifest. Although only the OVF and VMDK are imported directly, all four files must be present in the source directory during import—the certificate and manifest are validated even if not uploaded. The destination item name must match the VKr folder name exactly; mismatches prevent the Supervisor from resolving the image to a valid release.

After import, associate the Content Library with the Supervisor. Validate VKr availability using kubectl get kubernetesreleases before attempting cluster provisioning.

The Container Distribution Problem

Kubernetes workloads depend on container images. In connected environments, clusters pull from public registries—Docker Hub, GitHub Container Registry, Amazon ECR—without consideration for the network path. VKS itself requires images for its core components: the vSphere Kubernetes Release (VKr) artifacts, system pods (CoreDNS, kube-proxy, etcd), CNI plugins, and Cloud Provider Interface components.

In an isolated network segment, none of these external registries are reachable. The solution is a local container registry within the classification boundary.

Harbor is the recommended choice for VCF environments. It provides OCI-compliant image storage, vulnerability scanning with offline database updates, role-based access control that integrates with vSphere SSO, and Helm chart repository functionality. Harbor can be deployed as a Supervisor Service within VCF itself—the recommended pattern for integrated container registry services. Avoid hosting container images on in-band registries running within VKS clusters; upgrading the cluster requires images that the cluster itself is serving, creating a dependency loop that complicates lifecycle operations.

Implementation requires mirroring the complete image manifest for each supported Kubernetes version, transferring via approved cross-domain mechanisms, and establishing a synchronisation cadence aligned with patching requirements. Where an internet-connected staging system exists within the organisation, this provides the acquisition point. Where it doesn’t—common in highly compartmented environments—alternatives include acquiring images via a trusted external party, using a temporary isolated connection for controlled downloads, or receiving content through a cross-domain solution from a lower-classification network that does have connectivity. The acquisition method affects procedural complexity and approval timelines but doesn’t change the technical requirement: all images must reach the local registry before workloads can consume them. Plan storage capacity for all required images plus growth headroom.

Certificate Trust Throughout the Stack

Container runtimes validate TLS certificates when pulling images. Local registries use certificates from internal PKI, which clusters won’t trust by default. Symptoms include image pull failures, VKr deployment failures, and workload pods stuck in ImagePullBackOff.

Certificate trust must be configured at multiple levels. The Supervisor requires the CA certificate in its trust store. The embedded Harbor (if used) needs certificates from the same PKI. VKr VM templates must include the CA in the system trust store with containerd configured to trust the local registry.

This dependency chain means certificate expiry affects the entire stack. Plan renewal with adequate lead time and test certificate updates in non-production before rolling to operational clusters.

Planning Considerations

Successful VKS deployment in air-gapped environments depends on establishing the local container registry before any VKS configuration, mirroring complete image manifests for all supported VKr versions, configuring certificate trust throughout the stack (Supervisor, Harbor, VKr templates), documenting transfer procedures for images and packages aligned with security operating procedures, and integrating VKS lifecycle management into existing change management frameworks.

The procedural overhead is higher than connected environments, but the payoff is substantial: production Kubernetes on sovereign infrastructure, fully isolated from the public internet, with complete control over every component in the stack. Modern container workloads with uncompromising security—exactly where they need to be.


Spartan is a UK consultancy specialising in VMware Cloud Foundation for defence and government organisations.

Facebook
Twitter
LinkedIn