Calico Enterprise 3.23 (latest) documentation

vSphere Kubernetes Service (VKS)

Big picture

Install Calico Enterprise as a Supervisor Service on vSphere Kubernetes Service (VKS), and deploy it to guest clusters.

Before you begin

This guide requires:

VM Images: Ubuntu or Photon OVA images registered for a supported TKR version, with TKR in a healthy state. For Windows node pools, Windows Server OVA images are also required.
vSphere Resources: Sufficient capacity to provision guest clusters with at minimum:
- 1 control plane node
- 3 Linux worker nodes
OCI Registry: An OCI-compliant registry configured in vSphere for storing package manifests and container images.
Tigera registry credentials for accessing the Tigera container registry (contact the Tigera team).
VKS v3.8 and above
vCenter - v9.1.1 and above

Networking

Configuration	Dataplane	Encapsulation	BGP
Linux only (default)	IPTables	IPIP	Enabled

How to

Setting up Calico Enterprise on VKS has two main phases: set it as a Supervisor Service (Part 1) in vSphere Client (one time), then deploy it to a guest cluster (Part 2). For Air-gapped environments follow the additional steps in the Air-gapped environment section.

Part 1: Set up the Supervisor Service (one time cluster admin setup)

In the vSphere Client - Container Registries page setup the Tigera container registry's credentials for pulling Calico Enterprise images from Tigera's private container registry.
- Use quay.io as the Registry host URL, and Username/Password provided by Tigera. For air-gapped installs, add the private registry that holds the Calico Enterprise images instead (see Air-gapped private container registry).
In the vSphere Client - Supervisor Management page:
- Create a new Supervisor Service (or add a new version to an existing one).
- Upload Supervisor service Package and PackageMetadata resources (example below) and complete the setup workflow.
- Register the Supervisor Service and set registryName to match the registry name configured for Tigera's private container registry.

---
apiVersion: data.packaging.carvel.dev/v1alpha1
kind: Package
metadata:
  name: calico-enterprise-svs.tigera.io.3.23.3
spec:
  refName: calico-enterprise-svs.tigera.io
  version: 3.23.3
  template:
    spec:
      fetch:
      - imgpkgBundle:
          image: quay.io/tigera/calico-enterprise-svs:v3.23.3   #Change this to the registry where the carvel package is getting picked up
      template:
      - ytt:
          paths:
          - service
          - kapp
      - kbld:
          paths:
          - "-"
          - .imgpkg/images.yml
          - kbld/
      deploy:
      - kapp: {}
---
apiVersion: data.packaging.carvel.dev/v1alpha1
kind: PackageMetadata
metadata:
  name: calico-enterprise-svs.tigera.io
spec:
  displayName: "Calico Enterprise SVS"
  shortDescription: "Manages Calico Enterprise Addon Repository and Installation"

Part 2: Deploy a guest cluster with Calico Enterprise

Guest cluster prerequisites

Tigera license file (contact the Tigera team).
Dynamic PV provisioning via a StorageClass, required for Elasticsearch storage.

Step 1 (optional): Apply the AddonConfig

An AddonConfig is needed only if you want to change the default Calico Enterprise configuration, for example to create a cluster with Windows nodes. It is not required for a Linux-only cluster that uses the defaults.

The following example AddonConfig enables the Windows dataplane and disables policy recommendation.

info

Update the following fields before applying:

metadata.name and metadata.namespace
spec.values.helmValues.installation.registry to use Calico Enterprise images from a private registry (for air-gapped installs). This should be the same registry where Calico Enterprise images are pushed to using Air-gapped private container registry of this doc. Optionally spec.values.helmValues.installation.imagePath can be set if the Calico Enterprise images were pushed to a specific folder within the private registry.

note

All Calico Enterprise features (logCollector, logStorage, manager, monitor, policyRecommendation) are optional. Only apiServer is required. Set any unwanted features to false.

kubectl apply -f ./addon-config.yaml

apiVersion: addons.kubernetes.vmware.com/v1alpha1
kind: AddonConfig
metadata:
  name: <cluster-name>-calico-enterprise   # The AddonConfig name has to be in this format eg. cluster1-calico-enterprise
  namespace: <cluster-namespace>
spec:
  values:
    helmValues:
      # Every option from tigera-helm-charts/tigera-operator/values.yaml may be
      # set here. Anything not specified falls back to the default declared in
      # the AddonConfigDefinition schema. The fields below are illustrative;
      installation:
        calicoWindowsDataplane:
          enabled: true
      policyRecommendation:
        enabled: false

note

The Addon framework expects the AddonConfig name to use the format <cluster-name>-<addon-name>. If the name does not match this format, the framework ignores the AddonConfig and creates a new one with default values instead.

Step 2 (optional): Apply the Custom AddonInstall resource

A custom AddonInstall resource is needed only if you want to control which clusters get upgraded. It lets you upgrade clusters selectively. Use spec.clusters.selector.matchLabels to specify the labels that a cluster must have, and then add those labels to the Cluster resources you want to upgrade.

kubectl apply -f ./addon-install.yaml

apiVersion: addons.kubernetes.vmware.com/v1alpha1
kind: AddonInstall 
metadata: 
  labels:
    addon.kubernetes.vmware.com/addon-name: calico-enterprise 
    addon.kubernetes.vmware.com/addon-namespace: vmware-system-vks-public
    addons.kubernetes.vmware.com/category: cni
  name: <addon-install-name>
  namespace: <cluster-namespace>
spec: 
  addonRef:
    name: calico-enterprise
    namespace: vmware-system-vks-public 
  clusters:
  - selector: 
      matchLabels:
        <AddonInstallLabelKey>: <AddonInstallLabelValue>      # Eg. cluster-group: calico-group-1
    constraints:
      expression: cluster.cniRefName() = "calico-enterprise" && cluster.kubeProxyConfiguration().enabled == true
  releaseFilter:
    ref:
      name: calico-enterprise.tigera.io.3.23.3
  stopMatchingBehavior: Retain

Step 3: Apply the Cluster Resource

The following example creates a cluster with Linux and Windows node pools. To create a Linux-only cluster, remove the Windows node pool (windows-pool).

Update the following fields before applying:

metadata.name and metadata.namespace
spec.topology.variables[].value.cniRef (name and namespace of the Calico Enterprise addon)
If you applied a custom AddonInstall in Step 2, add its label (<AddonInstallLabelKey>: <AddonInstallLabelValue>) to metadata.labels.
Node pool configuration (VM class, storage class, replicas) as needed for your environment

kubectl apply -f ./cluster.yaml

apiVersion: cluster.x-k8s.io/v1beta2
kind: Cluster
metadata:
  name: <cluster-name>
  namespace: <cluster-namespace>
spec:
  clusterNetwork:
    pods: {cidrBlocks: [100.96.0.0/11]}
    services: {cidrBlocks: [100.64.0.0/13]} # required for Windows (becomes serviceCIDRs)
    serviceDomain: cluster.local
  topology:
    classRef: {name: builtin-generic-v3.8.0, namespace: vmware-system-vks-public}
    version: v1.36.4+vmware.2-vkr.3
    controlPlane:
      replicas: 1
      metadata: &ubuntu
        annotations: {run.tanzu.vmware.com/resolve-os-image: "os-name=ubuntu,os-version=24.04"}
    workers:
      machineDeployments:
      - {class: node-pool, name: linux-pool-1, replicas: 3, metadata: *ubuntu,
          variables: {overrides: [{name: vmClass, value: best-effort-large}]}}
      - {class: node-pool, name: linux-pool-2, replicas: 2, metadata: *ubuntu}
      - class: node-pool
        name: windows-pool
        replicas: 2
        metadata:
          annotations: {run.tanzu.vmware.com/resolve-os-image: "os-name=windows,os-version=2022"}
        variables: {overrides: [{name: vmClass, value: best-effort-xlarge}]}
    variables:
    - {name: bootstrapAddons, value: {cniRef: {name: calico-enterprise, namespace: vmware-system-vks-public}}}
    - {name: vmClass, value: best-effort-xlarge}
    - {name: storageClass,  value: vsan-esa-default-policy-raid5}
    - {name: vsphereOptions, value: {persistentVolumes: {defaultStorageClass: vsan-esa-default-policy-raid5}}}
    - {name: kubernetes, value: {certificateRotation: {enabled: true, renewalDaysBeforeExpiry: 90}}}
    # Must be cluster-level: the Calico ACD only reads spec.topology.variables.
    - {name: osConfiguration, value: {windowsToolsConfig: {excludeNICs: ["cali*"]}}}
    - name: node
      value:
        firewall:
          inboundRules:
          - {protocol: tcp, fromPort: 179, toPort: 179}       # BGP
          - {protocol: udp, fromPort: 4789, toPort: 4789}     # VXLAN
          - {protocol: tcp, fromPort: 5473, toPort: 5473}     # Typha
          - {protocol: udp, fromPort: 51820, toPort: 51820}   # WireGuard
          - {protocol: tcp, fromPort: 8444, toPort: 8444}     # Linseed
          - {protocol: tcp, fromPort: 9090, toPort: 9090}     # Prometheus
          - {protocol: tcp, fromPort: 9200, toPort: 9200}     # Elasticsearch
          - {protocol: tcp, fromPort: 9300, toPort: 9300}     # Elasticsearch transport
          - {protocol: tcp, fromPort: 9443, toPort: 9443}     # Manager UI

Step 4: Retrieve the Guest Cluster Kubeconfig

Can be copied from the Supervisor.

Step 5: Wait for Core Components

Monitor the Tigera status until calico and apiserver report as Available:

kubectl get tigerastatus

At this point, all nodes should be in Ready state.

Step 6 (Windows only): Configure Windows nodes

Skip this step for Linux-only clusters.

After apiserver reports as Available, set strictAffinity to true in the IPAM configuration:

kubectl patch ipamconfigurations default --type merge -p '{"spec":{"strictAffinity":true}}'

If the IPAM configuration does not exist yet, create it:

kubectl apply -f - <<EOF
apiVersion: projectcalico.org/v3
kind: IPAMConfiguration
metadata: {name: default}
spec: {strictAffinity: true}
EOF

Install kube-proxy for Windows:
```
curl -L https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/calico/kube-proxy/kube-proxy.yml | sed "s/KUBE_PROXY_VERSION/<YOUR_KUBERNETES_VERSION>/g" | kubectl apply -f -
```
Replace <YOUR_KUBERNETES_VERSION> with the Kubernetes version of the guest cluster, for example v1.36.4.

💡 Tip: The K8s version can be retrieved with kubectl version -o json | jq -r '.serverVersion.gitVersion'

ℹ️ Note: If the version returned is of the form v1.36.4+vmware.2, strip the +vmware.2 suffix unless a VMware-provided kube-proxy image exists in the sigwindowstools/kube-proxy repository. ℹ️ Note: For Air-gapped environments copy the kube-proxy.yaml to the air-gapped environment and make the changes.

Step 7: Apply the Tigera License

kubectl apply -f tigera-license.yaml

Step 8: Verify the Installation

Wait for all remaining components to become available:

kubectl get tigerastatus
kubectl get pods -A | grep -E 'tigera|calico'

Once all items in tigerastatus show Available, the installation is complete. You can now deploy workloads to validate networking.

Air-gapped environment

Air-gapped package registry

If the published bundles need to be moved into a private OCI registry (for example, an air-gapped install), follow these steps.

Air-gapped relocation prerequisites

imgpkg and yq available locally.
Network access to the source registry (or a tarball produced from it) and to the private registry.
Credentials to push to the private registry (imgpkg will prompt or use the local docker config).

Step 1: Copy the Supervisor Service (SVS) bundles to a tarball (source side)

Run on a host that has access to the source registry:

imgpkg copy --bundle <source-registry>/calico-enterprise-svs:v3.23.3  --to-tar calico-enterprise-svs.tar

tip

Direct registry-to-registry relocation is also supported. Replace --to-tar <file>.tar with --to-repo <private-registry>/<path> and skip Step 2.

Step 2: Push the SVS bundle into the private registry (destination side)

imgpkg copy --tar calico-enterprise-svs.tar --to-repo <customer-registry>/calico-enterprise-svs

Step 3: Patch Package Resource in `calico-enterprise-svs.yaml`

The only registry reference outside the bundles is the service-bundle pointer in the Package CR inside calico-enterprise-svs.yaml. This needs to be updated to the private registry:

yq eval '
  (select(.kind == "Package") | .spec.template.spec.fetch[0].imgpkgBundle.image)
    = "<private-registry>/calico-enterprise-svs:v3.23.3"
' -i calico-enterprise-svs.yaml

tip

Content-pin to digest (optional): imgpkg copy --to-repo prints the destination digest of the relocated bundle on its last line. Substituting :v3.23.3 with @sha256:<that-digest> in the yq command above will make the reference immutable.

Step 4: Upload `calico-enterprise-svs.yaml` i.e. Package and PackageMetadata resource to the Supervisor

Upload the new Package and PackageMetadata to the vSphere Client - Supervisor Management as in Part 1, 2 and continue with the remaining steps to create a guest cluster.

What does not work

docker pull / docker push on individual images. Re-tagging can change manifest digests, and the bundles' images.yml lock files are not rewritten — kapp-controller fetches will fail with manifest unknown.
Not Copying the svs bundle during imgpkg copy run. The other 2 bundles are pulled by the svs package.

Air-gapped private container registry: Relocate Calico Enterprise private images to a private registry

Follow the steps in Push Calico Enterprise images to your private registry to pull Calico Enterprise images from Tigera's private image registry to your private registry.

note

Only steps 1, 2, 3, 4 — i.e. docker pull, docker tag and docker push for Linux only need to be performed.

caution

Do not push Calico Enterprise private images to a public registry.

Next steps

Recommended

Recommended - Networking

For all networking options, see Determine best networking option.

Big picture​

Before you begin​

Networking​

How to​

Part 1: Set up the Supervisor Service (one time cluster admin setup)​

Part 2: Deploy a guest cluster with Calico Enterprise​

Guest cluster prerequisites​

Step 1 (optional): Apply the AddonConfig​

Step 2 (optional): Apply the Custom AddonInstall resource​

Step 3: Apply the Cluster Resource​

Step 4: Retrieve the Guest Cluster Kubeconfig​

Step 5: Wait for Core Components​

Step 6 (Windows only): Configure Windows nodes​

Step 7: Apply the Tigera License​

Step 8: Verify the Installation​

Air-gapped environment​

Air-gapped package registry​

Air-gapped relocation prerequisites​

Step 1: Copy the Supervisor Service (SVS) bundles to a tarball (source side)​

Step 2: Push the SVS bundle into the private registry (destination side)​

Step 3: Patch Package Resource in calico-enterprise-svs.yaml​

Step 4: Upload calico-enterprise-svs.yaml i.e. Package and PackageMetadata resource to the Supervisor​

What does not work​

Air-gapped private container registry: Relocate Calico Enterprise private images to a private registry​

Next steps​

Big picture

Before you begin

Networking

How to

Part 1: Set up the Supervisor Service (one time cluster admin setup)

Part 2: Deploy a guest cluster with Calico Enterprise

Guest cluster prerequisites

Step 1 (optional): Apply the AddonConfig

Step 2 (optional): Apply the Custom AddonInstall resource

Step 3: Apply the Cluster Resource

Step 4: Retrieve the Guest Cluster Kubeconfig

Step 5: Wait for Core Components

Step 6 (Windows only): Configure Windows nodes

Step 7: Apply the Tigera License

Step 8: Verify the Installation

Air-gapped environment

Air-gapped package registry

Air-gapped relocation prerequisites

Step 1: Copy the Supervisor Service (SVS) bundles to a tarball (source side)

Step 2: Push the SVS bundle into the private registry (destination side)

Step 3: Patch Package Resource in `calico-enterprise-svs.yaml`

Step 4: Upload `calico-enterprise-svs.yaml` i.e. Package and PackageMetadata resource to the Supervisor

What does not work

Air-gapped private container registry: Relocate Calico Enterprise private images to a private registry

Next steps