Skip to content

CA rotation impact#

Review the impact of each rotation before you choose which CAs to rotate:

Rotation                                  Requires worker node rejoin Affects child clusters Additional impact
Kubernetes CA Yes, all worker nodes Yes Invalidates all user kubeconfig files
ServiceAccount key pair Yes, all worker nodes Yes Invalidates all ServiceAccount tokens in the cluster
etcd CA No No Affects controller nodes only. Shortest downtime of the four
front-proxy CA No No Affects controller nodes only

User kubeconfig files#

The Kubernetes CA is the trust anchor in every kubeconfig file. After the rotation, every user must obtain a new kubeconfig file. The mkectl kubeconfig command prints to stdout, so each user must redirect the output to their kubeconfig file:

mkectl kubeconfig > mke.kubeconf.new
mv mke.kubeconf.new ~/.mke/mke.kubeconf

Before you begin the rotation, plan the redistribution of kubeconfig files, including those held by CI systems, dashboards, and other automation.

ServiceAccount tokens#

Rotating the pki/sa.{key,pub} key pair invalidates every ServiceAccount token in the cluster:

  • Pods that use projected tokens recover automatically, as the kubelet requests new tokens.
  • Workloads that use legacy token Secrets do not recover automatically, as legacy tokens are signed once and never re-signed.

After the rotation, identify the legacy token Secrets:

kubectl get secrets --all-namespaces --field-selector type=kubernetes.io/service-account-token

Delete each legacy token Secret so that the token controller creates a new token. Restart any workload that reads its token from a Secret rather than from a projected volume.

MKE 4 child clusters#

Warning

Child clusters are affected by the rotation and do not recover automatically.

Child control plane nodes hold credentials that the parent cluster issues. Rotating the ServiceAccount key pair of the parent cluster invalidates the ServiceAccount token that child nodes use to pull the k0s and etcd maintenance binaries from the parent cluster. In an air-gapped environment, the parent cluster is the only source of these binaries. Recreating the token Secret on the parent cluster does not propagate the new token to child nodes that hold the old one.

Existing child nodes continue to run. The failure occurs the next time that a child node is provisioned, replaced, scaled out, or rebooted.

Child nodes provisioned before the parent cluster was upgraded to MKE 4.2.0 may hold a copy of the parent cluster Kubernetes CA, which they use to verify the binary pull. Before you rotate the parent cluster Kubernetes CA, verify that the child cluster machines use the current bootstrap configuration.

If you run child clusters, contact Mirantis Support before you rotate the Kubernetes CA or the ServiceAccount key pair.