Migrate container runtime from Docker to containerd

Available since 2.28.4 (Cluster releases 17.3.4 and 16.3.4)

Caution

Due to the known issue 49678, which is resolved in MOSK management 2.29.0 after the management cluster update to the Cluster release 16.4.0, the HostOSConfiguration object may not work as expected after migration to containerd. For details, see the issue description.

Migration of container runtime from Docker to containerd is implemented for existing management and MOSK clusters. The use of containerd allows for better Kubernetes performance and component update without pod restart when applying fixes for CVEs.

Note

On greenfield deployments, containerd is the default container runtime since MOSK management 2.29.0 and MOSK 25.1. Before that, Docker remains the default option.

Precautions

Before the container runtime mirgation, consider the following precautions:

The migration involves machine cordoning and draining.
Cluster update is not allowed during migration to prevent machines from running different container runtimes. However, you can still scale clusters and replace nodes as required.
The migration is mandatory during the scope of MOSK management 2.29.x. The management cluster update to MOSK management 2.30.0 will be blocked on clusters still running Docker as the container runtime.

Note

If you have not upgraded the operating system distribution on your machines to Jammy yet, Mirantis recommends migrating machines from Docker to containerd on MOSK clusters together with distribution upgrade to minimize the maintenance window.

In this case, ensure that all cluster machines are updated at once during the same maintenance window to prevent machines from running different container runtimes.

Upgrade container runtime between releases

The following procedure applies to both management and MOSK clusters.

Verify that the MOSK cluster is updated to the Cluster release 17.3.4 or later as described in Cluster update.

Note

Management clusters must be updated to the Cluster release 16.3.4 or later.
Verify that the MOSK cluster distribution is upgraded to Ubuntu Jammy as described in Upgrade an operating system distribution.
Open the required Machine object for editing.

You can schedule more than one machine for migration at the same time. In this case, the process is automatically orchestrated without service interruption.
In the metadata.annotations section, add the following annotation to trigger migration to containerd runtime:
```
apiVersion: cluster.k8s.io/v1alpha1
kind: Machine
metadata:
  name: example-machine
  namespace: example-ns
  annotations:
    kaas.mirantis.com/preferred-container-runtime: containerd
```
The machine will be cordoned and drained, and container runtime will be migrated.

For description of the Machine object fields, see Machine resource.
Once migration is completed, verify that ContainerRuntimeContainerd condition is true in status.providerStatus.

For description of the Machine object status fields, see Machine status.
Repeat the procedure with the remaining machines.
Strongly recommended. Back up MKE as described in Mirantis Kubernetes Engine documentation: Back up MKE.

Since the procedure above modifies the cluster configuration, a fresh backup is required to restore the cluster in case further reconfigurations fail.
Important

Because the MKE restoration process is complicated, we strongly recommend contacting Mirantis support for assistance.

If you still decide to restore MKE from a backup on your own, you must scale down helm-controller on the cluster being restored if the MKE version of the affected cluster after the restore will differ from the MKE version in the ClusterRelease object that is set in MOSK Cluster objects in the management cluster:
- If you are restoring MKE on a management cluster: before starting the restore, scale down helm-controller on each affected MOSK cluster. This prevents unintended Ceph and OpenStack downgrades on MOSK clusters after the management cluster is restored.
- If you are restoring MKE on a MOSK cluster: immediately after the restore completes, scale down helm-controller. Because the restore rolls the cluster back to an older release, this prevents it from triggering a premature upgrade of Helm releases.

Caution

If an emergency related to containerd occurs on workloads before migration is complete on all machines, you can temporarily roll back containerd to Docker. Use the procedure above by changing the kaas.mirantis.com/preferred-container-runtime annotation from containerd to docker.