Upgrade Prerequisites#

Verify that you have the following components in place before you begin upgrading MKE 3 to MKE 4:

A MKE 3.9 cluster.

Note

Only MKE 3.9.x clusters can be upgraded to MKE 4.2.0 and later. For more information, refer to Upgrade from MKE 3.7, 3.8, or 3.9.
A backup of the MKE cluster. For comprehensive instruction on how to create an MKE 3 back up, refer to Back up MKE.
The latest MKE 4 CLI (mkectl) binary, installed on your local environment:
```
mkectl version
```
Example output:
```
Version: v4.2.0
```
A hosts.yaml file, prepared as follows:
```
hosts:
  - address: <host1-external-ip>
    port: <ssh-port>
    user: <ssh-user>
    keyPath: <path-to-ssh-key>
    # privateAddress: <node-internal-ip>  # optional, sets kubelet's --node-ip
    # privateInterface: <node-network-interface>  # optional, mutually exclusive with privateAddress
                                                  # when set, used to determine privateAddress
  - address: <host2-external-ip>
    port: <ssh-port>
    user: <ssh-user>
    keyPath: <path-to-ssh-key>
```
Important
- Confirm that all nodes are accessible via SSH, using the specified credentials.
- Confirm Docker user group membership, which is docker on Linux and docker-users on Windows. This allows non-administrative users to run Docker commands and thus interact with the Docker daemon without the need for sudo or administrator privileges.
- Use either external IP addresses or FQDNs. These must be resolvable and reachable from the system from which you are running mkectl. Mirantis recommends that you use a load balancer is recommended for high-availability setups.
- It is not necessary to declare a node role in the hosts.yaml file as they are automatically detected during upgrade.
Specific to Windows worker nodes

Before you upgrade an MKE 3 cluster with Windows worker nodes, ensure that:
- The Windows workers nodes are running Windows Server 2022.
- OpenSSH Server is installed, running, and set to start automatically on each Windows worker. mkectl connects to Windows nodes over SSH, as it does for Linux nodes, and thus the sshd service must be available throughout the upgrade.
  
  To install and enable OpenSSH Server:
  1. Install the OpenSSH Server feature:
    Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0
  2. Start the service and set it to start automatically at bootup:
    Start-Service sshd Set-Service -Name sshd -StartupType 'Automatic'
  3. Confirm that OpenSSH is listening:
    Get-Service sshd
  4. Confirm that the Windows worker is reachable from the machine running mkectl using the credentials listed in hosts.yaml:
    ssh -i <path-to-ssh-key> <ssh-user>@<windows-node-address>
- Linux node requirements are met. Windows nodes can only serve as worker nodes, thus control plane and manager nodes must be on Linux.
- You need at least one Linux manager node and one Linux worker to migrate an MKE 3 cluster to MKE 4.
- The cluster must have enough Linux nodes to host the Calico Typha component, which the Tigera operator scales automatically based on total node count (including Windows nodes).
  
  Total nodes (Linux + Windows) Minimum Linux nodes required
  
  <5 2 (the manager + worker above)
  
  5 or more 3
  
  400+ 4, plus 1 more per additional 200 nodes
The external address set in the apiServer.externalAddress field of the mke4.yaml configuration file. The external address is the domain name of the load balancer configured as described in System Requirements: Load balancer.

Calico migration to Kubernetes Datastore Driver (KDD) from etcd:

Warning

To upgrade successfully to MKE 4, the source MKE 3 cluster must be configured to use KDD.

Important

Before you attempt to enable KDD, Mirantis strongly suggests that you make a backup, as you cannot go back to non-KDD/etcd from KDD or recover from a failure in moving to KDD without rolling the cluster back to a known good state in which it was using an etcdv3 datastore.
1. Obtain the MKE 3 configuration file:
```
export MKE_USERNAME=<mke-username>
export MKE_PASSWORD=<mke-password>
export MKE_HOST=<mke-fqdn-or-ip-address>
AUTHTOKEN=$(curl --silent --insecure --data '{"username":"'$MKE_USERNAME'","password":"'$MKE_PASSWORD'"}' https://$MKE_HOST/auth/login | jq --raw-output .auth_token)
curl --silent --insecure -X GET "https://$MKE_HOST/api/ucp/config-toml" -H "accept: application/toml" -H "Authorization: Bearer $AUTHTOKEN" > mke-config.toml
```
2. In the cluster_config section of the MKE 3 configuration file, check the setting of the calico_kdd parameter. If it is set to true, skip the remaining steps. Otherwise, edit the setting to true.
3. Apply the modified MKE 3 configuration file:
```
$ curl --silent --insecure -X PUT -H "accept: application/toml" -H "Authorization: Bearer $AUTHTOKEN" --upload-file  'mke-config.toml' https://$MKE_HOST/api/ucp/config-toml
```
  On completion, the following confirmation displays:
```
{"message":"Calico datastore upgrade from etcd to kdd successful"}
```
Important
- The conversion of the Calico datastore from etcd to KDD typically takes about 20 seconds per node, depending on the size of the cluster.
- According to Tigera, the conversion to KDD freezes cluster networking, and thus new or replacement pods are not able to start. Existing workloads, however, continue to run and their network connectivity is not impacted.
- The steps above must be completed as a standalone procedure before beginning the MKE 4 upgrade process. The upgrade itself will be covered in the following sections.
- If your MKE 3 deployment uses an unmanaged CNI, this upgrade path is not currently supported.
- You must confirm the success of the move to KDD. Though KDD enablement can take some time, on successful completion you will receive a message that is similar to Calico datastore upgrade from etcd to KDD successful. If this message does not display, you cannot upgrade the cluster to MKE 4 because if there is an error with KDD migration, the datastore may not be unlocked. In such a case, you must use your backup to rollback to a known good state in which your cluster was using an etcdv3 datastore.

Total nodes (Linux + Windows)	Minimum Linux nodes required
<5	2 (the manager + worker above)
5 or more	3
400+	4, plus 1 more per additional 200 nodes