3.4.4

(2021-07-14)

Components

Component

Version

MKE

3.4.4

Kubernetes

1.20.7

Calico

3.19.0

Calico for Windows

3.19.0

Interlock

3.2.3

Interlock NGINX proxy

1.19.9

Istio Ingress

1.4.10

CoreDNS

1.7.0

RethinkDB

2.3.6

etcd

3.4.15

CSI Attacher

2.1.1

CSI Provisioner

1.4.0

CSI Snapshotter

1.2.2

CSI Resizer

0.4.0

CSI Node Driver Registrar

1.2.0

CSI Liveness Probe

1.1.0

What’s new

  • Added a beta version of a light-weight swarm-only mode for MKE, which supports only Swarm orchestration (MKE-8055).

  • MKE now supports IPVS proxier for kube-proxy, which offers the widest choice of load balancing algorithms and superior scalability compared to other kube-proxy modes (MKE-8088).

    Learn more

    Configure IPVS

  • Updated Kubernetes to version 1.20.7.

  • Updated Calico to version 3.19.0.

  • MKE now tags all analytics reports with the user license ID when telemetry is enabled. It does not, though, collect any further identifying information. In line with this change, the MKE configuration no longer contains the anonymize_tracking setting, and the MKE web UI no longer includes the Make data anonymous toggle (MKE-8316).

  • MKE no longer exposes the Interlock NGINX ServerNamesHashBucketSize setting. The setting was confusing users because MKE adaptively calculates the setting and overrides any manual input (MKE-8306).

  • Added the kubelet_pods_per_core setting to the cluster_config section of the MKE configuration file. Users can now set the maximum number of pods per core with this setting (MKE-8273).

  • Updated the version of MCR that ships with MKE to MCR 20.10.4 (FIELD-3667).

  • Improved MKE controller memory usage due to the high-load MKE database (FIELD-3540).

  • Improved the MKE database query performance for role-based access control (RBAC) information (FIELD-3540).

  • Added the authz_cache_timeout setting to the MKE configuration, which allows the caching of role-based access control (RBAC) information for non-Kubernetes MKE resource listing APIs. When enabled, this setting improves API performance and reduces the MKE database load. MKE does not enable the cache by default (FIELD-3540).

  • SAML now uses SHA-256 signatures rather than SHA-1 (FIELD-3902).

  • FELIX_LOGSEVERITYSCREEN can now adhere to a greater number of MKE log verbosity levels resulting in less log content when users do not want debug or error information (FIELD-2673).

  • At the start of the upgrade process, MKE now verifies the accessibility of the required ports on Linux nodes (MKE-7996).

Bug fixes

  • Fixed an issue wherein previously-rejected MKE component tasks caused MKE upgrades to fail (FIELD-4032).

  • Fixed an issue wherein failure to unpack the kubelet image to containerd caused the kubelet component to fail (FIELD-3939).

  • Fixed an issue with the MKE web UI wherein building an MSR setup command in Admin Settings failed to apply the selected node (FIELD-2475).

  • Fixed an issue wherein users could not create a customresourcedefinitions Kubernetes type (a type that is not represented in the web UI) (FIELD-2757).

  • Fixed an issue in the MKE web UI wherein back-up settings failed to save when the user selected ENCRYPT BACKUP without first deleting text entered in the Passphrase text box (FIELD-2603).

  • Restored the number of lines captured in the support dump logs to 10,000 from the original dsinfo.sh script (FIELD-4033).

  • Fixed an issue wherein adding manager nodes could result in an unhealthy etcd status (MKE-8367).

  • Reverted a recent fix that caused support dumps to fail and nodes to disconnect (FIELD-4011).

Known issues

  • The Kubernetes secure overlay network encryption solution does not encrypt traffic with the install-time default VXLAN dataplane that ships with MKE 3.3.0 and later (MKE-7779).

    Workaround:

    To use the secure overlay solution with MKE 3.3.0 and later, install MKE with the IPIP dataplane using the --secure-overlay and --calico-vxlan=false options.

    Note

    The issue does not affect existing environments upgraded from MKE versions older than 3.3.0, which remain on the IPIP dataplane.

  • Due to potential port conflicts between kubectl and NodePort, it may not be possible to use kubectl where a NodePort is established throughout the cluster (FIELD-3495).

    Workaround:

    Reconfigure the ephemeral port range on each container host to avoid overlapping ports:

    1. Create the file /etc/sysctl.d/kubelet_ephemeral_port.conf:

      net.ipv4.ip_local_port_range=35536 60999
      
    2. Load the change for the current boot:

      sudo sysctl -p /etc/sysctl.d/kubelet_ephemeral_port.conf
      
    3. Restart kubelet:

      docker restart ucp-kubelet
      

    Wherever possible, Mirantis recommends that you put the Kubernetes node that you plan to restart into drain status, which thereby migrates running pods to other nodes. In the event that the kubelet restart lasts longer than five minutes, this migration will minimize the potential impact on those services.

    Undertake any restart of the kubelet on a manager node with care, as this action will impact the services and API of any Kubernetes system pod that restarts concurrently, until the manager node kubelet operates normally.

    Note that this workaround may not be a viable option in a production environment, as restarting the kubelet can result in any of the following:

    • If the restart takes longer than five minutes, Kubernetes will stop all of the pods running on the node and attempt to start them on a different node.

    • Pod or service health checks can fail during the restart.

    • Kubernetes system metrics may fail or be inaccurately reported until the restart is complete.

    • If the restart takes too long or fails, Kubernetes may designate the node as unhealthy. This can result in Kubernetes removing the node from the orchestrating pods until it redesignates the node as healthy.

  • The upgrade from MKE 3.4.2 to 3.4.4 leaves Windows nodes on Kubernetes in a persistent non-ready state in which kubelet enters an endless start-exit loop. Swarm nodes are also impacted when they are switched to Kubernetes following an upgrade to 3.4.4 (MKE-8431).

    Workaround:

    After upgrading to MKE 3.4.4, perform the following steps on a manager node:

    1. Get the current configuration in a TOML file from the appropriate endpoint on an MKE manager node:

      AUTHTOKEN=$(curl -sk -d '{"username":"<username>","password":"<password>"}' \
      https://${nodeIP}/auth/login | jq -r .auth_token) && curl -sk -X \
      GET "https://${nodeIP}/api/ucp/config-toml" -H  \
      "accept: application/toml" -H  "Authorization: Bearer $AUTHTOKEN" > config.toml
      
    2. Write back the captured configuration and wait for one minute:

      AUTHTOKEN=$(curl -sk -d '{"username":"<username>","password":"<password>"}' \
      https://${nodeIP}/auth/login | jq -r .auth_token) && curl -sk -X \
      PUT -H  "accept: application/toml" -H "Authorization: Bearer $AUTHTOKEN" \
      --upload-file config.toml https://${nodeIP}/api/ucp/config-toml
      
    3. Delete the ucp-kubelet-win service:

      docker service rm ucp-kubelet-win
      
    4. Wait for the impacted worker nodes to enter a ready state.

  • After upgrading to MKE 3.4.0 or later, the strict affinity setting is enabled for Calico CNI and cannot be disabled. This can impact networking functionality in large Kubernetes clusters with a limited private IP space allocated for pods using the --pod-cidr MKE install flag.

    Mirantis strongly recommends that impacted customers wait to upgrade until this issue is resolved in an upcoming release (FIELD-4182).