3.3.16

(2022-04-05)

Components

Component

Version

MKE

3.3.16

Kubernetes

1.18.19

Calico

3.18.4

Calico for Windows

3.18.4

Interlock

3.3.3

Interlock NGINX proxy

1.21.1

Istio Ingress

1.4.10

CoreDNS

1.7.0

RethinkDB

2.3.6

etcd

3.4.3

CSI Attacher

2.1.1

CSI Provisioner

1.4.0

CSI Snapshotter

1.2.2

CSI Resizer

0.4.0

CSI Node Driver Registrar

1.2.0

CSI Liveness Probe

1.1.0

Openstack Cinder CSI plugin

1.20.3

What’s new

  • [MKE-8692] The Interlock proxy NGINX can now be run in debugging mode, which you must enable manually. Mirantis recommends that you not use debugging mode in production environments.

Bug fixes

  • [FIELD-4691] Fixed an issue with the backup and restore processes wherein the default timeout duration was not long enough to accommodate certain use cases. The default timeout duration is increased to 30 minutes and you can now customize the duration by using the optional --timeout flag with the backup and restore commands.

  • [FIELD-4629] Fixed an issue wherein there was excessive white space in the Interlock config template.

  • [FIELD-4623] Fixed an issue with the MKE web UI wherein the <user name> > Admin Settings > Upgrade page did not always report accurate upgrade options information.

  • [FIELD-4572] Fixed an issue with the MKE API wherein the output of GET /containers/json requests did not filter the running containers.

  • [FIELD-4567] Fixed an issue wherein attempts to sync LDAP groups aborted when the group was empty.

  • [FIELD-4459] Fixed an issue wherein licenses that are invalid or removed caused all custom collections to reset to the default collection.

  • [FIELD-4407] Fixed an issue wherein specifying a URI SAN using the --san flag with the install command caused the installation to fail.

  • [FIELD-4317] Fixed an issue with the MKE web UI wherein navigating to Dashboard > Manage Users & Teams and clicking the manually creating a user account link produced a blank page.

  • [FIELD-4713] Improved the performance of the MKE API for docker build commands.

  • [FIELD-4684] Added a banner warning to the MKE web UI concerning the expiration of client root CA certificates.

  • [FIELD-4684] The UCP client root CA certificate lifetime for new MKE clusters is now 20 years, extended from the previous 5-year lifetime.

  • [MKE-8538] Added documentation that enables the downloading of a limited support bundle on Windows nodes. Refer to Use PowerShell to obtain a support bundle for more information.

Known issues

  • [FIELD-4200] The calico-node firewalld-policy init container can disable the docker ingress routing mesh when reloading firewalld.

    Workaround:

    1. Prevent the issue from recurring by disabling firewalld:

      sudo systemctl disable --now firewalld
      
    2. Restore missing iptables chains by restarting dockerd:

      sudo systemctl restart docker
      

      Note

      Restarting dockerd stops all containers on the corresponding node. The node capacity will not be available to the cluster until the node returns to a healthy state in MKE. You must restart dockerd on manager nodes one node at a time, confirming the health of each one in MKE before moving on to the next.

    3. Confirm issue resolution by checking for the presence of the DOCKER-INGRESS iptables chain:

      sudo iptables --list DOCKER-INGRESS
      

      Expected output:

      Chain DOCKER-INGRESS (2 references)
      target     prot opt source               destination
      [...]
      
  • [MKE-8538] Only limited support bundles are available on Windows worker nodes.

    Workaround:

    Manually collect the Windows worker node logs.

  • [MKE-8738] Windows Kubernetes worker nodes can fail on long-haul runs, with a DiskPressure error that is similar to the following:

    time="2022-02-08T17:20:30Z" level=warning msg="error while removing container: failed to unprepare layer C:\\ProgramData\\containerd\\root\\io.containerd.snapshotter.v1.windows\\snapshots\\3707: hcsshim::UnprepareLayer - failed failed in Win32: The system could not find the instance specified. (0x801f0015): unknown"
    time="2022-02-08T17:20:30Z" level=fatal msg="failed to cleanup old containers: failed to unprepare layer C:\\ProgramData\\containerd\\root\\io.containerd.snapshotter.v1.windows\\snapshots\\3707: hcsshim::UnprepareLayer - failed failed in Win32: The system could not find the instance specified. (0x801f0015): unknown"
    

    Workaround:

    1. Identify the stopped task:

      C:\Users\Docker>ctr -n com.docker.ucp task ls
      

      Example output:

      TASK                  PID      STATUS
      ucp-tigera-felix      12012    RUNNING
      ucp-kube-proxy        7912     RUNNING
      ucp-kubelet-health    26616    STOPPED
      ucp-tigera-node       3236     RUNNING
      
    2. Identify the containerd-shim process that is associated with the stopped task:

      Get-CimInstance -ClassName Win32_Process \
      -Filter "Name like 'containerd-shim%'" | \
      select ProcessId,CommandLine | fl
      
    3. Stop the containerd-shim process that is associated with the stopped task:

      Stop-Process -Id <containerd-shim-pid> -Confirm -PassThru -Force