Install an unmanaged CNI plugin¶

Calico affords MKE secure networking functionality for container-to-container communication within Kubernetes. MKE manages the Calico lifecycle, packaging it at both the time of installation and upgrade, and fully supports its use with MKE

MKE also supports the use of alternative, unmanaged CNI plugins available on Docker Hub. Mirantis can provide limited instruction on basic configuration, but for detailed guidance on third-party CNI components, you must refer to the external product documentation or support.

Consider the following limitations before implementing an unmanaged CNI plugin:

MKE only supports implementation of an unmanaged CNI plugin at install time.
MKE does not manage the version or configuration of alternative CNI plugins.
MKE does not upgrade or reconfigure alternative CNI plugins. To switch from the managed CNI to an unmanaged CNI plugin, or vice versa, you must uninstall and then reinstall MKE.

Install an unmanaged CNI plugin on MKE¶

Verify that your system meets all MKE requirements and third-party CNI plugin requirements.
Install MKE with the --unmanaged-cni flag:
```
docker container run --rm -it --name ucp \
  -v /var/run/docker.sock:/var/run/docker.sock \
  mirantis/ucp:3.7.17 install \
  --host-address <node-ip-address> \
  --unmanaged-cni \
  --interactive
```
MKE components that require Kubernetes networking will remain in the Container Creating state in Kubernetes until a CNI is installed. Once the installation is complete, you can access MKE from a web browser. Note that the manager node will be unhealthy as the kubelet will report NetworkPluginNotReady. Additionally, the metrics in the MKE dashboard will also be unavailable, as this runs in a Kubernetes pod.
Download and configure the client bundle.

Review the status of the MKE components that run on Kubernetes:

kubectl get nodes

Example output:

NAME         STATUS     ROLES     AGE       VERSION
manager-01   NotReady   master    10m       v1.11.9-docker-1

kubectl get pods -n kube-system -o wide

Example output:

NAME                           READY     STATUS              RESTARTS   AGE       IP        NODE         NOMINATED NODE
compose-565f7cf9ff-gq2gv       0/1       Pending             0          10m       <none>    <none>       <none>
compose-api-574d64f46f-r4c5g   0/1       Pending             0          10m       <none>    <none>       <none>
kube-dns-6d96c4d9c6-8jzv7      0/3       Pending             0          10m       <none>    <none>       <none>
ucp-metrics-nwt2z              0/3       ContainerCreating   0          10m       <none>    manager-01   <none>

Install the unmanaged CNI plugin. Follow the CNI plugin documentation for specific installation instructions. The unmanaged CNI plugin install steps typically include:
1. Download the relevant upstream CNI binaries.
2. Place the CNI binaries in /opt/cni/bin.
3. Download the relevant CNI plugin Kubernetes Manifest YAML file.
4. Run kubectl apply -f <your-custom-cni-plugin>.yaml.
Caution

You must install the unmanaged CNI immediately after installing MKE and before joining any manager or worker nodes to the cluster.

Note

While troubleshooting a custom CNI plugin, you may want to access logs within the kubelet. Connect to an MKE manager node and run docker logs ucp-kubelet.

Verify the MKE installation¶

Upon successful installation of the CNI plugin, the relevant MKE components will have a Running status once the pods have become available.

To review the status of the Kubernetes components:

kubectl get pods -n kube-system -o wide

Example output:

NAME                           READY     STATUS    RESTARTS   AGE       IP            NODE         NOMINATED NODE
compose-565f7cf9ff-gq2gv       1/1       Running   0          21m       10.32.0.2     manager-01   <none>
compose-api-574d64f46f-r4c5g   1/1       Running   0          21m       10.32.0.3     manager-01   <none>
kube-dns-6d96c4d9c6-8jzv7      3/3       Running   0          22m       10.32.0.5     manager-01   <none>
ucp-metrics-nwt2z              3/3       Running   0          22m       10.32.0.4     manager-01   <none>
weave-net-wgvcd                2/2       Running   0          8m        172.31.6.95   manager-01   <none>

Note

Weave Net serves as the CNI plugin for the above example. If you are using an alternative CNI plugin, verify its status in the output.

Enable an unmanaged CNI for Windows Server nodes¶

When MKE is installed with --unmanaged-cni, the ucp-kube-proxy-win container on Windows nodes will not fully start, but will instead log the following suggestion in a loop:

example : [System.Environment]::SetEnvironmentVariable("CNINetworkName", "ElangoNet", [System.EnvironmentVariableTarget]::Machine)
example : [System.Environment]::SetEnvironmentVariable("CNISourceVip", "192.32.31.1", [System.EnvironmentVariableTarget]::Machine)

This occurs because kube-proxy requires more information to program routes for Kubernetes services.

To enable an unmanaged CNI for Windows Server nodes:

There are two options for supplying kube-proxy with the required information.

Deploy your own kube-proxy along with the CNI, as implemented by the kube-proxy manifest and documented in the Kubernetes 1.21 Windows Install Guide.
If using a VXLAN-based CNI, define the following variables:
- CNINetworkName must match the name of the Windows Kubernetes HNS network, which you can find either in the installation documentation for the third party CNI or by using hnsdiag list networks.
- CNISourceVip must use the value of the source VIP for this node, which should be available in the installation documentation for the third party CNI. Because the source VIP will be different for each node and can change across host reboots, Mirantis recommends setting this variable using a utility script.
The following is an example of how to define these variables using PowerShell:
[System.Environment]::SetEnvironmentVariable("CNINetworkName", "vxlan0", [System.EnvironmentVariableTarget]::Machine) [System.Environment]::SetEnvironmentVariable("CNISourceVip", "192.32.31.1", [System.EnvironmentVariableTarget]::Machine)