Install an unmanaged CNI plugin¶
Calico affords MKE secure networking functionality for container-to-container communication within Kubernetes. MKE manages the Calico lifecycle, packaging it at both the time of installation and upgrade, and fully supports its use with MKE
MKE also supports the use of alternative, unmanaged CNI plugins available on Docker Hub. Mirantis can provide limited instruction on basic configuration, but for detailed guidance on third-party CNI components, you must refer to the external product documentation or support.
Consider the following limitations before implementing an unmanaged CNI plugin:
MKE only supports implementation of an unmanaged CNI plugin at install time.
MKE does not manage the version or configuration of alternative CNI plugins.
MKE does not upgrade or reconfigure alternative CNI plugins. To switch from the managed CNI to an unmanaged CNI plugin, or vice versa, you must uninstall and then reinstall MKE.
Install an unmanaged CNI plugin on MKE¶
Verify that your system meets all MKE requirements and third-party CNI plugin requirements.
Install MKE with the
--unmanaged-cni
flag:docker container run --rm -it --name ucp \ -v /var/run/docker.sock:/var/run/docker.sock \ mirantis/ucp:3.6.20 install \ --host-address <node-ip-address> \ --unmanaged-cni \ --interactive
MKE components that require Kubernetes networking will remain in the
Container Creating
state in Kubernetes until a CNI is installed. Once the installation is complete, you can access MKE from a web browser. Note that the manager node will be unhealthy as the kubelet will reportNetworkPluginNotReady
. Additionally, the metrics in the MKE dashboard will also be unavailable, as this runs in a Kubernetes pod.Review the status of the MKE components that run on Kubernetes:
kubectl get nodes
Example output:
NAME STATUS ROLES AGE VERSION manager-01 NotReady master 10m v1.11.9-docker-1
kubectl get pods -n kube-system -o wide
Example output:
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE compose-565f7cf9ff-gq2gv 0/1 Pending 0 10m <none> <none> <none> compose-api-574d64f46f-r4c5g 0/1 Pending 0 10m <none> <none> <none> kube-dns-6d96c4d9c6-8jzv7 0/3 Pending 0 10m <none> <none> <none> ucp-metrics-nwt2z 0/3 ContainerCreating 0 10m <none> manager-01 <none>
Install the unmanaged CNI plugin. Follow the CNI plugin documentation for specific installation instructions. The unmanaged CNI plugin install steps typically include:
Download the relevant upstream CNI binaries.
Place the CNI binaries in
/opt/cni/bin
.Download the relevant CNI plugin Kubernetes Manifest YAML file.
Run kubectl apply -f <your-custom-cni-plugin>.yaml.
Caution
You must install the unmanaged CNI immediately after installing MKE and before joining any manager or worker nodes to the cluster.
Note
While troubleshooting a custom CNI plugin, you may want to access logs within the kubelet. Connect to an MKE manager node and run
docker logs ucp-kubelet
.
Verify the MKE installation¶
Upon successful installation of the CNI plugin, the relevant MKE
components will have a Running
status once the pods have become
available.
To review the status of the Kubernetes components:
kubectl get pods -n kube-system -o wide
Example output:
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE
compose-565f7cf9ff-gq2gv 1/1 Running 0 21m 10.32.0.2 manager-01 <none>
compose-api-574d64f46f-r4c5g 1/1 Running 0 21m 10.32.0.3 manager-01 <none>
kube-dns-6d96c4d9c6-8jzv7 3/3 Running 0 22m 10.32.0.5 manager-01 <none>
ucp-metrics-nwt2z 3/3 Running 0 22m 10.32.0.4 manager-01 <none>
weave-net-wgvcd 2/2 Running 0 8m 172.31.6.95 manager-01 <none>
Note
Weave Net serves as the CNI plugin for the above example. If you are using an alternative CNI plugin, verify its status in the output.
Enable an unmanaged CNI for Windows Server nodes¶
When MKE is installed with --unmanaged-cni
, the ucp-kube-proxy-win
container on Windows nodes will not fully start, but will instead log the
following suggestion in a loop:
example : [System.Environment]::SetEnvironmentVariable("CNINetworkName", "ElangoNet", [System.EnvironmentVariableTarget]::Machine)
example : [System.Environment]::SetEnvironmentVariable("CNISourceVip", "192.32.31.1", [System.EnvironmentVariableTarget]::Machine)
This occurs because kube-proxy requires more information to program routes for Kubernetes services.
To enable an unmanaged CNI for Windows Server nodes:
There are two options for supplying kube-proxy with the required information.
Deploy your own kube-proxy along with the CNI, as implemented by the kube-proxy manifest and documented in the Kubernetes 1.21 Windows Install Guide.
If using a VXLAN-based CNI, define the following variables:
CNINetworkName
must match the name of the Windows Kubernetes HNS network, which you can find either in the installation documentation for the third party CNI or by using hnsdiag list networks.CNISourceVip
must use the value of the source VIP for this node, which should be available in the installation documentation for the third party CNI. Because the source VIP will be different for each node and can change across host reboots, Mirantis recommends setting this variable using a utility script.
The following is an example of how to define these variables using PowerShell:
[System.Environment]::SetEnvironmentVariable("CNINetworkName", "vxlan0", [System.EnvironmentVariableTarget]::Machine) [System.Environment]::SetEnvironmentVariable("CNISourceVip", "192.32.31.1", [System.EnvironmentVariableTarget]::Machine)