Use Multus CNI to create multi-homed Pods¶
Available since MKE 3.7.0
In Kubernetes, by default, a Pod is only connected to a single network interface, which is the default network. Using Multus CNI, however, you can create a multi-home Pod that has multiple network interfaces.
The following example procedure attaches two network interfaces to a Pod,
net1
and net2
.
Configure kubectl for your MKE cluster.
Enable Multus CNI in the MKE cluster when you install MKE, using the
--multus-cni
flag with the MKE install CLI command.Install a different CNI plugin.
Run the following command on all nodes in the cluster:
CNI_PLUGIN_VERSION=v1.3.0 CNI_ARCH=amd64 curl -sL https://github.com/containernetworking/plugins/releases/download/${CNI_PLUGIN_VERSION}/cni-plugins-linux-${CNI_ARCH}-${CNI_PLUGIN_VERSION}.tgz | sudo tar xvz -C /opt/cni/bin/
Determine the primary network interface for the node. You will need this information to create the
NetworkAttachmentDefinitions
file.Note
The name of the primary interface can vary with the underlying network adapter.
Run the following command on the nodes locate the default network interface. The
Iface
column in the line with destinationdefault
indicates which interface to use.route
Note
eth0 is the primary network interface in most Linux distributions.
Example output:
Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface default ip-172-31-32-1. 0.0.0.0 UG 100 0 0 ens3 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 172.18.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker_gwbridge 172.19.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br-05e6200d101b 172.31.32.0 0.0.0.0 255.255.240.0 U 0 0 0 ens3 ip-172-31-32-1. 0.0.0.0 255.255.255.255 UH 100 0 0 ens3 192.168.219.0 0.0.0.0 255.255.255.192 U 0 0 0 *
Important
Run all ensuing commands on the machine upon which kubectl is configured.
Create
NetworkAttachmentDefinitions
to specify other networks.cat <<EOF | kubectl create -f - apiVersion: "k8s.cni.cncf.io/v1" kind: NetworkAttachmentDefinition metadata: name: macvlan-conf spec: config: '{ "cniVersion": "0.3.0", "type": "macvlan", "master": "eth0", "mode": "bridge", "ipam": { "type": "host-local", "subnet": "192.168.1.0/24", "rangeStart": "192.168.1.200", "rangeEnd": "192.168.1.216", "routes": [ { "dst": "0.0.0.0/0" } ], "gateway": "192.168.1.1" } }' EOF
Create a multi-homed Pod:
cat <<EOF | kubectl create -f - apiVersion: v1 kind: Pod metadata: name: samplepod annotations: k8s.v1.cni.cncf.io/networks: macvlan-conf spec: containers: - name: samplepod command: ["/bin/ash", "-c", "trap : TERM INT; sleep infinity & wait"] image: alpine EOF
Check the network interfaces of the Pod:
kubectl exec -it samplepod -- ip a
Example output:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 3: eth0@if490: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP link/ether 9e:02:e2:cf:4f:7e brd ff:ff:ff:ff:ff:ff inet 192.168.58.70/32 scope global eth0 valid_lft forever preferred_lft forever 4: net1@if492: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP link/ether 0e:b0:45:13:94:f7 brd ff:ff:ff:ff:ff:ff inet 192.168.1.200/24 brd 192.168.1.255 scope global net1 valid_lft forever preferred_lft forever
Interface
Description
lo
A loopback interface.
eth0
Default network interface of the Pod.
net1
New interface created with the macvlan configuration.
Add multiple additional interfaces:
cat <<EOF | kubectl create -f - apiVersion: v1 kind: Pod metadata: name: samplepod-multi annotations: k8s.v1.cni.cncf.io/networks: macvlan-conf,macvlan-conf spec: containers: - name: samplepod command: ["/bin/ash", "-c", "trap : TERM INT; sleep infinity & wait"] image: alpine EOF
The operation results in the addition of the
net1
andnet2
network interfaces to the Pod.