Use Multus CNI to create multi-homed Pods¶

Available since MKE 3.7.0

In Kubernetes, by default, a Pod is only connected to a single network interface, which is the default network. Using Multus CNI, however, you can create a multi-home Pod that has multiple network interfaces.

The following example procedure attaches two network interfaces to a Pod, net1 and net2.

Configure kubectl for your MKE cluster.
Enable Multus CNI in the MKE cluster when you install MKE, using the --multus-cni flag with the MKE install CLI command.

Install a different CNI plugin.

Run the following command on all nodes in the cluster:

CNI_PLUGIN_VERSION=v1.3.0
CNI_ARCH=amd64
curl -sL
https://github.com/containernetworking/plugins/releases/download/${CNI_PLUGIN_VERSION}/cni-plugins-linux-${CNI_ARCH}-${CNI_PLUGIN_VERSION}.tgz
| sudo tar xvz -C /opt/cni/bin/

Determine the primary network interface for the node. You will need this information to create the NetworkAttachmentDefinitions file.

Note

The name of the primary interface can vary with the underlying network adapter.

Run the following command on the nodes locate the default network interface. The Iface column in the line with destination default indicates which interface to use.

route

Note

eth0 is the primary network interface in most Linux distributions.

Example output:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref  Use Iface
default         ip-172-31-32-1. 0.0.0.0         UG    100    0    0   ens3
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0    0   docker0
172.18.0.0      0.0.0.0         255.255.0.0     U     0      0    0   docker_gwbridge
172.19.0.0      0.0.0.0         255.255.0.0     U     0      0    0   br-05e6200d101b
172.31.32.0     0.0.0.0         255.255.240.0   U     0      0    0   ens3
ip-172-31-32-1. 0.0.0.0         255.255.255.255 UH    100    0    0   ens3
192.168.219.0   0.0.0.0         255.255.255.192 U     0      0    0   *

Important

Run all ensuing commands on the machine upon which kubectl is configured.

Create NetworkAttachmentDefinitions to specify other networks.

cat <<EOF | kubectl create -f -
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: macvlan-conf
spec:
config: '{
      "cniVersion": "0.3.0",
      "type": "macvlan",
      "master": "eth0",
      "mode": "bridge",
      "ipam": {
      "type": "host-local",
      "subnet": "192.168.1.0/24",
      "rangeStart": "192.168.1.200",
      "rangeEnd": "192.168.1.216",
      "routes": [
         { "dst": "0.0.0.0/0" }
      ],
      "gateway": "192.168.1.1"
      }
   }'
EOF

Create a multi-homed Pod:

cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
name: samplepod
annotations:
   k8s.v1.cni.cncf.io/networks: macvlan-conf
spec:
containers:
   - name: samplepod
   command: ["/bin/ash", "-c", "trap : TERM INT; sleep infinity & wait"]
   image: alpine
EOF

Check the network interfaces of the Pod:

kubectl exec -it samplepod -- ip a

Example output:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
   link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
   inet 127.0.0.1/8 scope host lo
      valid_lft forever preferred_lft forever
3: eth0@if490: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP
   link/ether 9e:02:e2:cf:4f:7e brd ff:ff:ff:ff:ff:ff
   inet 192.168.58.70/32 scope global eth0
      valid_lft forever preferred_lft forever
4: net1@if492: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
   link/ether 0e:b0:45:13:94:f7 brd ff:ff:ff:ff:ff:ff
   inet 192.168.1.200/24 brd 192.168.1.255 scope global net1
      valid_lft forever preferred_lft forever

Interface	Description
`lo`	A loopback interface.
`eth0`	Default network interface of the Pod.
`net1`	New interface created with the macvlan configuration.

Add multiple additional interfaces:

cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
name: samplepod-multi
annotations:
   k8s.v1.cni.cncf.io/networks: macvlan-conf,macvlan-conf
spec:
containers:
   - name: samplepod
   command: ["/bin/ash", "-c", "trap : TERM INT; sleep infinity & wait"]
   image: alpine
EOF

The operation results in the addition of the net1 and net2 network interfaces to the Pod.