Disable TX offloading on NICs used by vRouter

In the following cases, a TCP-based service may not work on VMs:

• If the setup has nested VMs.

• If VMs are running in the ESXi hypervisor.

• If the Network Interface Cards (NICs) do not support the IP checksum calculation and generate an incorrect checksum. For example, the Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe NIC cards.

To resolve the issue, disable the transmit (TX) offloading on all OpenStack compute nodes for the affected NIC used by the vRouter as described below.

To identify the issue:

1. Verify whether ping is working between VMs on different hypervisor hosts and the TCP services are working.

2. Run the following command for the vRouter Agent and verify whether the output includes the number of Checksum errors:

   kubectl -n tf exec tf-vrouter-agent-XXXXX -c agent -- dropstats
   

3. Run the following command and verify if the output includes the cksum incorrect entries:

   kubectl -n tf exec tf-vrouter-agent-XXXXX -c agent -- tcpdump -i <tunnel interface> -v -nn | grep -i incorrect
   

   Example of system response:

   tcpdump: listening on <tunnel interface>, link-type EN10MB (Ethernet), capture size 262144 bytes
   <src ip.port> > <dst ip.port>: Flags [S.], cksum 0x43bf (incorrect -> 0xb8dc), \
   seq 1901889431, ack 1081063811, win 28960, options [mss 1420,sackOK,\
   TS val 456361578 ecr 41455995,nop,wscale 7], length 0
   <src ip.port> > <dst ip.port>: Flags [S.], cksum 0x43bf (incorrect -> 0xb8dc), \
   seq 1901889183, ack 1081063811, win 28960, options [mss 1420,sackOK,\
   TS val 456361826 ecr 41455995,nop,wscale 7], length 0
   <src ip.port> > <dst ip.port>: Flags [S.], cksum 0x43bf (incorrect -> 0xb8dc), \
   seq 1901888933, ack 1081063811, win 28960, options [mss 1420,sackOK,\
   TS val 456362076 ecr 41455995,nop,wscale 7], length 0
   

4. Run the following command for the vRouter Agent container and verify whether the output includes the information about a drop for an unknown reason:

   kubectl -n tf exec tf-vrouter-agent-XXXXX -c agent -- flow -l
   

To disable the TX offloading on NICs used by vRouter:

1. Open the TFOperator custom resource (CR) for editing:

   kubectl -n tf edit tfoperators.operator.tf.mirantis.com openstack-tf
   

2. Specify the DISABLE_TX_OFFLOAD variable with the "YES" value for the vRouter Agent container:

   API v1alpha1 Deprecated since MOSK 24.2

   spec:
     controllers:
       tf-vrouter:
         agent:
           containers:
           - name: agent
             env:
             - name: DISABLE_TX_OFFLOAD
               value: "YES"
   

   API v2 Available since MOSK 24.1

   spec:
     features:
       vRouter:
         disableTXOffload: true
   

   Warning

   Once you modify the TFOperator CR, the tf-vrouter-agent-<XXXXX> pods will not restart automatically because they use the OnDelete update strategy. Restart such pods manually, considering that the vRouter pods restart causes network services interruption for the VMs hosted on the affected nodes.

3. To disable TX offloading on a specific subset of nodes, use custom vRouter settings. For details, see Custom vRouter settings.

   Warning

   Once you add a new CustomSpec, a new daemon set will be generated and the tf-vrouter-agent-<XXXXX> pods will be automatically restarted. The vRouter pods restart causes network services interruption for VMs hosted on the affected node. Therefore, plan this procedure accordingly.