Configure NIC bonding

You can configure L2 templates for the management cluster to set up a bond network interface for the PXE/management network.

This configuration must be applied to the bootstrap templates, before you run the bootstrap script to deploy the management cluster.

Caution

  • This configuration requires each host in your management cluster to have at least two physical interfaces.

  • Connect at least two interfaces per host to an Ethernet switch that supports Link Aggregation Control Protocol (LACP) port groups and LACP fallback.

  • Configure an LACP group on the ports connected to the NICs of a host.

  • Configure the LACP fallback on the port group to ensure that the host can boot over the PXE network before the bond interface is set up on the host operating system.

  • Configure server BIOS for both NICs of a bond to be PXE-enabled.

  • If the server does not support booting from multiple NICs, configure the port of the LACP group that is connected to the PXE-enabled NIC of a server to be primary port. With this setting, the port becomes active in the fallback mode.

To configure a bond interface that aggregates two interfaces for the PXE/Management network:

  1. In kaas-bootstrap/templates/bm/ipam-objects.yaml.template:

    1. Configure only the following parameters for the declaration of {{nic 0}}, as shown in the example below:

      • dhcp4

      • dhcp6

      • match

      • set-name

      Remove other parameters.

    2. Add the declaration of the second NIC {{nic 1}} to be added to the bond interface:

      • Specify match:mac-address: {{mac 1}} to match the MAC of the desired NIC.

      • Specify set-name: {{nic 1}} to ensure the correct name of the NIC.

    3. Add the declaration of the bond interface bond0. It must have the interfaces parameter listing both Ethernet interfaces.

    4. Set the interfaces parameter of the management network bridge (k8s-lcm in the example below) to include bond0.

      Starting from Container Cloud 2.16.0, you can use any interface name for the LCM network traffic. The Subnet objects for the LCM network must have the ipam/SVC-k8s-lcm label. For details, see Service labels and their life cycle.

      Caution

      In Container Cloud 2.15.0, the configuration of LCM network by use of the k8s-lcm bridge is mandatory. Addresses for the LCM network are assigned from the Subnet objects marked with the ipam/SVC-k8s-lcm label. Use of ipam/SVC-k8s-lcm label for the LCM network Subnet objects is mandatory. For details, see Service labels and their life cycle.

    5. Set the addresses, gateway4, and nameservers fields of the management network bridge to fetch data from the kaas-mgmt subnet.

    6. Configure bonding options using the parameters field. The only mandatory option is mode. See the example below for details.

      Note

      You can set any mode supported by netplan and your hardware.

  2. Verify your configuration using the following example:

    kind: L2Template
    metadata:
      name: kaas-mgmt
      ...
    spec:
      ...
      npTemplate: |
        version: 2
        ethernets:
          {{nic 0}}:
            dhcp4: false
            dhcp6: false
            match:
              mac-address: {{mac 0}}
            set-name: {{nic 0}}
          {{nic 1}}:
            dhcp4: false
            dhcp6: false
            match:
              mac-address: {{mac 1}}
            set-name: {{nic 1}}
        bonds:
          bond0:
            interfaces:
              - {{nic 0}}
              - {{nic 1}}
            parameters:
              mode: 802.3ad
            dhcp4: false
            dhcp6: false
        bridges:
          k8s-lcm:
            interfaces: [bond0]
            addresses:
              - {{ip "k8s-lcm:kaas-mgmt"}}
            gateway4: {{gateway_from_subnet "kaas-mgmt"}}
            nameservers:
              addresses: {{nameservers_from_subnet "kaas-mgmt"}}
        ...
    
  3. Proceed to bootstrap your management cluster as described in Bootstrap a management cluster.