Server machines used in CPI reference architecture have 1 built-in dual port 1 GbE network interface card (NIC), and two additional 1/10 GbE NICs.
The built-in NIC is used for network boot of the servers. Only one interface is typically for PXE boot, the other one is kept unused for redundancy.
The first pair of 1/10 Gbit Ethernet interfaces is used for the management and control plane traffic. These interfaces should be connected to an access switch in 1 or 10 GbE mode.
In CPI referernce architecture, the interfaces of the first NIC are
joined in a bond logical interface in 802.3ad mode.
The second NIC with two interfaces is used for the data plane traffic and
storage traffic. On the operating system level, ports on this 1/10 GbE card
are joined into an LACP bond (Linux bond mode 802.3ad).
Recommended LACP load balancing method for both bond interfaces is
transmission hash policy based on TCP/UDP port numbers (xmit_hash_policy
layer3+4).
This NIC must be connected to an access switch in 10 GbE mode.
Note
The LACP configuration in 802.3ad mode on the server side must
be supported by the corresponding configuration of switching fabric.
See Switching fabric capabilities for details.