Managed cluster networking

A Kubernetes cluster networking is typically focused on connecting pods on different nodes. On bare metal, however, the cluster networking is more complex as it needs to facilitate many different types of traffic.

Kubernetes clusters managed by Mirantis Container Cloud have the following types of traffic:

  • PXE/Lifecycle management (LCM) network

    Enables the PXE boot of all bare metal machines in Container Cloud. Connects LCM agents running on the hosts to the Container Cloud LCM API. The LCM API is provided by the regional or management cluster.

    • LCM subnet

      Provides IP addresses that are statically allocated by the IPAM service to bare metal hosts. This network must be connected to the Kubernetes API endpoint of the regional cluster through an IP router. LCM agents running on managed clusters will connect to the regional cluster API through this router. LCM subnets may be different per managed cluster as long as this connection requirement is satisfied. The Virtual IP (VIP) address for load balancer that enables access to the Kubernetes API of the managed cluster must be allocated from the LCM subnet.

  • Kubernetes workloads network

    Technology Preview

    Serves as an underlay network for traffic between pods in the managed cluster. This network should not be shared between clusters.

    • Kubernetes workloads subnet

      Provides IP addresses that are assigned to nodes and used by Calico.

  • Kubernetes external network

    Serves ingress traffic to the managed cluster from the outside world. This network can be shared between clusters, but must have a dedicated subnet per cluster.

    • Services subnet

      Technology Preview

      Provides IP addresses for externally available load-balanced services. The address ranges for MetalLB are assigned from this subnet. This subnet must be unique per managed cluster.

  • Storage network

    Serves storage access and replication traffic from and to Ceph OSD services. The storage network does not need to be connected to any IP routers and does not require external access, unless you want to use Ceph from outside of a Kubernetes cluster. To use a dedicated storage network, define and configure both subnets listed below.

    • Storage subnet

      Provides IP addresses that are assigned to Ceph nodes. The Ceph OSD services bind to these addresses on their respective nodes. Serves Ceph access traffic from and to storage clients. This subnet is unique per managed cluster.

    • Storage replication subnet

      Provides IP addresses that are assigned to Ceph nodes. The Ceph OSD services bind to these addresses on their respective nodes. Serves Ceph internal replication traffic. This subnet is unique per managed cluster.

  • Out-of-Band (OOB) network

    Connects baseboard management controllers (BMCs) of the bare metal hosts. This network must not be accessible from the managed clusters.

The following diagram illustrates the networking schema of the Container Cloud deployment on bare metal with a managed cluster:

../../../_images/bm-cluster-l3-networking-multihomed.png