Managed cluster networking¶
A Kubernetes cluster networking is typically focused on connecting pods on different nodes. On bare metal, however, the cluster networking is more complex as it needs to facilitate many different types of traffic.
Kubernetes clusters managed by Mirantis Container Cloud have the following types of traffic:
- PXE network
Enables the PXE boot of all bare metal machines in Container Cloud. This network is not configured on the hosts in a managed cluster. It is used by the bare metal provider to provision additional hosts in managed clusters and is disabled on the hosts after provisioning is done.
- Life-cycle management (LCM) network
Connects LCM agents running on the hosts to the Container Cloud LCM API. The LCM API is provided by the regional or management cluster. The LCM network is also used for communication between
kubeletand the Kubernetes API server inside a Kubernetes cluster. The MKE components use this network for communication inside a swarm cluster.
- LCM subnet
Provides IP addresses that are statically allocated by the IPAM service to bare metal hosts. This network must be connected to the Kubernetes API endpoint of the regional cluster through an IP router. LCM agents running on managed clusters will connect to the regional cluster API through this router. LCM subnets may be different per managed cluster as long as this connection requirement is satisfied. The Virtual IP (VIP) address for load balancer that enables access to the Kubernetes API of the managed cluster must be allocated from the LCM subnet.
- Kubernetes workloads network
Serves as an underlay network for traffic between pods in the managed cluster. This network should not be shared between clusters.
- Kubernetes workloads subnet
Provides IP addresses that are assigned to nodes and used by Calico.
- Kubernetes external network
Serves ingress traffic to the managed cluster from the outside world. This network can be shared between clusters, but must have a dedicated subnet per cluster.
- Services subnet
Provides IP addresses for externally available load-balanced services. The address ranges for MetalLB are assigned from this subnet. This subnet must be unique per managed cluster.
- Storage network
Serves storage access and replication traffic from and to Ceph OSD services. The storage network does not need to be connected to any IP routers and does not require external access, unless you want to use Ceph from outside of a Kubernetes cluster. To use a dedicated storage network, define and configure both subnets listed below.
- Storage access subnet
Provides IP addresses that are assigned to Ceph nodes. The Ceph OSD services bind to these addresses on their respective nodes. Serves Ceph access traffic from and to storage clients. This is a public network in Ceph terms. 1 This subnet is unique per managed cluster.
- Storage replication subnet
Provides IP addresses that are assigned to Ceph nodes. The Ceph OSD services bind to these addresses on their respective nodes. Serves Ceph internal replication traffic. This is a cluster network in Ceph terms. 1 This subnet is unique per managed cluster.
- Out-of-Band (OOB) network
Connects baseboard management controllers (BMCs) of the bare metal hosts. This network must not be accessible from the managed clusters.
The following diagram illustrates the networking schema of the Container Cloud deployment on bare metal with a managed cluster: