Managed cluster networking¶
A Kubernetes cluster networking is typically focused on connecting pods on different nodes. On bare metal, however, the cluster networking is more complex as it needs to facilitate many different types of traffic.
Kubernetes clusters managed by Mirantis Container Cloud have the following types of traffic:
- PXE/Lifecycle management (LCM) network
Enables the PXE boot of all bare metal machines in Container Cloud. Connects LCM agents running on the hosts to the Container Cloud LCM API. The LCM API is provided by the regional or management cluster.
- LCM subnet
Provides IP addresses that are statically allocated by the IPAM service to bare metal hosts. This network must be connected to the Kubernetes API endpoint of the regional cluster through an IP router. LCM agents running on managed clusters will connect to the regional cluster API through this router. LCM subnets may be different per managed cluster as long as this connection requirement is satisfied. The Virtual IP (VIP) address for load balancer that enables access to the Kubernetes API of the managed cluster must be allocated from the LCM subnet.
- Kubernetes workloads network
Serves as an underlay network for traffic between pods in the managed cluster. This network should not be shared between clusters.
- Kubernetes workloads subnet
Provides IP addresses that are assigned to nodes and used by Calico.
- Kubernetes external network
Serves ingress traffic to the managed cluster from the outside world. This network can be shared between clusters, but must have a dedicated subnet per cluster.
- Services subnet
Provides IP addresses for externally available load-balanced services. The address ranges for MetalLB are assigned from this subnet. This subnet must be unique per managed cluster.
- Storage network
Serves storage access and replication traffic from and to Ceph OSD services. The storage network does not need to be connected to any IP routers and does not require external access, unless you want to use Ceph from outside of a Kubernetes cluster. To use a dedicated storage network, define and configure both subnets listed below.
- Storage subnet
Provides IP addresses that are assigned to Ceph nodes. The Ceph OSD services bind to these addresses on their respective nodes. Serves Ceph access traffic from and to storage clients. This subnet is unique per managed cluster.
- Storage replication subnet
Provides IP addresses that are assigned to Ceph nodes. The Ceph OSD services bind to these addresses on their respective nodes. Serves Ceph internal replication traffic. This subnet is unique per managed cluster.
- Out-of-Band (OOB) network
Connects baseboard management controllers (BMCs) of the bare metal hosts. This network must not be accessible from the managed clusters.
The following diagram illustrates the networking schema of the Container Cloud deployment on bare metal with a managed cluster: