Enable Ceph multinetwork¶
Ceph allows establishing multiple IP networks and subnet masks for clusters
with configured L3 network rules. In Container Cloud, you can configure
multinetwork through the network
section of the KaaSCephCluster
CR.
Ceph Controller uses this section to specify the Ceph networks for external
access and internal daemon communication. The parameters in the network
section use the CIDR notation, for example, 10.0.0.0/24
.
Before enabling multiple networks for a Ceph cluster, consider the following requirements:
Do not confuse the IP addresses you define with the public-facing IP addresses the network clients may use to access the services.
If you define more than one IP address and subnet mask for the public or cluster network, ensure that the subnets within the network can route to each other.
Include each IP address or subnet in the
network
section to IP tables and open ports for them as necessary.The pods of the Ceph OSD and RadosGW daemons use cross-pods health checkers to verify that the entire Ceph cluster is healthy. Therefore, each CIDR must be accessible inside Ceph pods.
Avoid using the
0.0.0.0/0
CIDR in thenetwork
section. With a zero range inpublicNet
and/orclusterNet
, the Ceph daemons behavior is unpredictable.
To enable multinetwork for Ceph:
Select from the following options:
If the Ceph cluster is not deployed on a managed cluster yet, edit the deployment
KaaSCephCluster
YAML template.If the Ceph cluster is already deployed on a managed cluster, open
KaaSCephCluster
for editing:kubectl -n <managedClusterProjectName> edit kaascephcluster
Substitute
<managedClusterProjectName>
with a corresponding value.
In the
clusterNet
and/orpublicNet
parameters of thecephClusterSpec.network
section, define a comma-separated array of CIDRs. For example:network: publicNet: 10.12.0.0/24,10.13.0.0/24 clusterNet: 10.10.0.0/24,10.11.0.0/24
Select from the following options:
If you are creating a managed cluster, save the updated
KaaSCephCluster
template to the corresponding file and proceed with the managed cluster creation.If you are configuring
KaaSCephCluster
of an existing managed cluster, exiting the text editor will apply the changes.
Once done, the specified network CIDRs will be passed to the Ceph daemons pods
through the rook-config-override
ConfigMap.