Add a Ceph cluster¶

After you add machines to your new bare metal managed cluster as described in Add a machine, create a Ceph cluster on top of this managed cluster using the Mirantis Container Cloud web UI.

For an advanced configuration through the KaaSCephCluster CR, see Mirantis Container Cloud Operations Guide: Ceph advanced configuration. To configure Ceph Controller through Kubernetes templates to manage Ceph nodes resources, see Mirantis Container Cloud Operations Guide: Enable Ceph tolerations and resources management.

The procedure below enables you to create a Ceph cluster with minimum three Ceph nodes that provides persistent volumes to the Kubernetes workloads in the managed cluster.

To create a Ceph cluster in the managed cluster:

Log in to the Container Cloud web UI with the m:kaas:namespace@operator or m:kaas:namespace@writer permissions.
Switch to the required project using the Switch Project action icon located on top of the main left-side navigation panel.
In the Clusters tab, click the required cluster name. The Cluster page with the Machines and Ceph clusters lists opens.
In the Ceph Clusters block, click Create Cluster.

Configure the Ceph cluster in the Create New Ceph Cluster wizard that opens:

Create new Ceph cluster¶
Section	Parameter name	Description
General settings	Name	The Ceph cluster name.
	Cluster Network	Replication network for Ceph OSDs. Must contain the CIDR definition and match the corresponding values of the cluster `L2Template` object or the environment network values.
	Public Network	Public network for Ceph data. Must contain the CIDR definition and match the corresponding values of the cluster `L2Template` object or the environment network values.
	Enable OSDs LCM	Select to enable LCM for Ceph OSDs.
Machines / Machine #1-3	Select machine	Select the name of the Kubernetes machine that will host the corresponding Ceph node in the Ceph cluster.
	Manager, Monitor	Select the required Ceph services to install on the Ceph node.
	Devices	Select the disk that Ceph will use. Warning Do not select the device for system services, for example, `sda`. Warning A Ceph cluster does not support removable devices that are hosts with hotplug functionality enabled. To use devices as Ceph OSD data devices, make them non-removable or disable the hotplug functionality in the BIOS settings for disks that are configured to be used as Ceph OSD data devices.
	Enable Object Storage	Select to enable the single-instance RGW Object Storage.

To add more Ceph nodes to the new Ceph cluster, click + next to any Ceph Machine title in the Machines tab. Configure a Ceph node as required.

Warning

Do not add more than 3 Manager and/or Monitor services to the Ceph cluster.
After you add and configure all nodes in your Ceph cluster, click Create.
Open the KaaSCephCluster CR for editing as described in Mirantis Container Cloud Operations Guide: Ceph advanced configuration.
Verify that the following snippet is present in the KaaSCephCluster configuration:
```
network:
  clusterNet: 10.10.10.0/24
  publicNet: 10.10.11.0/24
```

Configure the pools for Image, Block Storage, and Compute services.

Note

Ceph validates the specified pools. Therefore, do not omit any of the following pools.

spec:
  pools:
    - default: true
      deviceClass: hdd
      name: kubernetes
      replicated:
        size: 3
      role: kubernetes
    - default: false
      deviceClass: hdd
      name: volumes
      replicated:
        size: 3
      role: volumes
    - default: false
      deviceClass: hdd
      name: vms
      replicated:
        size: 3
      role: vms
    - default: false
      deviceClass: hdd
      name: backup
      replicated:
        size: 3
      role: backup
    - default: false
      deviceClass: hdd
      name: images
      replicated:
        size: 3
      role: images

Each Ceph pool, depending on its role, has a default targetSizeRatio value that defines the expected consumption of the total Ceph cluster capacity. The default ratio values for MOSK pools are as follows:

20.0% for a Ceph pool with role volumes
40.0% for a Ceph pool with role vms
10.0% for a Ceph pool with role images
10.0% for a Ceph pool with role backup

Once all pools are created, verify that an appropriate secret required for a successful deployment of the OpenStack services that rely on Ceph is created in the openstack-ceph-shared namespace:
```
kubectl -n openstack-ceph-shared get secrets openstack-ceph-keys
```
Example of a positive system response:
```
NAME                  TYPE     DATA   AGE
openstack-ceph-keys   Opaque   7      36m
```
Verify your Ceph cluster as described in Mirantis Container Cloud Operations Guide: Verify Ceph.