Mirantis Container Cloud (MCC) becomes part of Mirantis OpenStack for Kubernetes (MOSK)!

Starting with MOSK 25.2, the MOSK documentation set covers all product layers, including MOSK management (formerly Container Cloud). This means everything you need is in one place. Some legacy names may remain in the code and documentation and will be updated in future releases. The separate Container Cloud documentation site will be retired, so please update your bookmarks for continued easy access to the latest content.

Add, remove, or reconfigure Ceph nodes¶

Warning

This procedure is valid for MOSK clusters that use the deprecated KaaSCephCluster custom resource (CR) instead of the MiraCeph CR that is available since MOSK 25.2 as a new Ceph configuration entrypoint. For the equivalent procedure with the MiraCeph CR, refer to the following section:

Add, remove, or reconfigure Ceph nodes

Mirantis Ceph Controller simplifies a Ceph cluster management by automating LCM operations. This section describes how to add, remove, or reconfigure Ceph nodes.

Note

When adding a Ceph node with the Ceph Monitor role, if any issues occur with the Ceph Monitor, rook-ceph removes it and adds a new Ceph Monitor instead, named using the next alphabetic character in order. Therefore, the Ceph Monitor names may not follow the alphabetical order. For example, a, b, d, instead of a, b, c.

Add Ceph nodes on a managed cluster¶

Prepare a new machine for the required managed cluster as described in Add a machine. During machine preparation, update the settings of the related bare metal host profile for the Ceph node being replaced with the required machine devices as described in Create a custom bare metal host profile.
Open the KaasCephCluster CR of a managed cluster for editing:
```
kubectl edit kaascephcluster -n <managedClusterProjectName>
```
Substitute <managedClusterProjectName> with the corresponding value.
In the nodes section, specify the parameters for a Ceph node as required. For the parameters description, see Node parameters.

The example configuration of the nodes section with the new node:
Since MOSK 25.2
Example configuration with storageDevices:
nodes: kaas-node-5bgk6: roles: - mon - mgr storageDevices: - config: deviceClass: hdd fullPath: /dev/disk/by-id/scsi-SATA_HGST_HUS724040AL_PN1334PEHN18ZS
Example configuration with storageDeviceFilter:
kaas-node-5bgk6: roles: - mon - mgr storageDeviceFilter: config: deviceClass: hdd filterByPath: "^/dev/disk/by-id/scsi-SATA_HGST_.+$""
Since MOSK 23.3
nodes: kaas-node-5bgk6: roles: - mon - mgr storageDevices: - config: deviceClass: hdd fullPath: /dev/disk/by-id/scsi-SATA_HGST_HUS724040AL_PN1334PEHN18ZS
Before MOSK 23.3
nodes: kaas-node-5bgk6: roles: - mon - mgr storageDevices: - config: deviceClass: hdd name: sdb
Warning

Since MOSK 23.3, Mirantis highly recommends using the non-wwn by-id symlinks to specify storage devices in the storageDevices list. For details, see Addressing storage devices prior to MOSK 25.2.
Note
- To use a new Ceph node for a Ceph Monitor or Ceph Manager deployment, also specify the roles parameter.
- Reducing the number of Ceph Monitors is not supported and causes the Ceph Monitor daemons removal from random nodes.
- Removal of the mgr role in the nodes section of the KaaSCephCluster CR does not remove Ceph Managers. To remove a Ceph Manager from a node, remove it from the nodes spec and manually delete the mgr pod in the Rook namespace.

Verify that all new Ceph daemons for the specified node have been successfully deployed in the Ceph cluster. The fullClusterInfo section should not contain any issues.

kubectl -n <managedClusterProjectName> get kaascephcluster -o yaml

Remove a Ceph node from a managed cluster¶

Note

Ceph node removal presupposes usage of a KaaSCephOperationRequest CR. For workflow overview, spec and phases description, see High-level workflow of Ceph OSD or node removal.

Note

To remove a Ceph node with a mon role, first move the Ceph Monitor to another node and remove the mon role from the Ceph node as described in Move a Ceph Monitor daemon to another node.

Open the KaasCephCluster CR of a managed cluster for editing:
```
kubectl edit kaascephcluster -n <managedClusterProjectName>
```
Substitute <managedClusterProjectName> with the corresponding value.
In the spec.cephClusterSpec.nodes.<machineName> section, remove the required Ceph OSD node specification.

Caution

If any other roles are present on the node, remove only the Ceph OSD specification for the node.

For example:
```
spec:
  cephClusterSpec:
    nodes:
      worker-5: # remove the entire entry for the required node
        storageDevices: {...}
        roles: [...]
```

Create a YAML template for the KaaSCephOperationRequest CR. For example:

apiVersion: kaas.mirantis.com/v1alpha1
kind: KaaSCephOperationRequest
metadata:
  name: remove-osd-worker-5
  namespace: <managedClusterProjectName>
spec:
  osdRemove:
    nodes:
      worker-5:
        completeCleanUp: true
  kaasCephCluster:
    name: <kaasCephClusterName>
    namespace: <managedClusterProjectName>

Substitute <managedClusterProjectName> with the corresponding cluster namespace and <kaasCephClusterName> with the corresponding KaaSCephCluster name.

Apply the template on the management cluster in the corresponding namespace:
```
kubectl apply -f remove-osd-worker-5.yaml
```

Verify that the corresponding request has been created:

kubectl get kaascephoperationrequest remove-osd-worker-5 -n <managedClusterProjectName>

Verify that the removeInfo section appeared in the KaaSCephOperationRequest CR status:

kubectl -n <managedClusterProjectName> get kaascephoperationrequest remove-osd-worker-5 -o yaml

Verify that the cleanUpMap section matches the required removal and wait for the ApproveWaiting phase to appear in status:
```
kubectl -n <managedClusterProjectName> get kaascephoperationrequest remove-osd-worker-5 -o yaml
```
Example of system response:
```
status:
  phase: ApproveWaiting
```

Edit the KaaSCephOperationRequest CR and set the approve flag to true:

kubectl -n <managedClusterProjectName> edit kaascephoperationrequest remove-osd-worker-5

For example:

spec:
  osdRemove:
    approve: true

Review the status of the KaaSCephOperationRequest resource request processing. The valuable parameters are as follows:
- status.phase - the current state of request processing
- status.messages - the description of the current phase
- status.conditions - full history of request processing before the current phase
- status.removeInfo.issues and status.removeInfo.warnings - contain error and warning messages occurred during request processing

Verify that the KaaSCephOperationRequest has been completed. For example:

status:
  phase: Completed # or CompletedWithWarnings if there are non-critical issues

Remove the device cleanup jobs:

kubectl delete jobs -n ceph-lcm-mirantis -l app=miraceph-cleanup-disks

Reconfigure a Ceph node on a managed cluster¶

There is no hot reconfiguration procedure for existing Ceph OSDs and Ceph Monitors. To reconfigure an existing Ceph node, follow the steps below:

Remove the Ceph node from the Ceph cluster as described in Remove a Ceph node from a managed cluster.
Add the same Ceph node but with a modified configuration as described in Add Ceph nodes on a managed cluster.