Manage Ceph RBD or CephFS clients¶

Available since 2.23.1 (Cluster release 12.7.0)

The KaaSCephCluster resource allows managing custom Ceph RADOS Block Device (RBD) or Ceph File System (CephFS) clients. This section describes how to create, access, and remove Ceph RBD or CephFS clients.

For all supported parameters of Ceph clients, refer to Clients parameters.

Create an RBD or CephFS client¶

Edit the KaaSCephCluster resource by adding a new Ceph client to the spec section:

kubectl -n <managedClusterProjectName> edit kaascephcluster

Substitute <managedClusterProject> with the corresponding Container Cloud project where the managed cluster was created.

Example of adding an RBD client to the kubernetes-ssd pool:

spec:
  cephClusterSpec:
    clients:
    - name: rbd-client
      caps:
        mon: allow r, allow command "osd blacklist"
        osd: profile rbd pool=kubernetes-ssd

Example of adding a CephFS client to the cephfs-1 Ceph File System :

spec:
  cephClusterSpec:
    clients:
    - name: cephfs-1-client
      caps:
        mds: allow rwp
        mon: allow r, allow command "osd blacklist"
        osd: allow rw tag cephfs data=cephfs-1 metadata=*

For details about caps, refer to Ceph documentation: Authorization (capabilities).

Note

Ceph supports only providing of client access to the whole Ceph File System with all data pools in it.

Wait for created clients to become ready in the KaaSCephCluster status:

kubectl -n <managedClusterProject> get kaascephcluster -o yaml

Example output:

status:
  fullClusterInfo:
    blockStorageStatus:
      clientsStatus:
        rbd-client:
          present: true
          status: Ready
        cephfs-1-client:
          present: true
          status: Ready

Access data using an RBD or CephFS client¶

Using the KaaSCephCluster status, obtain secretInfo with the Ceph client credentials :

kubectl -n <managedClusterProject> get kaascephcluster -o yaml

Example output:

status:
  miraCephSecretsInfo:
    secretInfo:
      clientSecrets:
      - name: rbd-client
        secretName: rook-ceph-client-rbd-client
        secretNamespace: rook-ceph
      - name: cephfs-1-client
        secretName: rook-ceph-client-cephfs-1-client
        secretNamespace: rook-ceph

Use secretName and secretNamespace to access the Ceph client credentials from a managed cluster:
```
kubectl --kubeconfig <managedClusterKubeconfig> -n <secretNamespace> get secret <secretName> -o jsonpath='{.data.<clientName>}' | base64 -d; echo
```
Substitute the following parameters:
- <managedClusterKubeconfig> with a managed cluster kubeconfig
- <secretNamespace> with secretNamespace from the previous step
- <secretName> with secretName from the previous step
- <clientName> with the Ceph RBD or CephFS client name set in spec.cephClusterSpec.clients the KaaSCephCluster resource, for example, rbd-client
Example output:
```
AQAGHDNjxWYXJhAAjafCn3EtC6KgzgI1x4XDlg==
```
Using the obtained credentials, create two configuration files on the required workloads to connect them with Ceph pools or file systems:
- /etc/ceph/ceph.conf:
```
[default]
   mon_host = <mon1IP>:6789,<mon2IP>:6789,...,<monNIP>:6789
```
  where mon_host are the comma-separated IP addresses with 6789 ports of the current Ceph Monitors. For example, 10.10.0.145:6789,10.10.0.153:6789,10.10.0.235:6789.
- /etc/ceph/ceph.client.<clientName>.keyring:
```
[client.<clientName>]
    key = <cephClientCredentials>
```
  - <clientName> is a client name set in spec.cephClusterSpec.clients the KaaSCephCluster resource, for example, rbd-client
  - <cephClientCredentials> are the client credentials obtained in the previous steps. For example, AQAGHDNjxWYXJhAAjafCn3EtC6KgzgI1x4XDlg==
If the client caps parameters contain mon: allow r, verify the client access using the following command:
```
ceph -n client.<clientName> -s
```

Remove an RBD or CephFS client¶

Edit the KaaSCephCluster resource by removing the Ceph client from spec.cephClusterSpec.clients:
```
kubectl -n <managedClusterProject> edit kaascephcluster
```
Wait for the client to be removed from the KaaSCephCluster status in status.fullClusterInfo.blockStorageStatus.clientsStatus:
```
kubectl -n <managedClusterProject> get kaascephcluster -o yaml
```