Searching for results...

No results

Your search did not match anything from Mirantis documentation.
Check your spelling or try different keywords.

An error occurred

An error occurred while using the search.
Try your search again or contact us to let us know about it.

Newer documentation is now live.You are currently reading an older version.

Manage Ceph RBD or CephFS clients

Warning

This procedure is valid for MOSK clusters that use the MiraCeph custom resource (CR), which is available since MOSK 25.2 to replace the unsupported KaaSCephCluster resource. And MiraCeph will be automatically migrated to CephDeployment in MOSK 26.1. For details, see Deprecation Notes: KaaSCephCluster API on management clusters.

For the equivalent procedure with the unsupported KaaSCephCluster CR, refer to the following section:

Manage Ceph RBD or CephFS clients

The MiraCeph resource allows managing custom Ceph RADOS Block Device (RBD) or Ceph File System (CephFS) clients. This section describes how to create, access, and remove Ceph RBD or CephFS clients.

For all supported parameters of Ceph clients, refer to Clients parameters.

Create an RBD or CephFS client

  1. Edit the MiraCeph resource by adding a new Ceph client to the spec section:

    kubectl -n ceph-lcm-mirantis edit miraceph

    Example of adding an RBD client to the kubernetes-ssd pool:

    spec:
  clients:
  - name: rbd-client
    caps:
      mon: allow r, allow command "osd blacklist"
      osd: profile rbd pool=kubernetes-ssd

    Example of adding a CephFS client to the cephfs-1 Ceph File System:

    spec:
  clients:
  - name: cephfs-1-client
    caps:
      mds: allow rwp
      mon: allow r, allow command "osd blacklist"
      osd: allow rw tag cephfs data=cephfs-1 metadata=*

    For details about caps, refer to Ceph documentation: Authorization (capabilities).

    Note

    Ceph supports only providing of client access to the whole Ceph File System with all data pools in it.

  2. Wait for created clients to become ready in the MiraCephHealth status:

    kubectl -n ceph-lcm-mirantis get mchealth -o yaml

    Example output:

    status:
  fullClusterInfo:
    blockStorageStatus:
      clientsStatus:
        rbd-client:
          present: true
          status: Ready
        cephfs-1-client:
          present: true
          status: Ready

Access data using an RBD or CephFS client

  1. Using the MiraCephSecret status, obtain secretInfo with the Ceph client credentials :

    kubectl -n ceph-lcm-mirantis get mcsecret -o yaml

    Example output:

    status:
  secretInfo:
    clientSecrets:
    - name: rbd-client
      secretName: rook-ceph-client-rbd-client
      secretNamespace: rook-ceph
    - name: cephfs-1-client
      secretName: rook-ceph-client-cephfs-1-client
      secretNamespace: rook-ceph

  2. Use secretName and secretNamespace to access the Ceph client credentials from a MOSK cluster:

    kubectl -n <secretNamespace> get secret <secretName> -o jsonpath='{.data.<clientName>}' | base64 -d; echo

    Substitute the following parameters:

    • <secretNamespace> with secretNamespace from the previous step

    • <secretName> with secretName from the previous step

    • <clientName> with the Ceph RBD or CephFS client name set in spec.clients the MiraCeph resource, for example, rbd-client

    Example output:

    AQAGHDNjxWYXJhAAjafCn3EtC6KgzgI1x4XDlg==

  3. Using the obtained credentials, create two configuration files on the required workloads to connect them with Ceph pools or file systems:

    • /etc/ceph/ceph.conf:

      [default]
   mon_host = <mon1IP>:6789,<mon2IP>:6789,...,<monNIP>:6789

      where mon_host are the comma-separated IP addresses with 6789 ports of the current Ceph Monitors. For example, 10.10.0.145:6789,10.10.0.153:6789,10.10.0.235:6789.

    • /etc/ceph/ceph.client.<clientName>.keyring:

      [client.<clientName>]
    key = <cephClientCredentials>

      • <clientName> is a client name set in spec.clients of the MiraCeph resource. For example, rbd-client.

      • <cephClientCredentials> are the client credentials obtained in the previous steps. For example, AQAGHDNjxWYXJhAAjafCn3EtC6KgzgI1x4XDlg==.

  4. If the client caps parameters contain mon: allow r, verify the client access using the following command:

    ceph -n client.<clientName> -s

Remove an RBD or CephFS client

  1. Edit the MiraCeph resource by removing the Ceph client from spec.clients:

    kubectl -n ceph-lcm-mirantis edit miraceph

  2. Wait for the client to be removed from the MiraCephHealth status in status.fullClusterInfo.blockStorageStatus.clientsStatus:

    kubectl -n ceph-lcm-mirantis get mchealth -o yaml