Move Ceph Monitor before node replacement

This document describes how to migrate a Ceph Monitor to another node on bare metal-based clusters before node replacement.

Warning

Remove the Ceph Monitor role before the machine removal.

Warning

Make sure that the Ceph cluster always has an odd number of Ceph Monitors.

The procedure of a Ceph Monitor migration assumes that you temporarily move the Ceph Manager/Monitor to another (for example, worker) node. After a node replacement, we recommend migrating the Ceph Manager/Monitor to the new manager node.

OpenStack instances that boot from a Ceph RBD image or volume store the Ceph Monitor IP addresses. If during replacement the new node obtains a different address, those instances lose access to their disks until you reconfigure them. To keep the previous Monitor IP address on the new node, use the experimental staticIPRequest field when you create a new Machine object. For details, see Request a static IP address for a machine.

To migrate a Ceph Monitor to another machine:

  1. Experimental. Not for production use. To keep the previous Monitor IP address on the new node, from the CephDeployment object, capture the monitorIP value of the node that you replace.

  2. Move the Ceph Manager/Monitor daemon from the affected node to one of the worker machines as described in Move a Ceph Monitor daemon to another node.

  3. Delete the affected node.

  4. Add a new manager node without the Monitor and Manager role.

    Warning

    The addition of a new node with the Monitor and Manager role breaks the odd number quorum of Ceph Monitors.

    To keep the captured Ceph Monitor IP, create a new Machine object with staticIPRequest as described in Request a static IP address for a machine. Create the new machine only after IPAM has released the IP address of the deleted node.

  5. Move the previously migrated Ceph Manager/Monitor daemon to the new manager node as described in Move a Ceph Monitor daemon to another node.

  6. If you requested the previous Monitor IP address for the new node, in the CephDeployment object, set monitorIP of the new node to the captured IP address.