Ceph

Ceph monitors use their node host networks to interact with Ceph daemons. Ceph daemons communicate with each other over a specified cluster network and provide endpoints over the public network.

The messenger V2 (msgr2) or earlier V1 (msgr) protocols are used for communication between Ceph daemons.

Ceph daemon

Network

Protocol

Port

Description

Consumers

Manager (mgr)

Cluster network

msgr/msgr2

6800

Listens on the first available port of the 6800-7300 range

csi-rbdplugin,
csi-rbdprovisioner,
rook-ceph-mon

Metadata server (mds)

Cluster network

msgr/msgr2

6800

Listens on the first available port of the 6800-7300 range

csi-cephfsplugin,
csi-cephfsprovisioner

Monitor (mon)

LCM host network

msgr/msgr2

msgr:3300,
msgr2:6789

Monitor has separate ports for msgr and msgr2

Ceph clients
rook-ceph-osd,
rook-ceph-rgw

Ceph OSD (osd)

Cluster network

msgr/msgr2

6800-7300

Binds to the first available port from the 6800-7300 range

rook-ceph-mon,
rook-ceph-mgr,
rook-ceph-mds