Ceph¶
Ceph monitors use their node host networks to interact with Ceph daemons. Ceph daemons communicate with each other over a specified cluster network and provide endpoints over the public network.
The messenger V2 (msgr2) or earlier V1 (msgr) protocols are used for
communication between Ceph daemons.
Ceph daemon |
Network |
Protocol |
Port |
Description |
Consumers |
|---|---|---|---|---|---|
Manager ( |
Cluster network |
msgr/msgr2 |
6800,
9283
|
Listens on the first available port of the 6800-7300 range.
Uses 9283 port for exporting metrics.
|
csi-rbdplugin,csi-rbdprovisioner,rook-ceph-mon |
Metadata server ( |
Cluster network |
msgr/msgr2 |
6800 |
Listens on the first available port of the 6800-7300 range |
csi-cephfsplugin,csi-cephfsprovisioner |
Monitor ( |
LCM host network |
msgr/msgr2 |
msgr:3300,
msgr2:6789
|
Monitor has separate ports for |
Ceph clients
rook-ceph-osd,rook-ceph-rgw |
Ceph OSD ( |
Cluster network |
msgr/msgr2 |
6800-7300 |
Binds to the first available port from the 6800-7300 range |
rook-ceph-mon,rook-ceph-mgr,rook-ceph-mds |
Ceph network policies¶
Available since MOSK 24.1
Ceph Controller uses the NetworkPolicy objects for each Ceph daemon.
Each NetworkPolicy is applied to a pod with defined labels in the
rook-ceph namespace. It only allows the use of the ports specified in the
NetworkPolicy spec. Any other port is prohibited.
Ceph daemon |
Pod label |
Allowed ports |
|---|---|---|
Manager ( |
|
6800-7300,
9283
|
Monitor ( |
|
3300,
6789
|
Ceph OSD ( |
|
6800-7300 |
Metadata server ( |
|
6800-7300 |
Ceph Object Storage ( |
|
Values of the following fields in the Ceph cluster CR
spec:objectStorage.rgw.gateway.portobjectStorage.rgw.gateway.securePort |