Ceph¶
Ceph monitors use their node host networks to interact with Ceph daemons. Ceph daemons communicate with each other over a specified cluster network and provide endpoints over the public network.
The messenger V2 (msgr2
) or earlier V1 (msgr
) protocols are used for
communication between Ceph daemons.
Ceph daemon |
Network |
Protocol |
Port |
Description |
Consumers |
---|---|---|---|---|---|
Manager ( |
Cluster network |
msgr/msgr2 |
6800,
9283
|
Listens on the first available port of the 6800-7300 range.
Uses 9283 port for exporting metrics.
|
csi-rbdplugin ,csi-rbdprovisioner ,rook-ceph-mon |
Metadata server ( |
Cluster network |
msgr/msgr2 |
6800 |
Listens on the first available port of the 6800-7300 range |
csi-cephfsplugin ,csi-cephfsprovisioner |
Monitor ( |
LCM host network |
msgr/msgr2 |
msgr:3300,
msgr2:6789
|
Monitor has separate ports for |
Ceph clients
rook-ceph-osd ,rook-ceph-rgw |
Ceph OSD ( |
Cluster network |
msgr/msgr2 |
6800-7300 |
Binds to the first available port from the 6800-7300 range |
rook-ceph-mon ,rook-ceph-mgr ,rook-ceph-mds |
Ceph network policies¶
Available since MOSK 24.1
Ceph Controller uses the NetworkPolicy
objects for each Ceph daemon.
Each NetworkPolicy
is applied to a pod with defined labels in the
rook-ceph
namespace. It only allows the use of the ports specified in the
NetworkPolicy
spec
. Any other port is prohibited.
Ceph daemon |
Pod label |
Allowed ports |
---|---|---|
Manager ( |
|
6800-7300,
9283
|
Monitor ( |
|
3300,
6789
|
Ceph OSD ( |
|
6800-7300 |
Metadata server ( |
|
6800-7300 |
Ceph Object Storage ( |
|
Value from
spec.cephClusterSpec.objectStorage.rgw.gateway.port ,Value from
spec.cephClusterSpec.objectStorage.rgw.gateway.securePort |