Calculate target ratio for Ceph pools¶
Ceph pool target ratio defines for the Placement Group (PG) autoscaler the amount of data the pools are expected to acquire over time in relation to each other. You can set initial PG values for each Ceph pool. Otherwise, the autoscaler starts with the minimum value and scales up, causing a lot of data to move in the background.
You can allocate several pools to use the same device class, which is a solid
block of available capacity in Ceph. For example, if three pools
volumes-hdd) are set to use the
same device class
hdd, you can set the target ratio for Ceph pools to
provide 80% of capacity to the
volumes-hdd pool and distribute the
remaining capacity evenly between the two other pools. This way, Ceph pool
target ratio instructs Ceph on when to warn that a pool is running out of free
space and, at the same time, instructs Ceph on how many placement groups Ceph
should allocate/autoscale for a pool for better data distribution.
Ceph pool target ratio is not a constant value and you can change it according to new capacity plans. Once you specify target ratio, if the PG number of a pool scales, other pools with specified target ratio will automatically scale accordingly.
For details, see Ceph Documentation: Autoscaling Placement Groups.
To calculate target ratio for each Ceph pool:
Define raw capacity of the entire storage by device class:
kubectl -n rook-ceph exec -it $(kubectl -n rook-ceph get pod -l "app=rook-ceph-tools" -o name) -- ceph df
For illustration purposes, the procedure below uses raw capacity of 185 TB or 189440 GB.
Design Ceph pools with the considered device class upper bounds of the possible capacity. For example, consider the
hdddevice class that contains the following pools:
kubernetes-hddpool will contain not more than 2048 GB.
stacklight-hddpool will contain not more than 100000 GB.
dataPoolis replicated and Ceph Object Store is planned for intensive use, also calculate upper bounds for
Calculate target ratio for each considered pool. For example:
Pools upper bounds
kubernetes-hdd= 2048 GB
stacklight-hdd= 100000 GB
Summary capacity = 102048 GB
Total raw capacity = 189440 GB
Calculate pools fit factor using the (total raw capacity) / (pools summary capacity) formula. For example:
pools fit factor = 189440 / 102048 = 1.8563
Calculate pools upper bounds size using the (pool upper bounds) * (pools fit factor) formula. For example:
kubernetes-hdd = 2048 GB * 1.8563 = 3801.7024 GB stacklight-hdd = 100000 GB * 1.8563 = 185630 GB
Calculate pool target ratio using the (pool upper bounds) * 100 / (total raw capacity) formula. For example:
kubernetes-hdd = 3801.7024 GB * 100 / 189440 GB = 2.007 stacklight-hdd = 185630 GB * 100 / 189440 GB = 97.989
KaasCephClusterCR of a managed cluster for editing:
kubectl edit kaascephcluster -n <managedClusterProjectName>
<managedClusterProjectName>with the corresponding value.
spec.cephClusterSpec.poolssection, specify the calculated relatives as
targetSizeRatiofor each considered pool:
spec: cephClusterSpec: pools: - name: kubernetes deviceClass: hdd ... replicated: size: 3 targetSizeRatio: 2.007 - name: stacklight deviceClass: hdd ... replicated: size: 3 targetSizeRatio: 97.989
If Ceph Object Store
replicatedand a proper value is calculated, also specify it:
spec: cephClusterSpec: objectStorage: rgw: name: rgw-store ... dataPool: ... replicated: deviceClass: hdd size: 3 targetSizeRatio: <relative>
Verify that all target ratio has been successfully applied to the Ceph cluster:
kubectl -n rook-ceph exec -it $(kubectl -n rook-ceph get pod -l "app=rook-ceph-tools" -o name) -- ceph osd pool autoscale-status
Example of system response:
POOL SIZE TARGET SIZE RATE RAW CAPACITY RATIO TARGET RATIO EFFECTIVE RATIO BIAS PG_NUM NEW PG_NUM AUTOSCALE device_health_metrics 0 2.0 149.9G 0.0000 1.0 1 on kubernetes-hdd 2068 2.0 149.9G 0.0000 2.007 2.0076 1.0 32 on stacklight-hdd 2068 2.0 149.9G 0.0000 97.989 97.9894 1.0 256 on
Optional. Repeat the steps above for other device classes.