Host operating system disk layout
This section consolidates the host operating system (OS) disk layout for
Mirantis OpenStack for Kubernetes (MOSK) nodes by role. The layout is applied
during node provisioning using the BareMetalHostProfile object. Different
node roles require different disks and partitions. Use a dedicated host profile
per role rather than one profile for every machine.
Depending on the cluster type, apply the layout using the respective file:
- Management cluster
Edit
templates/bm/baremetalhostprofiles.yaml.templatebefore bootstrap. For reference, see Customize the default bare-metal host profile.
- MOSK cluster
Create one
BareMetalHostProfileper role and attach it from theMachineobject. For reference, see Create MOSK host profiles.
See also
Node roles
The following table maps node roles covered in this section to node types in the reference architecture.
When roles are collocated, add the disks of every collocated role and
increase reserved_host_* for the compute service. For details, see
Components collocation.
StackLight is always deployed. On a MOSK cluster, StackLight nodes are
selected using the stacklight: enabled label in Machine objects. For
details, see Add a machine and Logging, monitoring, and alerting. Placement decides
which nodes require a StackLight Local Volume Provisioner (LVP) disk:
Management cluster. StackLight always runs in HA mode on all three manager nodes, and no label is required. Size the management LVP disk to cover both the StackLight volumes and the management services.
MOSK cluster, HA mode (recommended). At least three nodes must carry the label. Otherwise, StackLight does not deploy. Label either three OpenStack control plane nodes, which then require a StackLight LVP disk in addition to the OpenStack LVP one, or three or more workers that are not OpenStack controllers, in which case only those workers require the disk.
MOSK cluster, non-HA mode. One labeled node is enough. If no node carries the label, StackLight spreads across all worker nodes, so every worker requires a StackLight LVP disk.
Caution
Do not remove the stacklight: enabled label from a node that
still contains StackLight local volumes without migrating the workloads
first. For details, see Deschedule StackLight Pods from a worker machine.
Common layout components
Every node role is assembled from the same components. The default generic
profile expects at least two devices: the system disk, which carries the
boot, cloud-init, and root partitions, and the LVP disk. Nodes in the
storage role take an additional clean raw disk for Ceph OSD. Declare the
devices in the host profile in the order of the Device column below. For
reference, see Default configuration of the host system storage.
The following rules and recommendations apply to every node role:
Identify disks by WWN,
by-path, or serial, not by/dev/sdXnames. Kernel names can change across reboots. For details, see Provisioning failure due to device naming issues in a bare-metal host profile.Do not mount the entire
/vardirectory on a separate disk. That breakscloud-initduring the first host boot. Mount more specific paths such as/var/lib/novainstead. For details, see Mounting recommendations for the /var directory.size: 0on a partition or logical volume consumes all remaining space in the parent device or volume group. Flexible units areKi,Mi,Gi. For details, see the description of thesizeunits in Create MOSK host profiles.On the management cluster, a RAID device is not supported for the first (boot) disk. For details, see Management cluster storage and Configure RAID support.
SSD or NVMe is recommended for the system and LVP disks. HDD is acceptable for Ceph OSD data, with SSD/NVMe for OSD metadata if devices are split.
System disk (boot and root)
The system disk is always the first device in the profile. The recommended media is SSD or NVMe.
The small boot partitions (bios_grub, ESP, config-2) are the same
for every role. The remaining space becomes the root LVM volume group unless
you define additional partitions in the host profile.
Note
For the validated large-cluster build, see Large clusters for the full per-node server specification.
Local Volume Provisioner (LVP)
LVP is a CSI driver deployed as a DaemonSet. It creates local persistent
volumes under /mnt/local-volumes from an ext4 file system on an LVM
volume named lvp. Networked Ceph is a poor fit for latency-sensitive
workloads such as MariaDB, so those services use LVP instead. For details,
see OpenStack cluster.
LVP capacity is only required where LVP consumers are scheduled, not on every node. Provision the LVP disk on each node role according to the consumers it hosts.
Cluster / role |
Consumers |
Comments |
|---|---|---|
Management cluster |
|
One LVP disk per manager. Recommended: 960 GB, minimum: 480 GB. The default |
OpenStack control plane |
|
Dedicated 1 × SSD 1000 GB for OpenStack LVP. |
OpenStack control plane in a Cephless cloud |
Tempest results ( |
Only when no Ceph-backed storage class is available. For details, see Cephless cloud. |
StackLight HA |
|
Dedicated 1 × SSD 1000 GB for StackLight LVP when StackLight is collocated with the OpenStack control plane. Requires four volumes per node. For details, see MOSK cluster hardware requirements. For production deployments, increase Prometheus and OpenSearch PVC sizes according to the guidance in OpenSearch and Prometheus storage sizing. In HA mode, all StackLight PVCs on a node share the same LVP pool and use the whole disk as available capacity. |
You can spawn LVP on top of several physical disks by adding multiple PVs to
one lvm_lvp volume group. See the example with multiple PVs in
Create MOSK host profiles.
Unpartitioned disks for Ceph
For each OSD data device in BareMetalHostProfile:
List the device with
wipe: true.Do not create partitions or a file system. The only exception is the dedicated metadata device described below.
Leave the device raw for Rook and Ceph to consume during deployment.
Mirantis strongly recommends defining OSD metadata partitions in the host profile as well. Configure a separate metadata device as follows:
One partition using the whole metadata disk (
size: 0).One volume group (for example,
bluedb).One logical volume per OSD on the node (for example,
meta_1,meta_2, and so on).Size of each logical volume: 1–4% of OSD data per the general Ceph recommendation, but Mirantis recommends at least 4%. If RADOS Gateway is enabled, 4% is the required minimum.
For more details, see Configure Ceph disks in a host profile and Configure a separate metadata device for Ceph OSD.
For the reference disk configuration of a dedicated Ceph node, see Storage node (Ceph).
Partitions for compute and other OpenStack services
These partitions are optional and applicable only to the roles that require them.
Purpose |
Host-profile object |
Typical size |
When to use |
|---|---|---|---|
Nova local images
(
qcow2 or raw backend) |
Dedicated disk or partition mounted at |
Extra capacity beyond the 500 GB operating system disk; depends on image and instance size. A dedicated NVMe normally receives a single partition spanning the whole device. |
Local instance root and ephemeral disks instead of Ceph. Mount
|
Nova LVM ephemeral
TechPreview
|
Partition |
Whole NVMe, or a 100 GiB partition on the system disk |
When |
Cinder LVM block storage
TechPreview
|
Partition |
Whole dedicated device, such as an NVMe |
Usually on compute nodes. Dedicated nodes are allowed. For details, see Enable LVM block storage. |
OpenStack MariaDB, messaging, other control plane PVCs |
LVP ( |
1000 GB SSD |
Control plane nodes. The data lives in LVP PVs. |
Glance / Cinder RBD |
Ceph pools, not host partitions |
Sized in Ceph, not in the host profile |
Default shared storage path. |
Layout by node role
Management cluster node
A management cluster contains three physical manager nodes. Adding more than three nodes is not supported.
Disk requirements:
Disk role |
Size |
Layout |
|---|---|---|
|
Recommended: 240 GB
Minimum: 120 GB NVMe or SSD
|
Standard boot and root partitions. RAID is not supported on this device. |
|
Recommended: 960 GB
Minimum: 480 GB NVMe or SSD
|
Entire disk → |
Note
A Ceph cluster is never deployed on the management cluster. Therefore, do not allocate Ceph OSD disks in the management host profile.
Provision the management cluster LVP pool to cover the following volumes:
Two volumes for management services, 50 GB in total
StackLight HA volumes, the same four volumes as on a MOSK StackLight node
mcc-cache, 20 GiB by default and 50–120 GiB as recommendedA volume for IAM MariaDB backups, mounted at
/mnt/local-volumes/iam/kaas-iam-backup/
The exact capacity depends on the StackLight, MariaDB, and mcc-cache volume
sizes. For details, see Management cluster storage and the disk0 and disk1
requirements in MOSK cluster hardware requirements.
MOSK Kubernetes manager node
Dedicated Kubernetes control plane of a MOSK cluster, comprising three nodes. Skip this node role if the compact control plane is used.
Disk requirements:
- System disk
1 × SSD or NVMe 500 GB.
- LVP
OpenStack MariaDB and StackLight HA are not scheduled on this node role when the control plane and StackLight are dedicated. Still, a second SSD matching the management cluster LVP pattern (480–960 GB) is reasonable if local PVs are required.
- Ceph OSD disks
None, unless storage is collocated.
OpenStack control plane node
An OpenStack control plane node hosts the OpenStack control plane (database, messaging, API, schedulers, conductors), gateway services, and StackLight. For labels required for this node role, see Node roles and selection.
Disk requirements:
Disk role |
Size |
Layout |
|---|---|---|
Operating system |
1 × SSD 500 GB |
Standard boot and root partitions. |
OpenStack LVP |
1 × SSD 1000 GB |
|
StackLight LVP |
1 × SSD 1000 GB if StackLight is collocated |
A separate disk from the OpenStack LVP one. Provides four volumes per node with the default sizes of 2, 10, 16, and 30 GB. For production, increase the Prometheus and OpenSearch volumes as described in OpenSearch and Prometheus storage sizing. For the reference disk configuration of this node role, see MOSK cluster hardware requirements. |
Ceph OSD |
None, unless collocated |
For details, see Control plane + storage. |
If StackLight is configured on dedicated nodes, omit the StackLight LVP disk on this node role and provision it on the StackLight nodes instead.
If gateway services are moved to dedicated tenant gateway nodes, this node role keeps the operating system and OpenStack LVP disks. Gateway nodes do not require a second LVP disk as per the hardware table.
In the large cluster blueprint, OpenStack controllers use 1 x NVMe or SSD 500 GB and 2 × NVMe 3.8 TB. Use NVMe for LVP and databases, not for the host operating system.
Compact control plane (manager + OpenStack control plane)
A compact control plane is available as a technical preview for small clouds. The Kubernetes manager, OpenStack control plane, gateway, and Open vSwitch run on the same three hosts. Do not use this topology for production-scale clouds. For labels required for each machine, see Node roles and selection.
Disk requirements:
Sum the disks of the Kubernetes manager and OpenStack control plane node roles, including StackLight if it is collocated on these nodes:
- System disk
At least the OpenStack control plane 500 GB SSD. Do not use the smaller management-only system disk of 120–240 GB.
- OpenStack LVP
1000 GB SSD.
- StackLight LVP
1000 GB SSD if StackLight is collocated.
- Ceph OSD disks
Only if storage is collocated.
Treat this node role as the most storage-dense control node: the root file system, OpenStack databases, and, typically, StackLight share the same host. Still, keep LVP for OpenStack and StackLight on separate disks as in the non-compact reference architecture.
Compute node
A compute node hosts the OpenStack compute plane (Nova, Cinder, Neutron), Open vSwitch, and custom Kubernetes workloads. For labels required for this node role, see Node roles and selection.
Disk requirements:
Disk role |
Size |
Layout |
|---|---|---|
Operating system |
1 × SSD 500 GB |
Standard boot and root partitions. Add an extra partition on this disk
only if Nova LVM ephemeral storage is placed there, for example,
|
LVP |
Not required as per the hardware table |
Required only if custom Kubernetes workloads that consume LVP are scheduled on the node. |
Nova local images |
Depends on the image and instance size |
Dedicated SSD or NVMe mounted at |
Nova LVM ephemeral TechPreview |
Entire NVMe or a large partition |
The |
Cinder LVM TechPreview |
Dedicated device |
The |
Ceph OSD |
None on a dedicated compute node |
For details, see Compute + storage (hyperconverged). |
When Nova uses local folders, add capacity for instance images on top of the 500 GB operating system disk.
For NVMe local storage, define the device in BareMetalHostProfile before
setting features:nova:images:backend to qcow2 or raw in the
OpenStackDeployment custom resource. For example:
fileSystems:
- fileSystem: ext4
partition: local-volumes-partition
mountPoint: /var/lib/nova
mountOpts: 'rw,noatime,nodiratime,lazytime'
Storage node (Ceph)
A storage node hosts the Ceph storage cluster. For labels required for this node role, see Node roles and selection. At least three nodes are required. For production deployments, use at least five nodes.
Disk requirements:
Disk role |
Size |
Layout |
|---|---|---|
Operating system |
1 × SSD 500 GB |
Standard boot and root partitions. |
LVP |
Not required |
Ceph OSDs do not consume LVP. |
OSD data |
Reference: 2 × HDD 2000 GB |
Entire disks, wiped and unpartitioned. |
OSD metadata (recommended) |
≥ 4% of OSD data per OSD |
Separate SSD or NVMe device partitioned into one logical volume per OSD. |
Note
A three-node Ceph cluster provides no hardware fault tolerance and is not eligible for recovery operations such as a disk or an entire node replacement. With the replication factor of 3, a cluster with fewer than three alive Ceph OSDs moves to the degraded state and restricts write operations until the number of alive OSDs is restored.
In the large cluster blueprint, compute and storage are mixed on the same servers as described in Compute + storage (hyperconverged), instead of using dedicated storage nodes.
Compute + storage (hyperconverged)
In hyperconverged OpenStack compute nodes, the same host acts as a Nova
hypervisor and a Ceph OSD node. To avoid overloading the host, sum the CPU,
RAM, NIC, and disk requirements of both roles and increase the
reserved_host_* parameters of the compute service for Ceph. For
details, see Limit HW resources for hyperconverged OpenStack compute nodes.
Disk requirements:
- System disk
1 x NVMe or SSD 500 GB.
- Nova local images, Nova LVM ephemeral, or Cinder LVM
Same as for the compute node role if the local instance or LVM backends are used. Keep these devices separate from the OSD disks.
- OSD data
Wiped raw disks. The large cluster blueprint uses 2 × NVMe 3.8 TB.
- OSD metadata
A separate device or logical volumes if WAL and DB are split from data.
- LVP
Typically unused unless custom Kubernetes workloads are placed on the node.
Caution
Do not place /var/lib/nova and Ceph OSDs on the same device.
Control plane + storage
This node role hosts the OpenStack control plane, typically along with the gateway services and StackLight, collocated with Ceph OSDs. Sum the control plane disks and the Ceph OSD disks.
Disk requirements:
- Operating system
500 GB SSD.
- OpenStack LVP
1000 GB SSD.
- StackLight LVP
1000 GB SSD if StackLight is collocated on these nodes.
- OSD data
Wiped raw disks. Reference configuration is 2 × HDD 2000 GB, or NVMe devices in a large cluster design.
- OSD metadata
Recommended. Separate SSD or NVMe logical volumes.
Caution
Keep LVP (databases, logs, metrics) and OSD data on different devices. Ceph recovery I/O on the same disk as MariaDB or OpenSearch starves the control plane.
Other MOSK cluster node roles
Tenant gateway (optional, dedicated)
A tenant gateway node is used when the Neutron L3, DHCP, and related agents are moved off the OpenStack control plane nodes. Up to three such nodes are supported. These nodes require a dedicated physical network (bond) for tenant traffic, which is a network constraint rather than a disk one.
Disk requirements:
- System disk
1 × SSD 500 GB for boot and root partitions only.
- LVP
Not required as per the hardware table.
Dedicated StackLight nodes
Used when StackLight is not collocated with the OpenStack control plane
nodes. HA mode requires at least three nodes with the
stacklight: enabled label.
Disk requirements:
- System disk
1 x NVMe or SSD 500 GB.
- StackLight LVP
1 × SSD 1000 GB, or 2 × NVMe 3.8 TB for data in the large cluster blueprint. Provisions four default volumes. For production, increase the Prometheus and OpenSearch volumes as described in OpenSearch and Prometheus storage sizing.
In non-HA mode, StackLight can spread across the worker nodes of a MOSK cluster
if no node has the stacklight: enabled label. HA mode always requires the
label, and the local volumes are bound to the labeled machines. For details,
see StackLight database modes.
Bootstrap (seed) node
The bootstrap node is not a cluster machine and is not provisioned using
BareMetalHostProfile. It is required only to bootstrap the management
cluster. Afterwards, you can reuse it as any node role.
It requires minimum resources: Ubuntu 24.04, 8 GB RAM, 4 CPUs, and 30 GB of
free disk space for the bootstrap cache, typically /var/lib/docker. For
details, see System requirements for the seed node.
Disk size cheat sheet
The values below are the documented reference values, not a capacity plan for a specific cloud. Scale the StackLight and Nova local disks according to the workload.
The table lists the values for the default cluster sizing. The large cluster blueprint uses a different set of disks, for example, 1 x NVMe or SSD 500 GB for the system disk and 2 × NVMe 3.8 TB for data. For details, see Large clusters.
The default StackLight LVP volume set per node is 2 + 10 + 16 + 30 GB. The
management cluster LVP additionally includes about 50 GB of management volumes
and the mcc-cache volume.