Large clusters

This section contains a sample architecture of the MOSK deployment that can handle approximately 10,000 virtual machines. The purpose is to illustrate the control plane hardware sizing and topology.

Hardware characteristics

Node roles layout

Role

Nodes count

Server type

Management cluster Kubernetes nodes

3

c3.small.x86

MOSK cluster Kubernetes master nodes

3

c3.small.x86

Openstack controller nodes

5

m3.large.x86

Openstack compute nodes

up to 500

m3.large.x86

Ceph nodes

3

m3.large.x86

Stacklight nodes

3

m3.large.x86

Server types description

Server type

CPU

RAM in GB

Disks

NICs

c3.small.x86

16 vCPU 3.4GHz

32

2 x 480 GB SSD

2 x 10 Gbps

m3.large.x86

64 vCPU 2.5GHz

256

2 x 240 GB SSD, 2 x 3.8 TB NVME

2 x 25 Gbps

Cluster architecture

Cluster architecture

Configuration

Value

Dedicated StackLight nodes

Yes

Dedicated Ceph OSD nodes

Yes

Dedicated control plane Kubernetes nodes

Yes

Cluster size in the OpenStackDeployment CR

medium

Successfully created OpenStack resources

Note

A successfully created resource has the Active status in the API and passes the functional tests, for example, its floating IP address is accessible. The MOSK cluster is considered to be able to handle the created resources if it successfully performs the LCM operations including the OpenStack services restart, both on the control and data plane.

Note

The key limiting factor for creating more OpenStack objects in this illustrative setup is available hardware resources (vCPU and RAM) on the compute nodes.

Created OpenStack resources

OpenStack resource

Created resources count

Virtual machines instances

11101

OVS ports (avg. per gateway node)

3517

Neutron ports

37337

Volumes

2784

Routers

2448

Networks

3383

Stacks

2419

Hardware resources utilization

Consumed hardware resources by a filled up cluster in the idle state

Node role

Load average

vCPU

RAM in GB

OpenStack control plane with gateway

10

10

100

OpenStack compute

30

25

160

Ceph

2

2

15

StackLight

10

8

102

Kubernetes control plane

10

6

13