Create a managed cluster¶
This section describes how to create an OpenStack-based managed cluster using the Mirantis Container Cloud web UI of the OpenStack-based management cluster.
To create an OpenStack-based managed cluster:
Available since Container Cloud 2.24.0. Optional. Technology Preview. Enable custom host names for cluster machines. When enabled, any machine host name in a particular region matches the related
Machine
object name. For example, instead of the defaultkaas-node-<UID>
, a machine host name will bemaster-0
. The custom naming format is more convenient and easier to operate with.For details, see Configure host names for cluster machines.
If you enabled this feature during management cluster bootstrap, skip this step, as the feature applies to any cluster type.
If you deploy Container Cloud on top of MOSK Victoria with Tungsten Fabric and use the default security group for newly created load balancers, add the following rules for the Kubernetes API server endpoint, Container Cloud application endpoint, and for the MKE web UI and API using the OpenStack CLI:
direction='ingress'
ethertype='IPv4'
protocol='tcp'
remote_ip_prefix='0.0.0.0/0'
port_range_max
andport_range_min
:'443'
for Kubernetes API and Container Cloud application endpoints'6443'
for MKE web UI and API
Log in to the Container Cloud web UI with the
m:kaas:namespace@operator
orm:kaas:namespace@writer
permissions.Switch to the required non-
default
project using the Switch Project action icon located on top of the main left-side navigation panel.To create a project, refer to Create a project for managed clusters.
Optional. In the SSH Keys tab, click Add SSH Key to upload the public SSH key(s) for VMs creation.
In the Credentials tab:
Click Add Credential to add your OpenStack credentials. You can either upload your OpenStack
clouds.yaml
configuration file or fill in the fields manually.Verify that the new credentials status is Ready. If the status is Error, hover over the status to determine the reason of the issue.
Optional. Enable proxy access to the cluster.
In the Proxies tab, configure proxy:
Click Add Proxy.
In the Add New Proxy wizard, fill out the form with the following parameters:
¶ Parameter
Description
Proxy Name
Name of the proxy server to use during cluster creation.
Region Removed in 2.26.0 (16.1.0 and 17.1.0)
From the drop-down list, select the required region.
HTTP Proxy
Add the HTTP proxy server domain name in the following format:
http://proxy.example.com:port
- for anonymous accesshttp://user:password@proxy.example.com:port
- for restricted access
HTTPS Proxy
Add the HTTPS proxy server domain name in the same format as for HTTP Proxy.
No Proxy
Comma-separated list of IP addresses or domain names.
For implementation details, see Proxy and cache support.
If your proxy requires a trusted CA certificate, select the CA Certificate check box and paste a CA certificate for a MITM proxy to the corresponding field or upload a certificate using Upload Certificate.
For the list of Mirantis resources and IP addresses to be accessible from the Container Cloud clusters, see Requirements for an OpenStack-based cluster.
In the Clusters tab, click Create Cluster and fill out the form with the following parameters as required:
Add Cluster name.
Configure general provider settings and the Kubernetes parameters:
¶ Section
Parameter
Description
General Settings
Provider
Select OpenStack.
Provider Credential
From the drop-down list, select the OpenStack credentials name that you have previously created.
Release Version
The Container Cloud version.
Proxy
Optional. From the drop-down list, select the proxy server name that you have previously created.
SSH Keys
From the drop-down list, select the SSH key name(s) that you have previously added for SSH access to VMs.
Container Registry
From the drop-down list, select the Docker registry name that you have previously added using the Container Registries tab. For details, see Define a custom CA certificate for a private Docker registry.
Provider
External Network
Type of the external network in the OpenStack cloud provider.
DNS Name Servers
Comma-separated list of the DNS hosts IPs for the OpenStack VMs configuration.
Configure Bastion
Optional. Configuration parameters for the Bastion node:
Flavor
Image
Availability Zone
Server Metadata
For the parameters description, see Add a machine.
Technology Preview: select Boot From Volume to boot the Bastion node from a block storage volume and select the required amount of storage (80 GB is enough).
Kubernetes
Node CIDR
The Kubernetes nodes CIDR block. For example,
10.10.10.0/24
.Services CIDR Blocks
The Kubernetes Services CIDR block. For example,
10.233.0.0/18
.Pods CIDR Blocks
The Kubernetes Pods CIDR block. For example,
10.233.64.0/18
.Note
The network subnet size of Kubernetes pods influences the number of nodes that can be deployed in the cluster. The default subnet size
/18
is enough to create a cluster with up to 256 nodes. Each node uses the/26
address blocks (64 addresses), at least one address block is allocated per node. These addresses are used by the Kubernetes pods withhostNetwork: false
. The cluster size may be limited further when some nodes use more than one address block.Optional General Settings
Enable Secure Overlay
Experimental, not recommended for production deployments. Removed in Cluster release 16.0.0.
Enable WireGuard for traffic encryption on the Kubernetes workloads network.
WireGuard configuration
Ensure that the Calico MTU size is at least 60 bytes smaller than the interface MTU size of the workload network. IPv4 WireGuard uses a 60-byte header. For details, see Set the MTU size for Calico.
Enable WireGuard by selecting the Enable WireGuard check box.
Caution
Changing this parameter on a running cluster causes a downtime that can vary depending on the cluster size.
For more details about WireGuard, see Calico documentation: Encrypt in-cluster pod traffic.
Parallel Upgrade Of Worker Machines
Available since the Cluster release 16.0.0.
The maximum number of the worker nodes to update simultaneously. It serves as an upper limit on the number of machines that are drained at a given moment of time. Defaults to
1
.You can configure this option after deployment before the cluster update.
Parallel Preparation For Upgrade Of Worker Machines
Available since the Cluster release 16.0.0.
The maximum number of worker nodes being prepared at a given moment of time, which includes downloading of new artifacts. It serves as a limit for the network load that can occur when downloading the files to the nodes. Defaults to
50
.You can configure this option after deployment before the cluster update.
Configure StackLight:
Section
Parameter name
Description
StackLight
Enable Monitoring
Selected by default. Deselect to skip StackLight deployment. You can also enable, disable, or configure StackLight parameters after deploying a managed cluster. For details, see Change a cluster configuration or Configure StackLight.
Enable Logging
Select to deploy the StackLight logging stack.
For details about the logging components, see Deployment architecture.
Note
The logging mechanism performance depends on the cluster log load. In case of a high load, you may need to increase the default resource requests and limits for
fluentdLogs
. For details, see StackLight configuration parameters: Resource limits.HA Mode
Select to enable StackLight monitoring in the HA mode. For the differences between HA and non-HA modes, see Deployment architecture.
StackLight Default Logs Severity Level
Log severity (verbosity) level for all StackLight components. The default value for this parameter is Default component log level that respects original defaults of each StackLight component. For details about severity levels, see Log verbosity.
StackLight Component Logs Severity Level
The severity level of logs for a specific StackLight component that overrides the value of the StackLight Default Logs Severity Level parameter. For details about severity levels, see Log verbosity.
Expand the drop-down menu for a specific component to display its list of available log levels.
OpenSearch
Logstash Retention Time
Skip this parameter since Container Cloud 2.26.0 (17.1.0, 16.1.0). It was removed from the code base and will be removed from the web UI in one of the following releases.
Available if you select Enable Logging. Specifies the
logstash-*
index retention time.Events Retention Time
Available if you select Enable Logging. Specifies the
kubernetes_events-*
index retention time.Notifications Retention
Available if you select Enable Logging. Specifies the
notification-*
index retention time and is used for Mirantis OpenStack for Kubernetes.Persistent Volume Claim Size
Available if you select Enable Logging. The OpenSearch persistent volume claim size.
Collected Logs Severity Level
Available if you select Enable Logging. The minimum severity of all Container Cloud components logs collected in OpenSearch. For details about severity levels, see Logging.
Prometheus
Retention Time
The Prometheus database retention period.
Retention Size
The Prometheus database retention size.
Persistent Volume Claim Size
The Prometheus persistent volume claim size.
Enable Watchdog Alert
Select to enable the Watchdog alert that fires as long as the entire alerting pipeline is functional.
Custom Alerts
Specify alerting rules for new custom alerts or upload a YAML file in the following exemplary format:
- alert: HighErrorRate expr: job:request_latency_seconds:mean5m{job="myjob"} > 0.5 for: 10m labels: severity: page annotations: summary: High request latency
For details, see Official Prometheus documentation: Alerting rules. For the list of the predefined StackLight alerts, see Operations Guide: Available StackLight alerts.
StackLight Email Alerts
Enable Email Alerts
Select to enable the StackLight email alerts.
Send Resolved
Select to enable notifications about resolved StackLight alerts.
Require TLS
Select to enable transmitting emails through TLS.
Email alerts configuration for StackLight
Fill out the following email alerts parameters as required:
To - the email address to send notifications to.
From - the sender address.
SmartHost - the SMTP host through which the emails are sent.
Authentication username - the SMTP user name.
Authentication password - the SMTP password.
Authentication identity - the SMTP identity.
Authentication secret - the SMTP secret.
StackLight Slack Alerts
Enable Slack alerts
Select to enable the StackLight Slack alerts.
Send Resolved
Select to enable notifications about resolved StackLight alerts.
Slack alerts configuration for StackLight
Fill out the following Slack alerts parameters as required:
API URL - The Slack webhook URL.
Channel - The channel to send notifications to, for example, #channel-for-alerts.
StackLight optional settings
Enable Reference Application
Deprecated since Container Cloud 2.28.0 (16.3.0). Available since 2.22.0 (11.6.0). Enables Reference Application that is a small microservice application that enables workload monitoring on non-MOSK managed clusters.
Note
For the feature support on MOSK deployments, refer to MOSK documentation: Deploy your first cloud application using automation.
Disabled by default. You can also enable this option after deployment from the Configure cluster menu.
Available since Container Cloud 2.24.0 and 2.24.2 for MOSK 23.2. Optional. Technology Preview. Enable the Linux Audit daemon auditd to monitor activity of cluster processes and prevent potential malicious activity.
Configuration for auditd
In the
Cluster
object, add the auditd parameters:spec: providerSpec: value: audit: auditd: enabled: <bool> enabledAtBoot: <bool> backlogLimit: <int> maxLogFile: <int> maxLogFileAction: <string> maxLogFileKeep: <int> mayHaltSystem: <bool> presetRules: <string> customRules: <string> customRulesX32: <text> customRulesX64: <text>
Configuration parameters for auditd:
enabled
Boolean, default -
false
. Enables theauditd
role to install the auditd packages and configure rules. CIS rules: 4.1.1.1, 4.1.1.2.enabledAtBoot
Boolean, default -
false
. Configures grub to audit processes that can be audited even if they start up prior to auditd startup. CIS rule: 4.1.1.3.backlogLimit
Integer, default - none. Configures the backlog to hold records. If during boot
audit=1
is configured, the backlog holds 64 records. If more than 64 records are created during boot, auditd records will be lost with a potential malicious activity being undetected. CIS rule: 4.1.1.4.maxLogFile
Integer, default - none. Configures the maximum size of the audit log file. Once the log reaches the maximum size, it is rotated and a new log file is created. CIS rule: 4.1.2.1.
maxLogFileAction
String, default - none. Defines handling of the audit log file reaching the maximum file size. Allowed values:
keep_logs
- rotate logs but never delete themrotate
- add a cron job to compress rotated log files and keep maximum 5 compressed files.compress
- compress log files and keep them under the/var/log/auditd/
directory. Requiresauditd_max_log_file_keep
to be enabled.
CIS rule: 4.1.2.2.
maxLogFileKeep
Integer, default -
5
. Defines the number of compressed log files to keep under the/var/log/auditd/
directory. Requiresauditd_max_log_file_action=compress
. CIS rules - none.mayHaltSystem
Boolean, default -
false
. Halts the system when the audit logs are full. Applies the following configuration:space_left_action = email
action_mail_acct = root
admin_space_left_action = halt
CIS rule: 4.1.2.3.
customRules
String, default - none. Base64-encoded content of the
60-custom.rules
file for any architecture. CIS rules - none.customRulesX32
String, default - none. Base64-encoded content of the
60-custom.rules
file for thei386
architecture. CIS rules - none.customRulesX64
String, default - none. Base64-encoded content of the
60-custom.rules
file for thex86_64
architecture. CIS rules - none.presetRules
String, default - none. Comma-separated list of the following built-in preset rules:
access
actions
delete
docker
identity
immutable
logins
mac-policy
modules
mounts
perm-mod
privileged
scope
session
system-locale
time-change
Since Container Cloud 2.28.0 (Cluster releases 17.3.0 and 16.3.0) in the Technology Preview scope, you can collect some of the preset rules indicated above as groups and use them in
presetRules
:ubuntu-cis-rules
- this group contains rules to comply with the Ubuntu CIS Benchmark recommendations, including the following CIS Ubuntu 20.04 v2.0.1 rules:scope
- 5.2.3.1actions
- same as 5.2.3.2time-change
- 5.2.3.4system-locale
- 5.2.3.5privileged
- 5.2.3.6access
- 5.2.3.7identity
- 5.2.3.8
perm-mod
- 5.2.3.9mounts
- 5.2.3.10session
- 5.2.3.11logins
- 5.2.3.12delete
- 5.2.3.13mac-policy
- 5.2.3.14modules
- 5.2.3.19
docker-cis-rules
- this group contains rules to comply with Docker CIS Benchmark recommendations, including thedocker
Docker CIS v1.6.0 rules 1.1.3 - 1.1.18.
You can also use two additional keywords inside
presetRules
:none
- select no built-in rules.all
- select all built-in rules. When using this keyword, you can add the!
prefix to a rule name to exclude some rules. You can use the!
prefix for rules only if you add theall
keyword as the first rule. Place a rule with the!
prefix only after theall
keyword.
Example configurations:
presetRules: none
- disable all preset rulespresetRules: docker
- enable only thedocker
rulespresetRules: access,actions,logins
- enable only theaccess
,actions
, andlogins
rulespresetRules: ubuntu-cis-rules
- enable all rules from theubuntu-cis-rules
grouppresetRules: docker-cis-rules,actions
- enable all rules from thedocker-cis-rules
group and theactions
rulepresetRules: all
- enable all preset rulespresetRules: all,!immutable,!sessions
- enable all preset rules exceptimmutable
andsessions
CIS controls
4.1.3 (time-change
)4.1.4 (identity
)4.1.5 (system-locale
)4.1.6 (mac-policy
)4.1.7 (logins
)4.1.8 (session
)4.1.9 (perm-mod
)4.1.10 (access
)4.1.11 (privileged
)4.1.12 (mounts
)4.1.13 (delete
)4.1.14 (scope
)4.1.15 (actions
)4.1.16 (modules
)4.1.17 (immutable
)Docker CIS controls
1.1.41.1.81.1.101.1.121.1.131.1.151.1.161.1.171.1.181.2.31.2.41.2.51.2.61.2.71.2.101.2.11
Click Create.
To monitor the cluster readiness, hover over the status icon of a specific cluster in the Status column of the Clusters page.
Once the orange blinking status icon becomes green and Ready, the cluster deployment or update is complete.
You can monitor live deployment status of the following cluster components:
Component
Description
Bastion
For the OpenStack-based management clusters, the Bastion node IP address status that confirms the Bastion node creation
Helm
Installation or upgrade status of all Helm releases
Kubelet
Readiness of the node in a Kubernetes cluster, as reported by kubelet
Kubernetes
Readiness of all requested Kubernetes objects
Nodes
Equality of the requested nodes number in the cluster to the number of nodes having the
Ready
LCM statusOIDC
Readiness of the cluster OIDC configuration
StackLight
Health of all StackLight-related objects in a Kubernetes cluster
Swarm
Readiness of all nodes in a Docker Swarm cluster
LoadBalancer
Readiness of the Kubernetes API load balancer
ProviderInstance
Readiness of all machines in the underlying infrastructure (virtual or bare metal, depending on the provider type)
Graceful Reboot
Readiness of a cluster during a scheduled graceful reboot, available since Cluster releases 15.0.1 and 14.0.0.
Infrastructure Status
Available since Container Cloud 2.25.0 for bare metal and OpenStack providers. Readiness of the following cluster components:
Bare metal: the
MetalLBConfig
object along with MetalLB and DHCP subnets.OpenStack: cluster network, routers, load balancers, and Bastion along with their ports and floating IPs.
LCM Operation
Available since Container Cloud 2.26.0 (Cluster releases 17.1.0 and 16.1.0). Health of all LCM operations on the cluster and its machines.
LCM Agent
Available since Container Cloud 2.27.0 (Cluster releases 17.2.0 and 16.2.0). Health of all LCM agents on cluster machines and the status of LCM agents update to the version from the current Cluster release.
For the history of a cluster deployment or update, refer to Inspect the history of a cluster and machine deployment or update.
Available since Container Cloud 2.24.0 and 2.24.2 for MOSK 23.2. Optional. Technology Preview. Enable the Linux Audit daemon auditd to monitor activity of cluster processes and prevent potential malicious activity.
Configuration for auditd
In the
Cluster
object, add the auditd parameters:spec: providerSpec: value: audit: auditd: enabled: <bool> enabledAtBoot: <bool> backlogLimit: <int> maxLogFile: <int> maxLogFileAction: <string> maxLogFileKeep: <int> mayHaltSystem: <bool> presetRules: <string> customRules: <string> customRulesX32: <text> customRulesX64: <text>
Configuration parameters for auditd:
enabled
Boolean, default -
false
. Enables theauditd
role to install the auditd packages and configure rules. CIS rules: 4.1.1.1, 4.1.1.2.enabledAtBoot
Boolean, default -
false
. Configures grub to audit processes that can be audited even if they start up prior to auditd startup. CIS rule: 4.1.1.3.backlogLimit
Integer, default - none. Configures the backlog to hold records. If during boot
audit=1
is configured, the backlog holds 64 records. If more than 64 records are created during boot, auditd records will be lost with a potential malicious activity being undetected. CIS rule: 4.1.1.4.maxLogFile
Integer, default - none. Configures the maximum size of the audit log file. Once the log reaches the maximum size, it is rotated and a new log file is created. CIS rule: 4.1.2.1.
maxLogFileAction
String, default - none. Defines handling of the audit log file reaching the maximum file size. Allowed values:
keep_logs
- rotate logs but never delete themrotate
- add a cron job to compress rotated log files and keep maximum 5 compressed files.compress
- compress log files and keep them under the/var/log/auditd/
directory. Requiresauditd_max_log_file_keep
to be enabled.
CIS rule: 4.1.2.2.
maxLogFileKeep
Integer, default -
5
. Defines the number of compressed log files to keep under the/var/log/auditd/
directory. Requiresauditd_max_log_file_action=compress
. CIS rules - none.mayHaltSystem
Boolean, default -
false
. Halts the system when the audit logs are full. Applies the following configuration:space_left_action = email
action_mail_acct = root
admin_space_left_action = halt
CIS rule: 4.1.2.3.
customRules
String, default - none. Base64-encoded content of the
60-custom.rules
file for any architecture. CIS rules - none.customRulesX32
String, default - none. Base64-encoded content of the
60-custom.rules
file for thei386
architecture. CIS rules - none.customRulesX64
String, default - none. Base64-encoded content of the
60-custom.rules
file for thex86_64
architecture. CIS rules - none.presetRules
String, default - none. Comma-separated list of the following built-in preset rules:
access
actions
delete
docker
identity
immutable
logins
mac-policy
modules
mounts
perm-mod
privileged
scope
session
system-locale
time-change
Since Container Cloud 2.28.0 (Cluster releases 17.3.0 and 16.3.0) in the Technology Preview scope, you can collect some of the preset rules indicated above as groups and use them in
presetRules
:ubuntu-cis-rules
- this group contains rules to comply with the Ubuntu CIS Benchmark recommendations, including the following CIS Ubuntu 20.04 v2.0.1 rules:scope
- 5.2.3.1actions
- same as 5.2.3.2time-change
- 5.2.3.4system-locale
- 5.2.3.5privileged
- 5.2.3.6access
- 5.2.3.7identity
- 5.2.3.8
perm-mod
- 5.2.3.9mounts
- 5.2.3.10session
- 5.2.3.11logins
- 5.2.3.12delete
- 5.2.3.13mac-policy
- 5.2.3.14modules
- 5.2.3.19
docker-cis-rules
- this group contains rules to comply with Docker CIS Benchmark recommendations, including thedocker
Docker CIS v1.6.0 rules 1.1.3 - 1.1.18.
You can also use two additional keywords inside
presetRules
:none
- select no built-in rules.all
- select all built-in rules. When using this keyword, you can add the!
prefix to a rule name to exclude some rules. You can use the!
prefix for rules only if you add theall
keyword as the first rule. Place a rule with the!
prefix only after theall
keyword.
Example configurations:
presetRules: none
- disable all preset rulespresetRules: docker
- enable only thedocker
rulespresetRules: access,actions,logins
- enable only theaccess
,actions
, andlogins
rulespresetRules: ubuntu-cis-rules
- enable all rules from theubuntu-cis-rules
grouppresetRules: docker-cis-rules,actions
- enable all rules from thedocker-cis-rules
group and theactions
rulepresetRules: all
- enable all preset rulespresetRules: all,!immutable,!sessions
- enable all preset rules exceptimmutable
andsessions
CIS controls
4.1.3 (time-change
)4.1.4 (identity
)4.1.5 (system-locale
)4.1.6 (mac-policy
)4.1.7 (logins
)4.1.8 (session
)4.1.9 (perm-mod
)4.1.10 (access
)4.1.11 (privileged
)4.1.12 (mounts
)4.1.13 (delete
)4.1.14 (scope
)4.1.15 (actions
)4.1.16 (modules
)4.1.17 (immutable
)Docker CIS controls
1.1.41.1.81.1.101.1.121.1.131.1.151.1.161.1.171.1.181.2.31.2.41.2.51.2.61.2.71.2.101.2.11
Proceed with Add a machine.
See also