Cluster resource¶

This section outlines the Cluster resource used the in MOSK that describes the cluster-level parameters.

For demonstration purposes, the Cluster custom resource (CR) is split into the following major sections:

metadata
spec:providerSpec
spec:providerSpec common
spec:providerSpec configuration
status:providerStatus common
status:providerStatus for cluster readiness
status:providerStatus for Open ID Connect
status:providerStatus for cluster releases

Warning

The fields of the Cluster resource that are located under the status section including providerStatus are available for viewing only. They are automatically generated by the bare metal cloud provider and must not be modified using API.

metadata¶

The Cluster CR contains the following fields:

apiVersion
API version of the object that is cluster.k8s.io/v1alpha1.
kind
Object type that is Cluster.

The metadata object field of the Cluster resource contains the following fields:

name
Name of a cluster. A MOSK cluster name is specified under the Cluster Name field in the Create Cluster wizard of the MOSK management console. A management cluster name is configurable in the bootstrap script.
namespace
Project in which the cluster object was created. The management cluster is always created in the default project. The MOSK cluster project equals to the selected project name.
labels
Key-value pairs attached to the object:
- kaas.mirantis.com/provider
  Provider type that is baremetal.
- kaas.mirantis.com/region
  Region name. The default region name for the management cluster is region-one.
  
  Note
  
  The kaas.mirantis.com/region label is removed from all MOSK objects in 24.1. Therefore, do not add the label starting with this release. On existing clusters updated to this release, or if added manually, MOSK ignores this label.
Warning

Labels and annotations that are not documented in this API Reference are generated automatically. Do not modify them using the API.

Configuration example:

apiVersion: cluster.k8s.io/v1alpha1
kind: Cluster
metadata:
  name: demo
  namespace: test
  labels:
    kaas.mirantis.com/provider: baremetal

spec:providerSpec¶

The spec object field of the Cluster object represents the BaremetalClusterProviderSpec subresource that contains a complete description of the desired bare metal cluster state and all details to create the cluster-level resources. It also contains the fields required for LCM deployment and integration of MOSK components.

The providerSpec object field contains the following generic fields:

apiVersion
API version of the object that is baremetal.k8s.io/v1alpha1
kind
Object type that is BaremetalClusterProviderSpec

Configuration example:

spec:
  ...
  providerSpec:
    value:
      apiVersion: baremetal.k8s.io/v1alpha1
      kind: BaremetalClusterProviderSpec

spec:providerSpec common¶

The common providerSpec object field of the Cluster resource contains the following fields:

credentials
Field reserved for other cloud providers, has an empty value. Disregard this field.
release
Name of the ClusterRelease object to install on a cluster.
helmReleases
List of enabled Helm releases from the Release object that run on a cluster.
proxy
Name of the Proxy object.
tls
TLS configuration for endpoints of a cluster.
- keycloak
  KeyCloak endpoint.
  
  tlsConfigRef
  Reference to the TLSConfig object.
- ui
  MOSK management console endpoint.
  
  tlsConfigRef
  Reference to the TLSConfig object.
For more details, see TLSConfig resource.
maintenance
Maintenance mode of a cluster. Prepares a cluster for maintenance and enables the possibility to switch machines into maintenance mode.
containerRegistries
List of the ContainerRegistries resources names.
ntpEnabled
NTP server mode. Boolean, enabled by default.

You can optionally disable NTP to disable the management of chrony configuration by MOSK and use your own system for chrony management. Otherwise, configure the regional NTP server parameters to be applied to all machines of MOSK clusters.

To configure NTP during management cluster bootstrap, see Configure optional settings. After the bootstrap, you can switch to the NTP module as a flexible way to manage different NTP settings. For details, see Configure NTP server.

audit

Optional. Technology preview. Auditing tools enabled on the cluster. Contains the following fields:

managed
Enables the auditd host operating system configuration module to manage auditd settings for machines of both management and MOSK clusters. The only supported value is false. The true value exists for backward compatibility and will be removed in one of the future releases.

For module details, refer to Host operating system configuration and GitHub auditd module project.

This field is automatically added after the management cluster update to 2.31.0 on MOSK clusters that have auditd enabled. You must migrate the auditd settings to the dedicated auditd module. For the procedure, see Migrate auditd settings from the Cluster object to the auditd module.

auditd ^Unsupported

Enables the Linux Audit daemon auditd to monitor activity of cluster processes and prevent potential malicious activity.

Caution

This parameter is unsupported. Use the dedicated auditd host operating system configuration module after cluster deployment instead. For details, refer to the following resources:

secureOverlay
Optional. Technology Preview. Enables WireGuard for traffic encryption on the Kubernetes workloads network. Boolean. Disabled by default.

Caution

Before enabling WireGuard, ensure that the Calico MTU size is at least 60 bytes smaller than the interface MTU size of the workload network. IPv4 WireGuard uses a 60-byte header. For details, see Set the MTU size for Calico.

Caution

Changing this parameter on a running cluster causes a downtime that can vary depending on the cluster size.

For more details about WireGuard, see Calico documentation: Encrypt in-cluster pod traffic.
useBGPAnnouncement
Optional. Technology preview. To enable the use of BGP announcement for the cluster API LB address, set to true. See Configure BGP announcement for cluster API LB address for details.

Configuration example:

spec:
  ...
  providerSpec:
    value:
      credentials: ""
      publicKeys:
        - name: bootstrap-key
      release: ucp-5-7-0-3-3-3-tp11
      helmReleases:
        - name: metallb
          values: {}
        ...
        - name: stacklight
          ...
      tls:
        keycloak:
          certificate:
            name: keycloak
          hostname: mosk-auth.example.com
        ui:
          certificate:
            name: ui
          hostname: mosk-ui.example.com
      containerRegistries:
      - demoregistry
      ntpEnabled: false
      ...

spec:providerSpec configuration¶

This section represents MOSK components that are enabled on a cluster. It contains the following fields:

management
Configuration for the management cluster components:
- enabled
  Management cluster components enabled (true) or disabled (false). This flag can be used to determine if the current cluster is a management cluster or a MOSK cluster.
- helmReleases
  List of the management cluster Helm releases that will be installed on the cluster. A Helm release includes the name and values fields. The specified values will be merged with relevant Helm release values of the management cluster in the Release object.
regional
List of regional cluster components for the provider:
- provider
  Provider type that is baremetal.
- helmReleases
  List of the regional Helm releases that will be installed on the cluster. A Helm release includes the name and values fields. The specified values will be merged with relevant regional Helm release values in the Release object.
release
Name of the KaaSRelease object.

Configuration example:

spec:
  ...
  providerSpec:
     value:
       kaas:
         management:
           enabled: true
           helmReleases:
             - name: kaas-ui
               values:
                 serviceConfig:
                   server: https://10.0.0.117
         regional:
           - helmReleases:
             - name: baremetal-provider
               values: {}
             provider: baremetal
           ...
         release: kaas-2-0-0

status:providerStatus common¶

^{Must not be modified using API}

The common providerStatus object field of the Cluster resource contains the following fields:

apiVersion
API version of the object that is baremetal.k8s.io/v1alpha1.
kind
Object type that is BaremetalClusterProviderStatus.
loadBalancerHost
Load balancer IP or host name of the cluster.
apiServerCertificate
Server certificate of Kubernetes API.
ucpDashboard
URL of the Mirantis Kubernetes Engine (MKE) Dashboard.
maintenance
Maintenance mode of a cluster. Prepares a cluster for maintenance and enables the possibility to switch machines into maintenance mode.

Configuration example:

status:
  providerStatus:
    apiVersion: baremetal.k8s.io/v1alpha1
    kind: BaremetalClusterProviderStatus
    loadBalancerHost: 10.0.0.100
    apiServerCertificate: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS…
    ucpDashboard: https://10.0.0.100:6443

status:providerStatus for cluster readiness¶

^{Must not be modified using API}

The providerStatus object field of the Cluster resource that reflects the cluster readiness contains the following fields:

persistentVolumesProviderProvisioned
Status of the persistent volumes provisioning. Prevents the Helm releases that require persistent volumes from being installed until some default StorageClass is added to the Cluster object.
helm
Details about the deployed Helm releases:
- ready
  Status of the deployed Helm releases. The true value indicates that all Helm releases are deployed successfully.
- releases
  List of the enabled Helm releases that run on the cluster:
  
  releaseStatuses
  List of the deployed Helm releases. The success: true field indicates that the release is deployed successfully.
  
  stacklight
  Status of the StackLight deployment. Contains URLs of all StackLight components. The success: true field indicates that StackLight is deployed successfully.
nodes
Details about the cluster nodes:
- ready
  Number of nodes that completed the deployment or update.
- requested
  Total number of nodes. If the number of ready nodes does not match the number of requested nodes, it means that a cluster is being currently deployed or updated.
notReadyObjects
The list of the services, deployments, and statefulsets Kubernetes objects that are not in the Ready state yet. A service is not ready if its external address has not been provisioned yet. A deployment or statefulset is not ready if the number of ready replicas is not equal to the number of desired replicas. Both objects contain the name and namespace of the object and the number of ready and desired replicas (for controllers). If all objects are ready, the notReadyObjects list is empty.

Configuration example:

status:
  providerStatus:
    persistentVolumesProviderProvisioned: true
    helm:
      ready: true
      releases:
        releaseStatuses:
          iam:
            success: true
          ...
        stacklight:
          alerta:
            url: http://10.0.0.106
          alertmanager:
            url: http://10.0.0.107
          grafana:
            url: http://10.0.0.108
          kibana:
            url: http://10.0.0.109
          prometheus:
            url: http://10.0.0.110
          success: true
    nodes:
      ready: 3
      requested: 3
    notReadyObjects:
      services:
        - name: testservice
          namespace: default
      deployments:
        - name: baremetal-provider
          namespace: kaas
          replicas: 3
          readyReplicas: 2
      statefulsets: {}

status:providerStatus for Open ID Connect¶

^{Must not be modified using API}

The oidc section of the providerStatus object field in the Cluster resource reflects the Open ID Connect configuration details. It contains the required details to obtain a token for a MOSK cluster and consists of the following fields:

certificate
Base64-encoded OIDC certificate.
clientId
Client ID for OIDC requests.
groupsClaim
Name of an OIDC groups claim.
issuerUrl
Issuer URL to obtain the representation of the realm.
ready
OIDC status relevance. If true, the status corresponds to the LCMCluster OIDC configuration.

Configuration example:

status:
  providerStatus:
    oidc:
      certificate: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUREekNDQWZ...
      clientId: kaas
      groupsClaim: iam_roles
      issuerUrl: https://10.0.0.117/auth/realms/iam
      ready: true

status:providerStatus for cluster releases¶

^{Must not be modified using API}

The releaseRefs section of the providerStatus object field in the Cluster resource provides the current and previous Cluster release versions as well as the one available for update. It contains the following fields:

actualRelease
Version of the Cluster release installed on the cluster.
current
Details of the currently installed Cluster release or, if the cluster is being updated to a newer release, the target Cluster release details:
- lcmType
  Type of the Cluster release (ucp).
- name
  Name of the Cluster release.
- version
  Version of the Cluster release.
- unsupportedSinceKaaSVersion
  Indicates that a KaaSRelease newer than the current one exists and that it does not support the current ClusterRelease.
- allowedNodeLabels
  Labels that can be assigned to cluster nodes using the spec.providerSpec.value.nodeLabels field of the Machine object.
previous
Details of the currently installed Cluster release or, if the cluster is being updated to a newer release, the Cluster release installed on the cluster before the update started. Contains the same fields as the current section.
available
List of Cluster releases available for update:
- name
  Name of the available Cluster release.
- version
  Version of the available Cluster release.
- rebootRequired
  Boolean flag indicating whether updating to the available Cluster release requires a node reboot.

Configuration example:

status:
  providerStatus:
    releaseRefs:
      actualRelease: 20.0.0+3.7.24
      available:
      - name: mke-20-1-0-3-7-28
        rebootRequired: true
        version: 20.1.0+3.7.28
      current:
        allowedNodeLabels:
        - displayName: Stacklight
          key: stacklight
          value: enabled
        - displayName: Node Rack ID
          key: rack-id
        lcmType: ucp
        name: mke-20-0-0-3-7-24
        version: 20.0.0+3.7.24
      previous:
        allowedNodeLabels:
        - displayName: Stacklight
          key: stacklight
          value: enabled
        - displayName: Node Rack ID
          key: rack-id
        lcmType: ucp
        name: mke-20-0-0-3-7-24
        version: 20.0.0+3.7.24

No results

An error occurred

Cluster resource¶

metadata¶

spec:providerSpec¶

spec:providerSpec common¶

spec:providerSpec configuration¶

status:providerStatus common¶

status:providerStatus for cluster readiness¶

status:providerStatus for Open ID Connect¶

status:providerStatus for cluster releases¶