Cluster¶
This section describes the Cluster
resource used in Mirantis Container
Cloud API. The Cluster
resource describes the cluster-level parameters.
For demonstration purposes, the Container Cloud Cluster
custom resource (CR) can be split into the following major sections:
Warning
The fields of the Cluster
resource that are located
under the status
section including providerStatus
are available for viewing only.
They are automatically generated by the AWS cloud provider
and must not be modified using Container Cloud API.
metadata¶
The Container Cloud Cluster
custom resource (CR) contains the following
fields:
apiVersion
cluster.k8s.io/v1alpha1
kind
Object type that is
Cluster
The metadata
object field of the Cluster
resource contains
the following fields:
name
Cluster name. A managed cluster name is set in the Cluster Name field of the Create Cluster wizard in the Container Cloud web UI. Management and regional cluster names are set in the bootstrap script.
namespace
Namespace in which the
Cluster
object is created. Management and regional clusters are created in thedefault
namespace. The namespace of a managed cluster matches the selected Project name in the Container Cloud web UI.
labels
Key-value pairs attached to the object:
kaas.mirantis.com/provider
Provider type that is
aws
for the AWS-based clusters
kaas.mirantis.com/region
Region name. The default region name for a management cluster is
region-one
. For regional clusters, it is configurable using theREGION
parameter in the bootstrap script.
Warning
Labels and annotations that are not documented in this API Reference are generated automatically by Container Cloud. Do not modify them using the Container Cloud API.
Configuration example:
apiVersion: cluster.k8s.io/v1alpha1
kind: Cluster
metadata:
name: demo
namespace: test
labels:
kaas.mirantis.com/provider: aws
kaas.mirantis.com/region: region-one
spec:providerSpec¶
The providerSpec
object field of the Cluster
resource
contains all required details to create the cluster-level resources.
It also contains fields required for LCM deployment and
the Container Cloud components integration.
The providerSpec
object field is custom for each cloud provider and
contains the following generic fields:
apiVersion
aws.kaas.mirantis.com/v1alpha1
kind
Object type that is
AWSClusterProviderSpec
Configuration example:
spec:
...
providerSpec:
value:
apiVersion: aws.kaas.mirantis.com/v1alpha1
kind: AWSClusterProviderSpec
spec:providerSpec common¶
The common providerSpec
object field of the Cluster
resource
contains the following fields:
credentials
Name of the
AWSCredential
object used by the cluster to connect to the provider back end
dedicatedControlPlane
Cluster control plane nodes to be tainted, defaults to
true
publicKeys
List of the
PublicKey
resource referencesname
Public key name
release
Name of the
ClusterRelease
object to install on a cluster
helmReleases
List of enabled Helm releases from the
Release
object that run on a cluster
proxy
Name of the
Proxy
object
tls
TLS configuration for endpoints of a cluster
keycloak
KeyCloak endpoint
tlsConfigRef
Reference to the
TLSConfig
object
ui
Web UI endpoint
tlsConfigRef
Reference to the
TLSConfig
object
For more details, see TLSConfig resource.
maintenance
Maintenance mode of a cluster. Prepares a cluster for maintenance and enables the possibility to switch machines into maintenance mode.
containerRegistries
List of the
ContainerRegistries
resources names.Note
For MOSK-based deployments, the feature support is available since MOSK 22.3.
region
AWS region where the cluster is located
Configuration example:
spec:
...
providerSpec:
value:
credentials: cloud-config
publicKeys:
- name: demo-key
release: mke-5-16-0-3-3-6
helmReleases:
- name: stacklight
values:
...
region: us-east-2
tls:
keycloak:
certificate:
name: keycloak
hostname: container-cloud-auth.example.com
ui:
certificate:
name: ui
hostname: container-cloud-ui.example.com
spec:providerSpec:networkSpec for AWS network¶
The networkSpec
object field of the Cluster
resource contains
the following fields to configure the AWS Network:
subnets
Subnets configuration that contains the following fields:
availabilityZone
Availability zone to use for this subnet in the cluster region.
cidrBlock
CIDR block to be used when the provider creates a managed VPC.
id
Unique identifier to reference the subnet.
isPublic
If
true
, a subnet is public. A subnet is public when it is associated with a route table that has a route to an Internet gateway.
natGatewayId
NAT gateway ID associated with the subnet.
routeTableId
Routing table ID associated with the subnet.
tags
List of tags describing the resource. Contains the key-value pairs that describe the subnet name, cluster UID, subnet role, and so on.
vpc
VPC configuration:
cidrBlock
CIDR block to be used when the provider creates a managed VPC. Defaults to 10.0.0.0/16.
id
ID of the VPC to be used when the provider creates resources
internetGatewayId
ID of the Internet gateway associated with the VPC
tags
List of tags describing the resource. It contains the key-value pairs that describe the VPC name, cluster UID, VPC role, and so on.
Configuration example:
spec:
...
providerSpec:
value:
networkSpec:
subnets:
- availabilityZone: us-east-2a
cidrBlock: 10.0.0.0/24
id: subnet-04cddf5343d03b32b
isPublic: true
natGatewayId: nat-039a062cb85ca45ce
routeTableId: rtb-05da80001e5c426aa
tags:
Name: 8dcf6cc2-f350-11ea-875a-0242ac110002-subnet-public
kubernetes.io/cluster/8dcf6cc2-f350-XXX-0242ac110002: owned
sigs.k8s.io/cluster-api-provider-aws/managed: "true"
sigs.k8s.io/cluster-api-provider-aws/role: public
...
vpc:
cidrBlock: 10.0.0.0/16
id: vpc-04e008da653bdb05f
internetGatewayId: igw-0e2164ed8de6c8d2b
tags:
Name: 8dcf6cc2-f350-11ea-875a-0242ac110002-vpc
kubernetes.io/cluster/8dcf6cc2-f350-XXX-0242ac110002: owned
sigs.k8s.io/cluster-api-provider-aws/managed: "true"
sigs.k8s.io/cluster-api-provider-aws/role: common
spec:providerSpec for Container Cloud configuration¶
This section represents the Container Cloud components that are enabled
on the cluster. It contains the kaas
section with the following fields:
management
Configuration for the management cluster components:
enabled
Cluster type:
true
- management clusterfalse
- regional or managed cluster
helmReleases
List of management cluster Helm releases that will be installed on a cluster. A Helm release includes the
name
andvalues
fields. Specified values will be merged with relevant management cluster Helm release values in theRelease
object.
regional
List of regional cluster components for each configured provider available for a specific region:
provider
Provider type
aws
helmReleases
List of regional Helm releases to be installed. A Helm release includes such fields as
name
andvalues
. Specified values will be merged with relevant regional Helm release values in theRelease
object.
release
Name of the Container Cloud
Release
object.
Configuration example:
spec:
...
providerSpec:
value:
kaas:
management:
enabled: true
helmReleases:
- name: kaas-ui
values:
serviceConfig:
server: <service_config>
regional:
- helmReleases:
- name: <provider_name>-provider
values: {}
provider: <provider_name>
- helmReleases:
- name: byo-provider
values: {}
provider: byo
release: kaas-2-0-0
spec:providerSpec for Bastion¶
The providerSpec
object field of the Cluster
resource
for the Bastion instance contains the following fields:
image
Image name to use for the Bastion instance.
redeployAllowed
Flag that allows redeploying the Bastion host to update the SSH keys. Defaults to
false
.
amiId
The ID of the AMI used to launch the bastion instance.
Configuration example:
providerSpec:
value:
bastion:
redeployAllowed: false
amiId: ami-2581aa40
status:providerStatus common¶
The common providerStatus
object field of the Cluster
resource
contains the following fields:
loadBalancerHost
Load balancer IP or host name of the cluster
loadBalancerStatus
Load balancer status
id
ID of the load balancer
ready
Readiness flag
status
Status details
apiServerCertificate
Server certificate of Kubernetes API
ucpDashboard
MKE Dashboard URL
maintenance
Maintenance mode of a cluster. Prepares a cluster for maintenance and enables the possibility to switch machines into maintenance mode.
Configuration example:
status:
...
providerStatus:
loadBalancerHost: 172.16.123.456
apiServerCertificate: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS…
ucpDashboard: https://172.16.123.456:6443
loadBalancerStatus:
id: 7851a962-1deb-11eb-8bec-0242ac11
ready: true
status: active
status:providerStatus for AWS network¶
Warning
Do not modify this section using API.
The network
section of the providerStatus
object field of the
Cluster
resource represents basic information about the associated
AWS network. It contains the following fields:
apiServerNetworkElb
The Kubernetes API server network load balancer:
arn
ARN to uniquely identify AWS resources.
attributes
List of extra attributes associated with the load balancer.
dnsName
DNS name of the load balancer.
name
Name of the load balancer.
scheme
Scheme of the load balancer that is Internet-facing or private.
securityGroups
Security groups configuration for the Bastion instance (
bastion
) and cluster nodes (controlplane
andnode
). Each subfield contains the following security group configuration:id
Unique identifier of the security group.
ingressRule
Inbound rules associated with the security group.
name
Security group name.
tags
List of key-value pairs of tags associated with the security group.
Configuration example:
status:
providerStatus:
network:
apiServerNetworkElb:
arn: arn:aws:elasticloadbalancing:us-east-2:643893117298:
loadbalancer/net/XXXXXXX/b6c7069ef42144eb
attributes:
- key: access_logs.s3.enabled
value: "false"
...
dnsName: XXXXXXX.elb.us-east-2.amazonaws.com
name: 8dcf6cc2-f350-11ea-875a-0242ac11
scheme: internet-facing
securityGroups:
bastion:
id: sg-07ce2664e174f2692
ingressRule:
- cidrBlocks:
- 0.0.0.0/0
description: SSH
fromPort: 22
protocol: tcp
sourceSecurityGroupIds: null
toPort: 22
name: 8dcf6cc2-f350-11ea-875a-0242ac110002-bastion
tags:
Name: 8dcf6cc2-f350-11ea-875a-0242ac110002-bastion
kubernetes.io/cluster/8dcf6cc2-f350-11ea-875a-0242ac11000: owned
sigs.k8s.io/cluster-api-provider-aws/managed: "true"
sigs.k8s.io/cluster-api-provider-aws/role: bastion
controlplane:
id: sg-02519a43ab2067af4
ingressRule:
...
node:
id: sg-00ae75ef271d78580
ingressRule:
...
status:providerStatus for Bastion¶
Warning
Do not modify this section using API.
The bastion
section of the providerStatus
object field of
the Cluster
resource contains the Bastion instance configuration:
ebsOptimized
Indicates whether the instance is optimized for Amazon EBS I/O
enaSupport
Specifies whether enhanced networking with ENA is enabled
id
Unique identifier of an instance
imageId
ID of the AMI used to launch the instance
instanceState
Current state of the instance
privateIp
Private IPv4 address assigned to the instance
publicIp
Public IPv4 address assigned to the instance, if applicable
securityGroupIds
List of security groups IDs of the instance
subnetId
ID of the instance subnet
tags
List of key-value pairs of tags associated with the instance
type
Instance type
Configuration example:
status:
providerStatus:
bastion:
ebsOptimized: false
enaSupport: true
id: i-08da6993be2f757a6
imageId: ami-2581aa40
instanceState: running
privateIp: 10.0.1.138
publicIp: 3.15.211.232
securityGroupIds:
- sg-07ce2664e174f2692
subnetId: subnet-0627c9cbcae053fec
tags:
Name: 8dcf6cc2-f350-11ea-875a-0242ac110002-bastion
kubernetes.io/cluster/8dcf6cc2-f350-11ea-875a-0242ac110002: owned
sigs.k8s.io/cluster-api-provider-aws/managed: "true"
sigs.k8s.io/cluster-api-provider-aws/role: bastion
type: t2.micro
status:providerStatus for Cluster readiness¶
Warning
Do not modify this section using API.
The providerStatus
object field of the Cluster
resource that reflects
cluster readiness contains the following fields:
persistentVolumesProviderProvisioned
Provision status of the provider persistent volumes (PVs). Used to prevent Helm releases that require PVs from being installed until some default
StorageClass
is present in the cluster.
helm
Status of deployed Helm releases:
ready
If all Helm releases have been deployed successfully, the value switches to
true
.
releases
List of enabled Helm Releases that run on a cluster:
releaseStatuses
List of Helm releases being deployed. Each release has the
success
field that switches totrue
once a release is deployed.
stacklight
Status of the StackLight deployment. Contains URLs of all StackLight components.
iam
Status of the IAM deployment. Contains URLs of the
keycloak
andapi
components.
decc
Status of the remaining
container cloud
components. Contains URLs of theui
,cache
, andproxy
components.
nodes
ready
Number of nodes that completed deployment or update.
requested
Total number of nodes. If the number of
ready
nodes does not match the number ofrequested
nodes, it means that a cluster is being updated.
ceph
ready
Ceph readiness flag.
message
Ceph status details.
ready
Cluster readiness flag. If
true
, the cluster is deployed successfully and all components are up and running.
conditions
List of objects status condition:
type
Object type
ready
Readiness flag
message
Status details
notReadyObjects
List of Kubernetes objects (
Service
,Deployment
, andStatefulSet
) that are not in theReady
state yet:Service
is not ready if its external address has not been provisioned yet.Deployment
orStatefulSet
is not ready if the number of ready replicas is not equal to the number of required replicas.
Contains the name and namespace of the object and the number of ready and required replicas for controllers. If all objects are ready, the
notReadyObjects
list is empty.
Configuration example:
status:
providerStatus:
persistentVolumesProviderProvisioned: true
helm:
ready: true
releases:
decc:
cache:
url: >-
https://a618e3d36d7f44f2e8d56bbcc53ffbf7-1765661812.us-east-2.elb.amazonaws.com
proxy:
url: >-
http://a0d8d8966e0d24f50aead0942da92456-2114585625.us-east-2.elb.amazonaws.com:3128
ui:
url: >-
https://a43fe72c644de41ae9db3cc77dd992d5-566275388.us-east-2.elb.amazonaws.com
iam:
api:
url: >-
https://a08d8bdd8553b49a88ab8e663d384001-1745154108.us-east-2.elb.amazonaws.com
keycloak:
url: >-
https://a2b58b6a3ee3c4884b034fd791ebff6d-1687192379.us-east-2.elb.amazonaws.com
releaseStatuses:
admission-controller:
success: true
iam:
success: true
iam-controller:
success: true
kaas-exporter:
success: true
kaas-public-api:
success: true
kaas-ui:
success: true
lcm-controller:
...
stacklight:
alerta:
url: http://172.16.248.170
alertmanager:
url: http://172.16.247.217
grafana:
url: http://172.16.248.49
kibana:
url: http://172.16.245.164
prometheus:
url: http://172.16.249.211
success: true
nodes:
ready: 3
requested: 3
notReadyObjects:
services:
- name: testservice
namespace: default
deployments:
- name: <provider_name>-provider
namespace: kaas
replicas: 3
readyReplicas: 2
statefulsets: {}
ready: false
ceph:
- message: Ceph cluster has been configured successfully
ready: true
conditions:
- message: Helm charts are successfully installed(upgraded).
ready: true
type: Helm
- message: Kubernetes objects are fully up.
ready: true
type: Kubernetes
- message: All requested nodes are ready.
ready: true
type: Nodes
status:providerStatus for Open ID Connect¶
Warning
Do not modify this section using API.
The oidc
section of the providerStatus
object field
of the Cluster
resource reflects the Open ID Connect (OIDC) configuration
details. It contains the required details to obtain a cluster token
and contains the following fields:
certificate
Base64-encoded OIDC certificate.
clientId
Client ID for OIDC requests.
groupsClaim
Name of an OIDC groups claim.
issuerUrl
Isuer URL to get the representation of the realm.
ready
OIDC status relevance. Is
true
if the status fits the configuration of the LCMCluster OIDC.
Configuration example:
status:
providerStatus:
oidc:
certificate: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUREekNDQWZ...
clientId: kaas
groupsClaim: iam_roles
issuerUrl: https://172.16.243.211/auth/realms/iam
ready: true
status:providerStatus for Cluster releases¶
Warning
Do not modify this section using API.
The releaseRefs
section of the providerStatus
object field
of the Cluster
resource provides the current Cluster release version
as well as the one available for upgrade. It contains the following fields:
current
Details of the currently installed Cluster release:
lcmType
Type of the Cluster release (
mke
)
name
Name of the Cluster release resource
version
Release version
unsupportedSinceKaaSVersion
Indicates that a newer Container Cloud release exists and it does not support the current Cluster release
available
List of releases available for upgrade that contains the
name
andversion
fields
Configuration example:
status:
providerStatus:
releaseRefs:
available:
- name: mke-5-15-0-3-4-0-dev
version: 5.15.0+3.4.0-dev
current:
lcmType: mke
name: mke-5-14-0-3-3-0-beta1
version: 5.14.0+3.3.0-beta1