Add a machine¶
After you create a new managed cluster that is based on Equinix Metal as described in Create a managed cluster, proceed with adding machines to this cluster using the Container Cloud web UI.
You can also use the instruction below to scale up an existing managed cluster.
To add a machine to a managed cluster:
Available since Container Cloud 2.22.0 as Technology Preview. If you enabled the
customparameter in the
providerSpec.value.networksection of the
Clusterobject, customize network configuration for the cluster machines:
Advanced network configuration for machines
Subnetobjects with the
ipam/SVC-dhcp-rangelabels and any number of
L2Templateobjects with advanced network configuration for machines. For details, see descriptions of Subnet and L2Template objects.
Apply the IPAM configuration template to create the
L2Templateobjects. For example:
./kaas-bootstrap/bin/kubectl apply -n <managedClusterProjectName> \ kaas-bootstrap/templates/equinixmetalv2/ipam-objects.yaml.template
providerSpec.value.networksection for every machine:
providerSpec: value: # ... network: l2TemplateSelector: name: SET_L2TEMPLATE_NAME
l2TemplateSelectorparameter contains a link to the
L2Templateobject with advanced host networking configuration for the machine. The
namefield contains the name of the
L2Templateobject to use.
For details, see descriptions of Subnet and L2Template objects.
Verify that the servers of a particular type and data center combination are available for the machines deployment as described in Verify the capacity of the Equinix Metal facility.
Log in to the Container Cloud web UI with the
Switch to the required project using the Switch Project action icon located on top of the main left-side navigation panel.
In the Clusters tab, click the required cluster name. The cluster page with Machines list opens.
On the cluster page, click Create Machine.
Fill out the form with the following parameters as required:
Create Machines Pool
Select to create a set of machines with the same provider spec to manage them as a single unit. Enter the machine pool name in the Pool Name field.
Specify the number of machines to create. If you create a machine pool, specify the replicas count of the pool.
Select Manager or Worker to create a Kubernetes manager or worker node.
The required minimum number of manager machines is three for HA. A cluster can have more than three manager machines but only an odd number of machines.
In an even-sized cluster, an additional machine remains in the
Pendingstate until an extra manager machine is added. An even number of manager machines does not provide additional fault tolerance but increases the number of node required for etcd quorum.
The required minimum number of worker machines for the Container Cloud workloads is two. If the multiserver mode is enabled for StackLight, add three worker nodes.
Machine type to provision the Equinix Metal server. From the drop-down list, select a server to provision for your project. Pay attention to the machine capacity:
Normal - the facility has a lot of available machines. Prioritize this machine type over others.
Limited - the facility has a limited number of machines. Do not request many machines of this type.
Unknown - Container Cloud cannot fetch information about the capacity level since the feature is disabled.
Enable the feature with a user-level token in the
Credentialsobject used for cluster deployment. To add a user-level token:
Log in to the Equinix Metal console.
Select the project used for the Container Cloud deployment.
In Profile Settings > Personal API Keys, capture the existing API Key or create a new one:
Click Add New Key.
Fill in the Description and select the Read/Write permissions.
Click Add Key.
In the Credentials tab of the Container Cloud web UI, add the user-level token obtained in the previous step.
Mirantis highly recommends using the
c3.small.x86machine type for the control plane machines deployed with private network to prevent hardware issues with incorrect BIOS boot order.
Hardware Reservation ID Technology Preview
Optional. The ID of an Equinix Metal reserved hardware.
Fill out this field to use a reserved hardware on your Equinix Metal server.
Skip this field if you are deploying Equinix Metal servers on demand.
Available if Manual Ceph Configuration was selected during the cluster creation.
Select the Ceph machine role for Ceph Controller to automatically create the Ceph node based on the Equinix machine hardware storage:
Monitor and Manager to deploy Ceph Monitor and Ceph Manager
Storage to deploy Ceph OSD
To specify the Ceph node manually through the
KaaSCephClusterresource, do not select the Ceph machine role.
Recommended minimal number of Ceph node roles:
Manager and Monitor
3 (for HA)
Optional. A positive numeral value that defines the order of machine upgrade during a cluster update.
You can change the upgrade order later on an existing cluster. For details, see Change the upgrade order of a machine or machine pool.
Consider the following upgrade index specifics:
The first machine to upgrade is always one of the control plane machines with the lowest
upgradeIndex. Other control plane machines are upgraded one by one according to their upgrade indexes.
false, worker machines are upgraded only after the upgrade of all control plane machines finishes. Otherwise, they are upgraded after the first control plane machine, concurrently with other control plane machines.
If several machines have the same upgrade index, they have the same priority during upgrade.
If the value is not set, the machine is automatically assigned a value of the upgrade index.
Select the required node labels for the worker machine to run certain components on a specific node. For example, for the StackLight nodes that run OpenSearch and require more resources than a standard node, select the StackLight label. The list of available node labels is obtained from
allowedNodeLabelsof your current
valuefield is not defined in
allowedNodeLabels, select the check box of the required label and define an appropriate custom value for this label to be set to the node. For example, the
node-typelabel can have the
storage-ssdvalue to meet the service scheduling logic on a particular machine.
Due to the known issue 23002 fixed in Container Cloud 2.21.0, a custom value for a predefined node label cannot be set using the Container Cloud web UI. For a workaround, refer to the issue description.
If you deploy StackLight in the HA mode (recommended):
Add the StackLight label to minimum three worker nodes. Otherwise, StackLight will not be deployed until the required number of worker nodes is configured with the StackLight label.
Removal of the StackLight label from worker nodes along with removal of worker nodes with StackLight label can cause the StackLight components to become inaccessible. It is important to correctly maintain the worker nodes where the StackLight local volumes were provisioned. For details, see Delete a cluster machine.
To obtain the list of nodes where StackLight is deployed, refer to Upgrade managed clusters with StackLight deployed in HA mode.
If you move the StackLight label to a new worker machine on an existing cluster, manually deschedule all StackLight components from the old worker machine, which you remove the StackLight label from. For details, see Deschedule StackLight Pods from a worker machine.
You can add node labels after deploying a worker machine. On the Machines page, click the More action icon in the last column of the required machine field and select Configure machine.
Repeat the steps above for the remaining machines.
Monitor the deploy or update live status of the machine:
- Quick status
On the Clusters page, in the Managers or Workers column. The green status icon indicates that the machine is Ready, the orange status icon indicates that the machine is Updating.
- Detailed status
In the Machines section of a particular cluster page, in the Status column. Hover over a particular machine status icon to verify the deploy or update status of a specific machine component.
You can monitor the status of the following machine components:
Readiness of a node in a Kubernetes cluster
Health and readiness of a node in a Docker Swarm cluster
LCM readiness status of a node
Readiness of a node in the underlying infrastructure (virtual or bare metal, depending on the provider type)
The machine creation starts with the Provision status. During provisioning, the machine is not expected to be accessible since its infrastructure (VM, network, and so on) is being created.
Other machine statuses are the same as the
Uninitialized - the machine is not yet assigned to an
Pending - the agent reports a node IP address and host name.
Prepare - the machine executes
StateItemsthat correspond to the
preparephase. This phase usually involves downloading the necessary archives and packages.
Deploy - the machine executes
StateItemsthat correspond to the
deployphase that is becoming a Mirantis Kubernetes Engine (MKE) node.
Ready - the machine is being deployed.
Upgrade - the machine is being upgraded to the new MKE version.
Reconfigure - the machine executes
StateItemsthat correspond to the
reconfigurephase. The machine configuration is being updated without affecting workloads running on the machine.
Once the status changes to Ready, the deployment of the cluster components on this machine is complete.
You can also monitor the live machine status using API:
kubectl get machines <machineName> -o wide
Example of system response since Container Cloud 2.23.0:
NAME READY LCMPHASE NODENAME UPGRADEINDEX REBOOTREQUIRED WARNINGS demo-0 true Ready kaas-node-c6aa8ad3 1 false
For the history of a machine deployment or update, refer to Inspect the history of a cluster and machine deployment or update.
If a machine is stuck in the Provision state due to the exceeded machine quota, the Provider Instance field of a machine live status contains the Machine quota exceeded message. Delete such machine using the More menu located in the last column of the machine details.
If the minimal machine requirement is not met as described in Requirements for an Equinix Metal based cluster, create a new machine with the Normal machine capacity label before you can delete the stuck machine to proceed with cluster deployment.
Verify the status of the cluster nodes as described in Connect to a Mirantis Container Cloud cluster.
An operational managed cluster must contain a minimum of 3 Kubernetes manager nodes to meet the etcd quorum and 2 Kubernetes worker nodes.
The deployment of the cluster does not start until the minimum number of nodes is created.
A machine with the manager node role is automatically deleted during the cluster deletion.
Deletion of the manager nodes is allowed for non-MOSK-based clusters within the Technology Preview features scope for the purpose of node replacement or recovery.