Bare Metal#

Mirantis k0rdent Enterprise can deploy managed clusters on bare metal servers using the Metal3 infrastructure provider. This implementation is based on the bare metal Cluster API Provider, Metal3 CAPM3, and provides out-of-tree (OOT) bare metal provisioning capabilities.

CAPM3 works by enabling you to add a representation of each bare metal server as a Kubernetes object. Mirantis k0rdent Enterprise can then assemble these machine objects into a cluster.

Structure#

The bare metal infrastructure provider is represented as a set of Helm charts. It includes the following charts:

baremetal-operator installs the Bare Metal Operator
capm3-crds installs the CustomResourceDefinition objects for the Metal3 CAPM3 and IPAM components
cluster-api-provider-metal3 installs the Metal3 CAPM3 provider and Metal3 IP Address Manager
ironic installs OpenStack Ironic and accompanying components needed by Metal3 CAPM3

Prerequisites#

An installed Mirantis k0rdent Enterprise management cluster. Follow the instructions in Install Mirantis k0rdent Enterprise to create a management cluster with Mirantis k0rdent Enterprise running. Prepare this cluster according to the Metal3 host configuration guide.
Supported hardware as documented in the Metal3 hardware compatibility guide
You should still have Helm installed from your installation of Mirantis k0rdent Enterprise. If not, install it again.

Prepare Mirantis k0rdent Enterprise for Bare Metal clusters#

Follow these instructions to make Mirantis k0rdent Enterprise capable of deploying bare metal clusters:

Create the required objects for the OOT CAPM3 provider

Create the necessary Kubernetes objects to install the out-of-tree CAPM3 provider. Just as with other providers, these include a HelmRepository, ProviderTemplate, and ClusterTemplate.

kubectl create -f - <<EOF
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: oot-capm3-repo
  namespace: kcm-system
  labels:
    k0rdent.mirantis.com/managed: "true"
spec:
  type: oci
  url: 'oci://registry.mirantis.com/k0rdent-bm/charts/'
  interval: 10m0s
---
apiVersion: k0rdent.mirantis.com/v1alpha1
kind: ProviderTemplate
metadata:
  name: cluster-api-provider-metal3-0-1-1
  annotations:
    helm.sh/resource-policy: keep
spec:
  helm:
    chartSpec:
      chart: cluster-api-provider-metal3
      version: 0.1.1
      interval: 10m0s
      sourceRef:
        kind: HelmRepository
        name: oot-capm3-repo
---
apiVersion: k0rdent.mirantis.com/v1alpha1
kind: ClusterTemplate
metadata:
  annotations:
    helm.sh/resource-policy: keep
  labels:
    k0rdent.mirantis.com/component: kcm
  name: capm3-standalone-cp-0-1-1
  namespace: kcm-system
spec:
  helm:
    chartSpec:
      chart: capm3-standalone-cp
      version: 0.1.1
      interval: 10m0s
      reconcileStrategy: ChartVersion
      sourceRef:
        kind: HelmRepository
        name: oot-capm3-repo
EOF

Verify the ProviderTemplate is valid

Check that the ProviderTemplate has been created successfully:

kubectl get providertemplates cluster-api-provider-metal3-0-1-1

NAME                              VALID
cluster-api-provider-metal3-0-1-1 true

Configure the Management object

Edit the Management object to add the CAPM3 provider configuration:

kubectl edit managements.k0rdent.mirantis.com

Add the following configuration to the providers section:

- name: cluster-api-provider-metal3
  template: cluster-api-provider-metal3-0-1-1
  config:
    global:
      ironic:
        enabled: true # networking configuration ("ironic.networking" section) should be defined prior to enabling ironic
    ironic:
      networking:
        dhcp: # used by DHCP server to assign IPs to hosts during PXE boot
          rangeBegin: <DHCP_RANGE_START>      # e.g., 10.0.1.51
          rangeEnd: <DHCP_RANGE_END>          # e.g., 10.0.1.55
        interface: <PROVISION_INTERFACE>      # e.g., bond0 - interface of the management cluster node connected to BM hosts provision network
        ipAddress: <KEEPALIVED_VIP>          # e.g., 10.0.1.50 - keepalived VIP for DHCP server and Ironic services. This VIP will be configured on the <PROVISION_INTERFACE>, it must be in the same L3 network as DHCP range if no dhcp-relay used between management cluster and child cluster hosts.        # By default, "ubuntu-noble-hwe-2025-05-15-15-22-56.qcow2" is the only image available.
    # You can define custom OS images here if needed bu adding new resources:
    # resources:
    #   static:
    #     images:
    #       ubuntu-24.04-server-cloudimg-amd64.img:
    #         sha256sum: 071fceadf1ea57a388ff7a1ccb4127155d691a511f6a207b4c11b120563855e2
    #         url: https://cloud-images.ubuntu.com/releases/noble/release/ubuntu-24.04-server-cloudimg-amd64.img

Wait for the Management object to be ready

Monitor the Management object status:
```
kubectl get managements.k0rdent.mirantis.com -w
```
This process usually takes as long as 5 minutes. If the Management object doesn't become Ready, refer to the Troubleshooting section.

Verify the ClusterTemplate is valid

Check that the ClusterTemplate has been created successfully:

kubectl -n kcm-system get clustertemplates capm3-standalone-cp-0-1-1

NAME                        VALID
capm3-standalone-cp-0.1.1   true

Enroll bare metal machines#

The next step is to create BareMetalHost objects to represent your bare metal machines so Mirantis k0rdent Enterprise can manage them. For each bare metal machine, create two objects: a Secret and a BareMetalHost. For detailed instructions, see the Metal3 BareMetalHost enrollment guide (just Enrolling, not Provisioning), or follow these instructions.

Note

You don't need to provision bare metal hosts at this stage. Provisioning should happen later as part of this process.

Create credential Secret objects

You need to provide BMC credentials for every BareMetalHost using Secret objects. For example:

apiVersion: v1
kind: Secret
metadata:
  name: <BMH_NAME>-bmc-secret
  namespace: <NAMESPACE>
type: Opaque
data:
  username: <BASE64_ENCODED_BMC_USERNAME>
  password: <BASE64_ENCODED_BMC_PASSWORD>

Create BareMetalHost objects

A BareMetalHost object represents the physical machine, as in:

apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
metadata:
  name: <BMH_NAME>
  namespace: <NAMESPACE>
spec:
  online: true
  bmc:
    address: <BMC_ADDRESS>  # e.g., ipmi://192.168.1.100:623
    credentialsName: <BMH_NAME>-bmc-secret
    #disableCertificateVerification: true # only needed when using redfish protocol
  bootMACAddress: <MAC_ADDRESS>

Wait for BareMetalHost objects to complete enrollment

Monitor your BareMetalHost objects until they are available:

kubectl get bmh -n <NAMESPACE>

NAME      STATE       CONSUMER   ONLINE   ERROR   AGE
child-1   available              true             4d17h
child-2   available              true             4d17h
child-3   available              true             4d17h

For more information about BareMetalHost states, see the Metal3 state machine documentation.

Create the cluster#

You need to create several objects before Mirantis k0rdent Enterprise can create a bare metal cluster.

Create the credential objects

Since CAPM3 doesn't require cloud credentials, create dummy Secret and Credential objects to satisfy ClusterDeployment requirements:

apiVersion: v1
kind: Secret
metadata:
  name: capm3-cluster-secret
  namespace: <NAMESPACE>
  labels:
    k0rdent.mirantis.com/component: "kcm"
type: Opaque
---
apiVersion: k0rdent.mirantis.com/v1alpha1
kind: Credential
metadata:
  name: capm3-stub-credential
  namespace: <NAMESPACE>
spec:
  description: CAPM3 Credentials
  identityRef:
    apiVersion: v1
    kind: Secret
    name: capm3-cluster-secret
    namespace: <NAMESPACE>

Create the ConfigMap resource-template

Create an empty ConfigMap resource-template:

apiVersion: v1
kind: ConfigMap
metadata:
  name: capm3-cluster-credential-resource-template
  namespace: <NAMESPACE>
  labels:
    k0rdent.mirantis.com/component: "kcm"
  annotations:
    projectsveltos.io/template: "true"

Deploy a test cluster

Create a ClusterDeployment to test your bare metal configuration. Start with a capm3-example.yaml file. This one creates a cluster with 1 control node and 2 workers:

apiVersion: k0rdent.mirantis.com/v1alpha1
kind: ClusterDeployment
metadata:
  name: capm3-example
  namespace: <NAMESPACE>
spec:
  template: capm3-standalone-cp-0-1-1
  credential: capm3-stub-credential
  dryRun: false
  config:
    clusterAnnotations: {}
    clusterLabels: {}
    clusterNetwork:
      pods:
        cidrBlocks:
          - 10.243.0.0/16
      services:
        cidrBlocks:
          - 10.95.0.0/16
    controlPlane:
      # the image that was uploaded by default
      checksum: http://<IRONIC_HTTP_ENDPOINT>:6180/images/ubuntu-noble-hwe-2025-05-15-15-22-56.qcow2.sha256sum
      image: http://<IRONIC_HTTP_ENDPOINT>:6180/images/ubuntu-noble-hwe-2025-05-15-15-22-56.qcow2
      keepalived:
        authPass: <VRRP_PASSWORD> # optional, from 4 to 8 letters
        enabled: true
        virtualIP: <CLUSTER_API_VIP>/24  # e.g., 10.0.1.70/24
      preStartCommands:
        - sudo useradd -G sudo -s /bin/bash -d /home/user1 -p $(openssl passwd -1 myuserpass) user1 # define your user here. it can be used e.g. for debugging.
        - sudo apt update # for Ubuntu
        - sudo apt install jq -y # for Ubuntu
        #- sudo dnf makecache # for RedHat
        #- sudo dnf install jq -y # for RedHat
        # jq is used in K0sControlPlane object to parse cloud-init data that is required for Metal3 provider
      files:
        - path: /home/user1/.ssh/authorized_keys
          permissions: "0644"
          content: "<SSH_PUBLIC_KEY>"
    controlPlaneNumber: 1
    dataTemplate:
      metaData:
        ipAddressesFromIPPool:
          - key: provisioningIP
            name: pool-pxe
        objectNames:
          - key: name
            object: machine
          - key: local-hostname
            object: machine
          - key: local_hostname
            object: machine
        prefixesFromIPPool:
          - key: provisioningCIDR
            name: pool-pxe
      networkData:
        links:
          ethernets:
            - id: <INTERFACE_NAME>  # e.g., ens3
              type: phy
              macAddress:
                fromHostInterface: <INTERFACE_NAME>
        networks:
          ipv4:
            - id: pxe
              ipAddressFromIPPool: pool-pxe
              link: <INTERFACE_NAME>
        routes:
          - gateway:
              fromIPPool: pool-pxe
            network: 0.0.0.0
            prefix: 0
        services:
          dns:
            - <DNS_SERVER_IP>   # e.g., 8.8.8.8
    ipPools:
      - name: pool-pxe
        pools:
          - end: <IP_POOL_END>      # e.g., 10.0.1.65
            gateway: <GATEWAY_IP>   # e.g., 10.0.1.1
            prefix: 24
            start: <IP_POOL_START>  # e.g., 10.0.1.61
    k0s:
      api:
        externalAddress: <CLUSTER_API_VIP>  # e.g., 10.0.1.70
      telemetry:
        enabled: false
      version: v1.32.3+k0s.0
    worker:
      # the image that was uploaded by default
      checksum: http://<IRONIC_HTTP_ENDPOINT>:6180/images/ubuntu-noble-hwe-2025-05-15-15-22-56.qcow2.sha256sum
      image: http://<IRONIC_HTTP_ENDPOINT>:6180/images/ubuntu-noble-hwe-2025-05-15-15-22-56.qcow2
      preStartCommands:
        - sudo useradd -G sudo -s /bin/bash -d /home/user1 -p $(openssl passwd -1 myuserpass) user1 # define your user here. it can be used e.g. for debugging.
        - sudo apt update # for Ubuntu
        - sudo apt install jq -y # for Ubuntu
        #- sudo dnf makecache # for RedHat
        #- sudo dnf install jq -y # for RedHat
        # jq is used in K0sControlPlane object to parse cloud-init data that is required for Metal3 provider
      files:
        - path: /home/user1/.ssh/authorized_keys
          permissions: "0644"
          content: "<SSH_PUBLIC_KEY>"
    workersNumber: 2

Apply the YAML to your management cluster:

kubectl apply -f capm3-example.yaml

Monitor the provisioning process

Watch the BareMetalHost objects as they transition through provisioning states:

kubectl -n <NAMESPACE> get bmh -w

You should see the hosts transition from available to provisioning to provisioned:

NAME      STATE         CONSUMER                     ONLINE   ERROR   AGE
child-1   available                                  true             16m
child-2   available                                  true             16m
child-3   available                                  true             16m
child-2   provisioning  capm3-example-cp-templ-0     true             16m
child-2   provisioned   capm3-example-cp-templ-0     true             18m
child-1   provisioning  capm3-example-md-txr9f-k8z9d true             18m
child-3   provisioning  capm3-example-md-txr9f-lkc5c true             18m
child-3   provisioned   capm3-example-md-txr9f-lkc5c true             21m
child-1   provisioned   capm3-example-md-txr9f-k8z9d true             21m

Also monitor the Metal3Machine objects that are part of the cluster:

kubectl -n <NAMESPACE> get metal3machine -w

NAME                           AGE     PROVIDERID                                              READY   CLUSTER        PHASE
capm3-example-cp-templ-0       5m17s   metal3://kcm-system/child-2/capm3-example-cp-templ-0  true    capm3-example
capm3-example-md-txr9f-k8z9d   2m40s   metal3://kcm-system/child-1/capm3-example-md-txr9f-k8z9d true capm3-example
capm3-example-md-txr9f-lkc5c   2m40s   metal3://kcm-system/child-3/capm3-example-md-txr9f-lkc5c true capm3-example

Access the deployed cluster

Once all machines are running, retrieve the KUBECONFIG for your new cluster:

kubectl get secret -n <NAMESPACE> capm3-example-kubeconfig -o jsonpath='{.data.value}' | base64 -d > capm3-example-kubeconfig.kubeconfig
KUBECONFIG="capm3-example-kubeconfig.kubeconfig" kubectl get pods -A

Cleanup

To clean up bare metal resources, delete the child cluster by deleting the ClusterDeployment:

kubectl get clusterdeployments -A

NAMESPACE    NAME            READY   STATUS
bm-example   capm3-example   True    ClusterDeployment is ready

kubectl delete clusterdeployments capm3-example -n <NAMESPACE>

clusterdeployment.k0rdent.mirantis.com "capm3-example" deleted

Cluster deletion may take several minutes.

Watch the BareMetalHost objects as they transition through provisioning states:

kubectl -n <NAMESPACE> get bmh -w

You should see the hosts transition from provisioned to available:

NAME      STATE          CONSUMER                     ONLINE   ERROR   AGE
child-1   deprovisioning capm3-example-md-txr9f-k8z9d true             31m
child-2   provisioned    capm3-example-cp-templ-0     true             31m
child-3   deprovisioning capm3-example-md-txr9f-lkc5c true             31m
child-1   available                                   true             36m
child-3   available                                   true             37m
child-2   deprovisioning capm3-example-cp-templ-0     true             37m
child-2   available                                   true             43m

Then, the available hosts can be used to deploy another cluster, using the same BareMetalHost objects.

Troubleshooting#

If you run into difficulties, you might find the solution here.

`ProviderTemplate` is not valid#

If the ProviderTemplate shows as invalid, inspect the object for error messages:

kubectl get providertemplates cluster-api-provider-metal3-0-1-1 -oyaml

Common issues include incorrect credentials for accessing artifacts or connection problems.

`Management` object does not become ready#

If the Management object remains in a non-ready state, inspect it for error messages:

kubectl get managements.k0rdent.mirantis.com -oyaml

If you see Ironic-related errors, check the Ironic deployment:

kubectl -n kcm-system get deployment/cluster-api-provider-metal3-ironic

If Ironic is not ready, verify its configuration:

kubectl -n kcm-system get cm ironic-bmo-config -oyaml

Ensure the configuration matches your network environment, particularly the PROVISIONING_INTERFACE, PROVISIONING_IP, and DHCP_RANGE settings.

The `HelmRelease` does not become ready#

Check to see if the HelmRelease object is Ready, and if not, why:

kubectl -n kcm-system get helmrelease cluster-api-provider-metal3

NAME                          AGE    READY   STATUS
cluster-api-provider-metal3   164m   False   Helm install failed for release kcm-system/cluster-api-provider-metal3 with chart cluster-api-provider-metal3@0.1.0-9d6d9c8: context deadline exceeded

If you see this error, delete the HelmRelease:

kubectl -n kcm-system delete helmrelease cluster-api-provider-metal3

Kubernetes will automatically reinstall the HelmRelease.

Useful resources#

For additional troubleshooting guidance, refer to the Metal3 troubleshooting documentation.

For more information about bare metal cluster configuration options, see: