Skip to content

How-To Deploy HCO on a Mirantis k0rdent Enterprise Environment#

This guide outlines the step-by-step process for using Mirantis k0rdent Enterprise to deploy the components of Mirantis k0rdent Virtualization on child clusters.

The first step in this process is to deploy the Hyperconverged Cluster Operator (HCO) on the target cluster. The HCO simplifies deployment of additional Mirantis k0rdent Virtualization components such as KubeVirt, CDI, networking, and kubevirt-manager, providing a unified method to manage virtual machine workloads on that child cluster.

Prerequisites#

Before proceeding, ensure you have the following in place:

  • A deployed k0rdent child cluster with cert-manager installed (It's required for HCO webhook service) – must have CRDs enabled.
  • The kubectl utility installed and configured.
  • kubeconfig files for both the KCM (k0rdent Control Manager) and the child clusters.

Note for MKE 4k: If you are using MKE 4k, services should be added through the MKE Config CR instead of patching ClusterDeployment directly. See the MKE 4k-specific section below.

cert-manager#

In order for the HCO webhook service to function properly, you need to have cert-manager deployed and enabled on the target child cluster.

To determine whether cert-manager is deployed, describe the ClusterDeployment, as in:

kubectl describe ClusterDeployment <CLUSTER-NAME> -n <CLUSTER-NAMESPACE>

For example, if you've created a ClusterDeployment named ksi-managed-cluster in the kcm-system namespace, you'd use:

kubectl describe ClusterDeployment ksi-managed-cluster -n kcm-system

...
      - name: cert-manager
        namespace: cert-manager
        template: cert-manager-1-18-2
        values: |
          cert-manager:
            crds:
              enabled: true
...

If you don't see the cert-manager service, you can go ahead and add it. You can add it from the k0rdent Catalog, or you can add it manually.

To add it manually, start by adding the new spec in a file called clusterdeployment-patch.yaml:

spec:
  serviceSpec:
    services:
      - name: cert-manager
        namespace: cert-manager
        template: cert-manager-1-18-2
        values: |
          cert-manager:
            crds:
              enabled: true
    # continueOnError: true  # uncomment for troubleshooting

Then, with the KUBECONFIG pointing at the management cluster, apply the patch to the cluster to tell it to add this service to the ClusterDeployment:

For standard k0rdent child clusters: 

kubectl patch clusterdeployment ksi-managed-cluster -n kcm-system --type=merge --patch-file clusterdeployment-patch.yaml

For MKE 4k: See the MKE 4k-specific section below for the correct method to add services.

Wait for the ClusterDeployment to be ready:

kubectl get clusterdeployments -A

Install HCO#

Deploying HCO in a Mirantis k0rdent Enterprise child environment relies on the HCO ServiceTemplate, which you can install from the Mirantis k0rdent Enterprise Catalog:

For standard k0rdent child clusters: 

helm install hco-service-template oci://registry.mirantis.com/k0rdent-enterprise-catalog/hco-service-template --version 1.16.1-mira -n kcm-system

For MKE 4k (self-adopted cluster): 

helm install hco-service-template oci://registry.mirantis.com/k0rdent-enterprise-catalog/hco-service-template --version 1.16.1-mira -n k0rdent

Verify that the HCO ServiceTemplate has been added and is VALID:

For standard k0rdent: 

kubectl -n kcm-system get servicetemplate

For MKE 4k: 

kubectl -n k0rdent get servicetemplate


NAME                      VALID
cert-manager-1-18-2       true
dex-0-19-1                true
external-secrets-0-11-0   true
hco-1-16-1-mira           true
ingress-nginx-4-11-0      true
ingress-nginx-4-11-3      true
kyverno-3-2-6             true
velero-8-1-0              true

The ServiceTemplate manifest directs the deployment of HCO using the Helm chart:

apiVersion: k0rdent.mirantis.com/v1beta1
kind: ServiceTemplate
metadata:
  name: hco-1-16-1-mira
  namespace: kcm-system  # Use k0rdent for MKE 4k
spec:
  helm:
    chartSpec:
      chart: hco
      interval: 10m0s
      reconcileStrategy: ChartVersion
      sourceRef:
        kind: HelmRepository
        name: kubevirt-repo
      version: 1.16.1-mira

By specifying the chart version and the reconcile strategy, this template ensures that HCO remains up to date with minimal manual intervention.

Steps#

To deploy HCO, you will add the relevant templates and changes to the management cluster. These changes will then affect the relevant child cluster. To install and verify HCO, follow these steps:

1. Add HCO Service Definitions to the ClusterDeployment#

Integrate HCO into your existing Mirantis k0rdent Enterprise child cluster or include it when creating a new ClusterDeployment object by updating the spec.serviceSpec.services array with the following definition:

spec:
  ...
  serviceSpec:
    services:
    - name: hco
      namespace: kubevirt
      template: hco-1-16-1-mira
  ...

This step links the HCO service definition to the Mirantis k0rdent Enterprise cluster, instructing it to deploy the HCO operator in the kubevirt namespace on the child cluster represented by the ClusterDeployment.

For MKE 4k: See the MKE 4k-specific section below for the correct method to add services.

2. Verify HCO Deployment on the Child Cluster#

To verify the installation, switch your KUBECONFIG context to the Mirantis k0rdent Enterprise child cluster and verify the HCO deployment. For example, run:

kubectl -n kubevirt get pods

Wait until the pod named hyperconverged-cluster-operator-xxxxxxxxxx-xxxxx is in the Ready state.

Monitoring pod status on the child cluster is critical to ensure that the operator is running and ready to manage virtualization components.

3. Apply the HCO Custom Resource (CR)#

Once the HCO operator is ready, apply the HCO CustomResource to the child cluster to activate Mirantis k0rdent Virtualization and its subcomponents (such as network and storage plugins):

apiVersion: hco.kubevirt.io/v1beta1
kind: HyperConverged
metadata:
  name: kubevirt-hyperconverged
  namespace: kubevirt
spec:
  featureGates:
    downwardMetrics: true
  infra:
    nodePlacement:
      nodeSelector:
        kubernetes.io/os: linux
  platform: k0s

Applying the HCO CR is the final step that triggers the deployment of all virtualization components, ensuring that your cluster is fully equipped to handle VM workloads.

MKE 4k Specific Instructions#

Important: For MKE 4k environments (self-adopted clusters), services must be added through the MKE Config CR (MkeConfig), not by directly patching the ClusterDeployment. The MKE Config CR is the source of truth for services on MKE 4k clusters.

Namespace Differences#

  • Standard k0rdent child clusters: KCM runs in the kcm-system namespace
  • MKE 4k (self-adopted cluster): KCM runs in the k0rdent namespace

Adding Services in MKE 4k#

Instead of using kubectl patch to modify the ClusterDeployment, add services to the MKE Config CR in the mke namespace:

  1. Get the MKE Config: 

    kubectl get mkeconfigs.mke.mirantis.com -n mke

  2. Edit the MKE Config to add HCO service: 

    kubectl edit mkeconfig mke -n mke

  3. Add HCO to the spec.services array:

apiVersion: mke.mirantis.com/v1alpha1
kind: MkeConfig
metadata:
  name: mke
  namespace: mke
spec:
  ...
  services:
    - name: hco
      namespace: kubevirt
      template: hco-1-16-1-mira
  ...

If you also need to add cert-manager, include it in the services array:

spec:
  services:
    - name: cert-manager
      namespace: cert-manager
      template: cert-manager-1-18-2
      values: |
        cert-manager:
          crds:
            enabled: true
    - name: hco
      namespace: kubevirt
      template: hco-1-16-1-mira

After saving the MKE Config, verify that HCO has been added to the ClusterDeployment:

kubectl get clusterdeployment <cluster-name> -n k0rdent -o jsonpath='{.spec.serviceSpec.services[*].name}' | tr ' ' '\n'

You should see hco in the list, and the ClusterDeployment should show the updated service count (e.g., 9/9 instead of 8/8):

kubectl get clusterdeployment <cluster-name> -n k0rdent

Why this matters: Directly patching the ClusterDeployment with kubectl patch will be reverted by the Helm chart reconciliation process in MKE 4k. The Helm chart that manages adopted clusters reconciles periodically and resets the ClusterDeployment spec to match the MKE Config CR.

For more information on adding services in MKE 4k, refer to the MKE 4k Add Services Documentation.

Known Issues#

Create DataVolume Error#

You may experience a problem in which the DataVolume cannot be created with the specified data source. If so, the cause is a failure in the importer image, due to the following error:

blockdev: cannot open /dev/cdi-block-volume: Permission denied

For more information on this problem, see the CDI project documentation.