Inspect the history of a cluster and machine deployment or update

Available since 2.22.0

Using the ClusterDeploymentStatus, ClusterUpgradeStatus, MachineDeploymentStatus, and MachineUpgradeStatus objects, you can inspect historical data of cluster and machine deployment or update stages, their time stamps, statuses, and failure messages, if any.

Caution

The order of cluster and machine update stages may not always be sorted by a time stamp but have an approximate logical order due to several components running simultaneously.

View the history using the web UI

1. Log in to the Container Cloud web UI with the m:kaas:namespace@operator or m:kaas:namespace@writer permissions.

2. Switch to the required project using the Switch Project action icon on top of the main left-side navigation panel.

3. In the Clusters tab, click the More action icon in the last column of the required cluster area and select History to display details of the ClusterDeploymentStatus or ClusterUpgradeStatus object, if any.

4. In the window that opens, click the required object to display the object stages, their time stamps, and statuses.

   Object names match the initial and/or target Cluster release versions and MKE versions of the cluster at a specific date and time. For example, 11.6.0+3.5.5 (initial version) or 11.5.0+3.5.5 -> 11.6.0+3.5.5.

   If any stage fails, hover over the Failure status field to display the failure message.

5. Optional. Inspect the deployment and update status of the cluster machines:

   1. In the Clusters tab, click the required cluster name. The cluster page with Machines list opens.

   2. Click More action icon in the last column of the required machine area and select History.

View the history using the CLI

1. Log in to a local machine where your management cluster kubeconfig is located and where kubectl is installed.

2. Select from the following options:

   • Inspect the cluster or machine deployment history using the ClusterDeploymentStatus or MachineDeploymentStatus object:

     ./bin/kubectl --kubeconfig <pathToClusterKubeconfig> get clusterdeploymentstatus
     
     ./bin/kubectl --kubeconfig <pathToClusterKubeconfig> get machinedeploymentstatus
     

     Example extract from a ClusterDeploymentStatus object

     kind: ClusterDeploymentStatus
     metadata:
       annotations:
         lcm.mirantis.com/new-history: "true"
       creationTimestamp: "2022-12-13T15:25:49Z"
       name: test-managed
       namespace: default
       ownerReferences:
       - apiVersion: cluster.k8s.io/v1alpha1
         kind: Cluster
         name: test-managed
     release: 11.6.0+3.5.5
     stages:
     - message: ""
       name: Network prepared
       status: Success
       success: true
       timestamp: "2022-12-13T15:27:19Z"
     - message: ""
       name: Load balancers created
       status: Success
       success: true
       timestamp: "2022-12-13T15:27:56Z"
     - message: ""
       name: IAM objects created
       status: Success
       success: true
       timestamp: "2022-12-13T15:27:21Z"
     - message: ""
       name: Kubernetes API started
       status: Success
       success: true
       timestamp: "2022-12-13T15:57:05Z"
     - message: ""
       name: Helm-controller deployed
       status: Success
       success: true
       timestamp: "2022-12-13T15:57:13Z"
     - message: ""
       name: HelmBundle created
       status: Success
       success: true
       timestamp: "2022-12-13T15:57:15Z"
     - message: ""
       name: Certificates configured
       status: Success
       success: true
       timestamp: "2022-12-13T15:58:29Z"
     - message: ""
       name: All machines of the cluster are ready
       status: Success
       success: true
       timestamp: "2022-12-13T16:04:49Z"
     - message: ""
       name: OIDC configured
       status: Success
       success: true
       timestamp: "2022-12-13T16:04:07Z"
     - message: ""
       name: Cluster is ready
       status: Success
       success: true
       timestamp: "2022-12-13T16:14:04Z"
     

     Example extract from a MachineDeploymentStatus object

     apiVersion: kaas.mirantis.com/v1alpha1
     kind: MachineDeploymentStatus
     metadata:
       creationTimestamp: "2022-12-16T09:57:07Z"
       name: test-managed-2-master-cwtf8
       namespace: default
       ownerReferences:
       - apiVersion: cluster.k8s.io/v1alpha1
         kind: Machine
         name: test-managed-2-master-cwtf8
     release: 11.6.0+3.5.5
     stages:
     - message: ""
       name: Provider instance ready
       status: Success
       success: true
       timestamp: "2022-12-16T09:59:28Z"
     - message: ""
       name: Prepare phase done
       status: Success
       success: true
       timestamp: "2022-12-16T10:12:05Z"
     - message: ""
       name: Deploy phase done
       status: Success
       success: true
       timestamp: "2022-12-16T10:16:36Z"
     - message: ""
       name: Reconfigure phase done
       status: Success
       success: true
       timestamp: "2022-12-16T10:16:40Z"
     

   • Inspect the cluster or machine update history using the ClusterUpgradeStatus and MachineUpgradeStatus objects:

     ./bin/kubectl --kubeconfig <pathToClusterKubeconfig> get clusterupgradestatus
     
     ./bin/kubectl --kubeconfig <pathToClusterKubeconfig> get machineupgradestatus
     

     Example extract from a ClusterUpgradeStatus object

     apiVersion: kaas.mirantis.com/v1alpha1
     fromRelease: 11.5.0+3.5.5
     kind: ClusterUpgradeStatus
     metadata:
       annotations:
         lcm.mirantis.com/new-history: "true"
       creationTimestamp: "2022-12-16T09:02:20Z"
       name: test-managed-11.6.0+3.5.5
       namespace: default
       ownerReferences:
       - apiVersion: cluster.k8s.io/v1alpha1
         kind: Cluster
         name: test-managed
     stages:
     - message: ""
       name: Requested cluster maintenance
       status: Success
       success: true
       timestamp: "2022-12-16T09:02:02Z"
     - message: ""
       name: Cluster workload locks are inactive
       status: Success
       success: true
       timestamp: "2022-12-16T09:02:02Z"
     - message: ""
       name: Worker nodes MKE upgraded
       status: Success
       success: true
       timestamp: "2022-12-16T09:47:48Z"
     - message: ""
       name: Control plane nodes MKE upgraded
       status: Success
       success: true
       timestamp: "2022-12-16T09:37:33Z"
     - message: ""
       name: Cluster maintenance request removed
       status: Success
       success: true
       timestamp: "2022-12-16T09:48:16Z"
     toRelease: 11.6.0+3.5.5
     

     Example extract from a MachineUpgradeStatus object for a control plane machine

     apiVersion: kaas.mirantis.com/v1alpha1
     fromRelease: 11.5.0+3.5.5
     kind: MachineUpgradeStatus
     metadata:
       annotations:
         lcm.mirantis.com/new-history: "true"
       creationTimestamp: "2022-12-16T09:02:24Z"
       name: test-managed-master-j2vmj-11.6.0+3.5.5
       namespace: default
       ownerReferences:
       - apiVersion: cluster.k8s.io/v1alpha1
         kind: Machine
         name: test-managed-master-j2vmj
     stages:
     - message: ""
       name: Agent upgraded
       status: Success
       success: true
       timestamp: "2022-12-16T09:02:22Z"
     - message: ""
       name: Machine Upgraded
       status: Success
       success: true
       timestamp: "2022-12-16T09:37:14Z"
     toRelease: 11.6.0+3.5.5
     

     Example extract from a MachineUpgradeStatus object for a worker machine

     apiVersion: kaas.mirantis.com/v1alpha1
     fromRelease: 11.5.0+3.5.5
     kind: MachineUpgradeStatus
     metadata:
       annotations:
         lcm.mirantis.com/new-history: "true"
       creationTimestamp: "2022-12-16T09:02:58Z"
       name: test-managed-node-ggzn2-11.6.0+3.5.5
       namespace: default
       ownerReferences:
       - apiVersion: cluster.k8s.io/v1alpha1
         kind: Machine
         name: test-managed-node-ggzn2
     stages:
     - message: ""
       name: Agent upgraded
       status: Success
       success: true
       timestamp: "2022-12-16T09:02:56Z"
     - message: ""
       name: Previous machines are deployed
       status: Success
       success: true
       timestamp: "2022-12-16T09:42:20Z"
     - message: ""
       name: Node maintenance requested
       status: Success
       success: true
       timestamp: "2022-12-16T09:42:13Z"
     - message: ""
       name: Node workload locks inactive
       status: Success
       success: true
       timestamp: "2022-12-16T09:42:14Z"
     - message: ""
       name: Kubernetes drained
       status: Success
       success: true
       timestamp: "2022-12-16T09:42:14Z"
     - message: ""
       name: Deploy phase done
       status: Success
       success: true
       timestamp: "2022-12-16T09:43:33Z"
     - message: ""
       name: Reconfigure phase done
       status: Success
       success: true
       timestamp: "2022-12-16T09:43:36Z"
     - message: ""
       name: Machine Upgraded
       status: Success
       success: true
       timestamp: "2022-12-16T09:48:15Z"
     - message: ""
       name: Kubernetes uncordoned
       status: Success
       success: true
       timestamp: "2022-12-16T09:47:47Z"
     - message: ""
       name: Node maintenance request removed
       status: Success
       success: true
       timestamp: "2022-12-16T09:47:48Z"
     toRelease: 11.6.0+3.5.5
     

   Object names match the initial and/or target Cluster release versions and MKE versions of the cluster. For example, 11.5.0+3.5.5 (initial version) or 11.5.0+3.5.5 -> 11.6.0+3.5.5. Each object displays the update stages, their time stamps, and statuses. If any stage fails, the success field contains a failure message.