Collect cluster logs¶
While operating your management, regional, or managed cluster, you may require collecting and inspecting the cluster logs to analyze cluster events or troubleshoot issues.
To collect cluster logs:
Verify that the bootstrap directory is updated.
Select from the following options:
For clusters deployed using Container Cloud 2.11.0 or later:
./container-cloud bootstrap download --management-kubeconfig <pathToMgmtKubeconfig> \ --target-dir <pathToBootstrapDirectory>
For clusters deployed using the Container Cloud release earlier than 2.11.0 or if you deleted the
kaas-bootstrapfolder, download and run the Container Cloud bootstrap script:
wget https://binary.mirantis.com/releases/get_container_cloud.sh chmod 0755 get_container_cloud.sh ./get_container_cloud.sh
kubeconfigof the required cluster. The management or regional cluster
kubeconfigfiles are created during the last stage of the management or regional cluster bootstrap. To obtain a managed cluster
kubeconfig, see Connect to a Mirantis Container Cloud cluster.
Obtain the private SSH key of the required cluster:
For a managed cluster, this is an SSH key added in the Container Cloud web UI before the managed cluster creation.
For a management or regional cluster,
ssh_keyis created during bootstrap of the corresponding cluster in the same directory as the bootstrap script.
If the initial version of your Container Cloud management cluster was earlier than 2.6.0,
openstack_tmpand is located at
Depending on the cluster type that you require logs from, run the corresponding command:
For a management cluster:
./container-cloud collect logs --management-kubeconfig <pathToMgmtClusterKubeconfig> \ --key-file <pathToMgmtClusterPrivateSshKey> \ --cluster-name <clusterName> --cluster-namespace <clusterProject>
For a regional cluster:
./container-cloud collect logs --management-kubeconfig <pathToMgmtClusterKubeconfig> \ --key-file <pathToRegionalClusterSshKey> --kubeconfig <pathToRegionalClusterKubeconfig> \ --cluster-name <clusterName> --cluster-namespace <clusterProject>
For a managed cluster:
./container-cloud collect logs --management-kubeconfig <pathToMgmtClusterKubeconfig> \ --key-file <pathToManagedClusterSshKey> --kubeconfig <pathToManagedClusterKubeconfig> \ --cluster-name <clusterName> --cluster-namespace <clusterProject>
Substitute the parameters enclosed in angle brackets with the corresponding values of your cluster.
Directory path to save logs. The default value is
logs/. For example,
--extendedAvailable since 2.19.0
Output the extended version of logs that contains system and MKE logs, logs from LCM Ansible and LCM Agent along with cluster events and Kubernetes resources description and logs.
--extendedflag, the basic version of logs is collected, which is sufficient for most use cases. The basic version of logs contains all events, Kubernetes custom resources, and logs from all Container Cloud components. This version does not require passing
Before Container Cloud 2.19.0, the extended version of logs is collected by default.
For the logs structure, see Collect the bootstrap logs.
If you require logs of a cluster update, inspect the following folders on the control plane nodes:
/objects/namespaced/<namespaceName>/core/pods/lcm-lcm-controller- <controllerID>/for the
/objects/namespaced/<namespaceName>/core/pods/<cloudProviderName-ID>/for logs of the cloud provider controller. For example,
/system/<controllerMachineName>/mke/) for the MKE support dump. The
dsinfo/dsinfo.txtfile contains Docker and system information of the MKE configuration set before and after update.
events.logfor cluster events logs.
Technology Preview. For bare metal clusters, assess the Ironic pod logs:
Extract the content of the
'message'fields from every log message:
kubectl -n kaas logs <ironicPodName> -c syslog | jq -rM '.message'
Extract the content of the
'message'fields from the
ironic_conductorsource log messages:
kubectl -n kaas logs <ironicPodName> -c syslog | jq -rM 'select(.source == "ironic_conductor") | .message'
The syslog container collects logs generated by Ansible during the node deployment and cleanup and outputs them in the JSON format.
Compress the collected log files and send the archive to the Mirantis support team.