Collect cluster logs¶
While operating your management, regional, or managed cluster, you may require collecting and inspecting the cluster logs to analyze cluster events or troubleshoot issues.
To collect cluster logs:
Verify that the bootstrap directory is updated.
Select from the following options:
For clusters deployed using Container Cloud 2.11.0 or later:
./container-cloud bootstrap download --management-kubeconfig <pathToMgmtKubeconfig> \ --target-dir <pathToBootstrapDirectory>
For clusters deployed using the Container Cloud release earlier than 2.11.0 or if you deleted the
kaas-bootstrap
folder, download and run the Container Cloud bootstrap script:wget https://binary.mirantis.com/releases/get_container_cloud.sh chmod 0755 get_container_cloud.sh ./get_container_cloud.sh
Obtain
kubeconfig
of the required cluster. The management or regional clusterkubeconfig
files are created during the last stage of the management or regional cluster bootstrap. To obtain a managed clusterkubeconfig
, see Connect to a Mirantis Container Cloud cluster.Obtain the private SSH key of the required cluster:
For a managed cluster, this is an SSH key added in the Container Cloud web UI before the managed cluster creation.
For a management or regional cluster,
ssh_key
is created during bootstrap of the corresponding cluster in the same directory as the bootstrap script.Note
If the initial version of your Container Cloud management cluster was earlier than 2.6.0,
ssh_key
is namedopenstack_tmp
and is located at~/.ssh/
.
Depending on the cluster type that you require logs from, run the corresponding command:
For a management cluster:
./container-cloud collect logs --management-kubeconfig <pathToMgmtClusterKubeconfig> \ --key-file <pathToMgmtClusterPrivateSshKey> \ --cluster-name <clusterName> --cluster-namespace <clusterProject>
For a regional cluster:
./container-cloud collect logs --management-kubeconfig <pathToMgmtClusterKubeconfig> \ --key-file <pathToRegionalClusterSshKey> --kubeconfig <pathToRegionalClusterKubeconfig> \ --cluster-name <clusterName> --cluster-namespace <clusterProject>
For a managed cluster:
./container-cloud collect logs --management-kubeconfig <pathToMgmtClusterKubeconfig> \ --key-file <pathToManagedClusterSshKey> --kubeconfig <pathToManagedClusterKubeconfig> \ --cluster-name <clusterName> --cluster-namespace <clusterProject>
Substitute the parameters enclosed in angle brackets with the corresponding values of your cluster.
Optionally, add
--output-dir
that is a directory path to collect logs. The default value islogs/
. For example,logs/<clusterName>/events.log
.For the logs structure, see Collect the bootstrap logs.
If you require logs of a cluster update, inspect the following folders on the control plane nodes:
/objects/namespaced/<namespaceName>/core/pods/lcm-lcm-controller- <controllerID>/
for thelcm-controller
logs./objects/namespaced/<namespaceName>/core/pods/<cloudProviderName-ID>/
for logs of the cloud provider controller. For example,vsphere-provider-5b96fb4fd6-bhl7g
./system/mke/<controllerMachineName>/
(or/system/<controllerMachineName>/mke/
) for the MKE support dump. Thedsinfo/dsinfo.txt
file contains Docker and system information of the MKE configuration set before and after update.events.log
for cluster events logs.
Technology Preview. For bare metal clusters, assess the Ironic pod logs:
Extract the content of the
'message'
fields from every log message:kubectl -n kaas logs <ironicPodName> -c syslog | jq -rM '.message'
Extract the content of the
'message'
fields from theironic_conductor
source log messages:kubectl -n kaas logs <ironicPodName> -c syslog | jq -rM 'select(.source == "ironic_conductor") | .message'
The syslog container collects logs generated by Ansible during the node deployment and cleanup and outputs them in the JSON format.
Compress the collected log files and send the archive to the Mirantis support team.