Collect cluster logs

Collect cluster logsΒΆ

While operating your management, regional, or managed cluster, you may require collecting and inspecting the cluster logs to analyze cluster events or troubleshoot issues.

To collect cluster logs:

  1. Select from the following options:

    • If you did not delete the kaas-bootstrap folder from the bootstrap node, log in to the bootstrap node.

    • If you deleted the kaas-bootstrap folder:

      1. Log in to a local machine running Ubuntu 18.04 where kubectl is installed.

      2. Download and run the Container Cloud bootstrap script:

        wget https://binary.mirantis.com/releases/get_container_cloud.sh
        
        chmod 0755 get_container_cloud.sh
        
        ./get_container_cloud.sh
        
  2. Obtain kubeconfig of the required cluster. The management or regional cluster kubeconfig files are created during the last stage of the management or regional cluster bootstrap. To obtain a managed cluster kubeconfig, see Connect to a Mirantis Container Cloud cluster.

  3. Obtain the private SSH key of the required cluster:

    • For a managed cluster, this is an SSH key added in the Container Cloud web UI before the managed cluster creation.

    • For a management or regional cluster, ssh_key is created during bootstrap of the corresponding cluster in the same directory as the bootstrap script.

      Note

      If the initial version of your Container Cloud management cluster was earlier than 2.6.0, ssh_key is named openstack_tmp and is located at ~/.ssh/.

  4. Depending on the cluster type that you require logs from, run the corresponding command:

    • For a management cluster:

      kaas collect logs --management-kubeconfig <pathToMgmtClusterKubeconfig> \
      --key-file <pathToMgmtClusterPrivateSshKey> \
      --cluster-name <clusterName> --cluster-namespace <clusterProject>
      
    • For a regional cluster:

      kaas collect logs --management-kubeconfig <pathToMgmtClusterKubeconfig> \
      --key-file <pathToRegionalClusterSshKey> --kubeconfig <pathToRegionalClusterKubeconfig> \
      --cluster-name <clusterName> --cluster-namespace <clusterProject>
      
    • For a managed cluster:

      kaas collect logs --management-kubeconfig <pathToMgmtClusterKubeconfig> \
      --key-file <pathToManagedClusterSshKey> --kubeconfig <pathToManagedClusterKubeconfig> \
      --cluster-name <clusterName> --cluster-namespace <clusterProject>
      

    Substitute the parameters enclosed in angle brackets with the corresponding values of your cluster.

    Optionally, add --output-dir that is a directory path to collect logs. The default value is logs/. For example, logs/<clusterName>/events.log.

    For the logs structure, see Collect the bootstrap logs.

  5. Technology Preview. For bare metal clusters, assess the Ironic pod logs:

    • Extract the content of the 'message' fields from every log message:

      kubectl -n kaas logs <ironicPodName> -c syslog | jq -rM '.message'
      
    • Extract the content of the 'message' fields from the ironic_conductor source log messages:

      kubectl -n kaas logs <ironicPodName> -c syslog | jq -rM 'select(.source == "ironic_conductor") | .message'
      

    The syslog container collects logs generated by Ansible during the node deployment and cleanup and outputs them in the JSON format.