Verify the StackLight LMA components after deployment¶

Once you install the StackLight LMA components as described in Install the StackLight LMA components, verify that all components have been successfully deployed and all services are up and running.

To verify the StackLight LMA components:

1. Log in to the Salt Master node.

2. Verify that all the monitoring services running in Docker Swarm have their expected number of replicas:

   salt -C 'I@docker:client:stack:monitoring' cmd.run 'docker service ls'
   

   Example:

   root@sup01:~# docker service ls
   ID           NAME                              MODE       REPLICAS IMAGE
   j0hrlth0agyx monitoring_server                 replicated 1/1      prometheus:latest
   pqeqda711a69 dashboard_grafana                 replicated 1/1      grafana/grafana:latest
   xrdmspdexojs monitoring_pushgateway            replicated 2/2      pushgateway:latest
   xztynkgfo1pu monitoring_alertmanager           replicated 2/2      alertmanager:latest
   i2xc7j9ei81k monitoring_remote_agent           replicated 1/1      telegraf:latest
   

3. Verify the status of the containers:

   salt -C 'I@docker:swarm:role:master and I@prometheus:server' cmd.run \
   'docker service ps $(docker stack services -q monitoring)'
   

4. Inspect the monitoring containers logs for any unusual entries:

   salt -C 'I@docker:swarm:role:master and I@prometheus:server' cmd.run \
   'for i in $(docker stack services -q monitoring); do docker service logs --tail 10 $i; done'
   

5. Verify that the Fluentd service is running:

   salt -C 'I@fluentd:agent' service.status td-agent
   

6. Verify Prometheus Relay:

   salt -C 'I@prometheus:relay' service.status prometheus-relay
   

7. If deployed, verify Prometheus long-term storage:

   salt -C 'I@prometheus:relay' service.status prometheus
   

8. Verify the Prometheus web UI:

   1. Connect to the Prometheus web UI as described in the corresponding section of the MCP Operations Guide.

   2. From the Status drop-down list, select Targets.

   3. Verify that all targets are in the UP state.

   4. Click the Alerts tab.

   5. Verify that no alerts are active.

9. Verify the Alertmanager web UI:

   1. Connect to the Alertmanager web UI as described in Use the Alertmanager web UI.

   2. Click Alerts.

   3. Verify that no alerts are active.

10. Verify the Grafana dashboards:

    1. Enter the prx VIP on port 3000 by default.

    2. Authenticate using your credentials as described in Connect to Grafana. You should be redirected to the Grafana Home page with a list of available dashboards sorted by name.

    3. Verify that all nodes are listed in the System dashboard.

11. Verify the Kibana dashboards by connecting to Kibana as described in the Connect to Kibana.