Troubleshoot RabbitMQ¶

To troubleshoot RabbitMQ:

1. Log in to any OpenStack messaging node.

2. Verify the RabbitMQ cluster status:

   rabbitmqctl cluster_status
   

3. Verify the AMQP messaging RabbitMQ status:

   curl -s -o rabbitmqadmin http://$(hostname -s):15672/cli/rabbitmqadmin;
   chmod 0755 ./rabbitmqadmin
   adm_creds=$(salt-call pillar.get rabbitmq:server:admin --out=yaml \
     | egrep ':\ (.+)' | tr '\n' ' ' | sed 's/name: /-u /; s/password: /-p /')
   ./rabbitmqadmin -H $(hostname -s) $adm_creds show overview
   ./rabbitmqadmin -H $(hostname -s) $adm_creds list queues
   ./rabbitmqadmin -H $(hostname -s) $adm_creds list queues vhost name node messages \
     message_stats.publish_details.rate | grep -v " 0"
   ./rabbitmqadmin -H $(hostname -s) $adm_creds -V /openstack list queues vhost name \
   node messages message_stats.publish_details.rate | grep -v " 0"
   

4. List the number of messages and consumers for each queue. Select vhost, otherwise, you will work with the default queue that is /. The number of messages should be low and number of consumers above 0:

   rabbitmqctl list_queues -p /openstack messages consumers name
   

5. Print stuck nodes if any:

   rabbitmqctl eval 'rabbit_diagnostics:maybe_stuck().'