Troubleshoot Cassandra for OpenContrail 4.x

Troubleshoot Cassandra for OpenContrail 4.x

In case of issues with Cassandra not starting in OpenContrail 4.x, the system response of the doctrail analyticsdb contrail-status command on the nal nodes can be as follows:

== Contrail Database ==
contrail-database:            active
kafka:                        active
contrail-database-nodemgr:    initializing (Cassandra state detected DOWN.
                              Disk space for analytics db not retrievable.)

Workaround:

  1. Remove all files from the /var/lib/cassandra/ folder:

    rm -rf /var/lib/cassandra/*
    
  2. Log in to the nal node in question.

  3. Restart the service:

    doctrail analyticsdb service contrail-database restart
    
  4. Verify the Cassandra status is active:

    doctrail analyticsdb contrail-status
    
  5. Verify that the replication status of Cassandra is UN:

    doctrail analyticsdb nodetool status
    

    Example of system response:

    Datacenter: datacenter1
    =======================
    Status=Up/Down
    |/ State=Normal/Leaving/Joining/Moving
    --  Address        Load       Tokens  Owns  Host ID                               Rack
    UN  172.17.98.161  255.56 MB  256     ?     eafd7b42-b977-44fa-8c5d-6bed193ac87f  rack1
    UN  172.17.98.163  389.98 MB  256     ?     89493967-bfdb-4300-bb39-08fb6f6d63f1  rack1
    UN  172.17.98.162  292.7 MB   256     ?     91444200-c370-4a6e-b7f8-cab0e78f39eb  rack1