Verify the status of Tungsten Fabric services¶

This section describes how to verify the status of the Tungsten Fabric services using the TF Operator tf-status tool that resides in tf-tool-status containers among the services. Using this tool, you can also verify the status of the Cassandra, ZooKeeper, Kafka, Redis, and RabbitMQ third-party services supported by TF.

A tf-tool-status container provides access to the contrail-status Python script through HTTP. To verify the TF services status, use the following options:

• Run contrail-status.py directly from the tf-tool-status container console.

• Send an HTTP request to the Kubernetes node IP, using the tf-tool-status container port, which is 8888 by default. The tf-status pod is deployed as a DaemonSet and does not expose any Kubernetes service. For third-party services, the tool uses the tf-tool-status-party pod.

• Send an HTTP request to tf-tool-status-aggregator, a Kubernetes microservice that aggregates statuses from all hosts that handle at least one TF service or a third-party service.

  Caution

  For third-party services, due to a limitation, tf-tool-status-aggregator may return an error. To obtain the status of a service pod, use direct access or send an HTTP request to the tf-tool-status container.

• Verify the status of TF services

• Verify the status of third-party services

To verify the status of the TF services:

Select from the following options:

• Run contrail-status.py directly from the console of the tf-tool-status container:

  kubectl -n tf exec <TF-TOOL-STATUS-POD> -- python /root/contrail-status.py
  

  For the debug mode, use the --debug flag:

  kubectl -n tf exec <TF-TOOL-STATUS-POD> -- python /root/contrail-status.py --debug
  

• Send an HTTP request to the tf-tool-status container:

  1. Obtain the list of nodes that handle tf-tool-status containers:

     kubectl -n tf get pod --selector app=tf-tool-status -o wide
     

  2. For the required host, obtain its port provided by the container:

     kubectl -n tf get pod <TF-TOOL-STATUS-POD> -o custom-columns=NAME:.metadata.name,HOST:.spec.nodeName,HOST-IP:.status.hostIP,PORT:.spec.containers[0].ports[0].hostPort
     

     Example of system response:

     NAME                  HOST        HOST-IP      PORT
     tf-tool-status-xxxx   host_name   10.10.0.12   8888
     

  3. Using the obtained IP and port, request tf-tool-status from any node in the environment. For example:

     curl 10.10.0.12:8888
     
     # or
     
     curl 10.10.0.12:8888/json   # for output in JSON format
     
     # or
     
     curl 10.10.0.12:8888/debug  # for the debug mode
     

     This process may take a certain amount of time depending on the services on the host.

• Send an HTTP request to the tf-tool-status-aggregator Kubernetes service:

  1. Obtain the CLUSTER-IP address of the Kubernetes aggregator service:

     kubectl -n tf get svc --selector app=tf-tool-status-aggregator -o wide
     

     Example of system response:

     NAME                        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
     tf-tool-status-aggregator   ClusterIP   10.96.145.46   <none>        80/TCP    46h
     

     The service from the example above is provided on port 80.

  2. Obtain the status of the service. Substitute <CLUSTER-IP> with the obtained tf-tool-status-aggregator service CLUSTER-IP.

     Note

     For help with tf-tool-status-aggregator, use curl <CLUSTER-IP>.

     • To obtain the list of tf-status pods and basic information about them:

       curl <CLUSTER-IP>/pod-list
       

       Example of system response:

       Name: tf-tool-status-2h4pq
       IP: 10.10.0.27
       UUID: a2feb3d1-62d5-11ea-b302-02420a0a010b
       HostIP: 10.10.0.27
       Host: ps-ws-w2myzajhovir-1-xskgrqlroqob-server-hqrfdu5kxhcc
       

     • To obtain an aggregated status of all TF services:

       # shows aggregated statuses sorted by host/pod
       curl <CLUSTER-IP>/status
       
       # output in JSON format by host/pod
       curl <CLUSTER-IP>/status/json
       
       # the same as /status
       curl <CLUSTER-IP>/status/node
       
       # shows aggregated statuses sorted by services, such as config-api, webui
       curl <CLUSTER-IP>/status/group
       

To verify the status of third-party services:

1. Obtain the IP address and port of the tf-tool-status containers for third-party services:

   kubectl -n tf get pod -l tungstenfabric=status-party -o custom-columns=NAME:.metadata.name,IP:.status.podIP,PORT:.spec.containers[0].ports[0].containerPort
   

   Example of system response:

   NAME                                    IP              PORT
   tf-tool-status-party-7dfdb9dbb6-5pw8q   192.168.62.79   8080
   

2. Request the third-party service status. For example:

   curl 192.168.62.79:8080