Verify the status of Tungsten Fabric services

Verify the status of Tungsten Fabric services

Caution

This feature is available starting from MOS Ussuri Update.

This section describes how to verify the status of the Tungsten Fabric services using the TF Operator tf-status tool that resides in tf-tool-status containers among the services. Using this tool, you can also verify the status of the Cassandra, ZooKeeper, Kafka, Redis, and RabbitMQ third-party services supported by TF.

A tf-tool-status container provides access to the contrail-status Python script through HTTP. To verify the TF services status, use the following options:

  • Run contrail-status.py directly from the tf-tool-status container console.

  • Send an HTTP request to the Kubernetes node IP, using the tf-tool-status container port, which is 8888 by default. The tf-status pod is deployed as a DaemonSet and does not expose any Kubernetes service. For third-party services, the tool uses the tf-tool-status-party pod.

  • Send an HTTP request to tf-tool-status-aggregator, a Kubernetes microservice that aggregates statuses from all hosts that handle at least one TF service or a third-party service.

    Caution

    For third-party services, due to a limitation, tf-tool-status-aggregator may return an error. To obtain the status of a service pod, use direct access or send an HTTP request to the tf-tool-status container.

  • Verify the status of TF services

  • Verify the status of third-party services


To verify the status of the TF services:

Select from the following options:

  • Run contrail-status.py directly from the console of the tf-tool-status container:

    kubectl -n tf exec <TF-TOOL-STATUS-POD> -- python /root/contrail-status.py
    

    For the debug mode, use the --debug flag:

    kubectl -n tf exec <TF-TOOL-STATUS-POD> -- python /root/contrail-status.py --debug
    
  • Send an HTTP request to the tf-tool-status container:

    1. Obtain the list of nodes that handle tf-tool-status containers:

      kubectl -n tf get pod --selector app=tf-tool-status -o wide
      
    2. For the required host, obtain its port provided by the container:

      kubectl -n tf get pod <TF-TOOL-STATUS-POD> -o custom-columns=NAME:.metadata.name,HOST:.spec.nodeName,HOST-IP:.status.hostIP,PORT:.spec.containers[0].ports[0].hostPort
      

      Example of system response:

      NAME                  HOST        HOST-IP      PORT
      tf-tool-status-xxxx   host_name   10.10.0.12   8888
      
    3. Using the obtained IP and port, request tf-tool-status from any node in the environment. For example:

      curl 10.10.0.12:8888
      
      # or
      
      curl 10.10.0.12:8888/json   # for output in JSON format
      
      # or
      
      curl 10.10.0.12:8888/debug  # for the debug mode
      

      This process may take a certain amount of time depending on the services on the host.

  • Send an HTTP request to the tf-tool-status-aggregator Kubernetes service:

    1. Obtain the CLUSTER-IP address of the Kubernetes aggregator service:

      kubectl -n tf get svc --selector app=tf-tool-status-aggregator -o wide
      

      Example of system response:

      NAME                        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
      tf-tool-status-aggregator   ClusterIP   10.96.145.46   <none>        80/TCP    46h
      

      The service from the example above is provided on port 80.

    2. Obtain the status of the service. Substitute <CLUSTER-IP> with the obtained tf-tool-status-aggregator service CLUSTER-IP.

      Note

      For help with tf-tool-status-aggregator, use curl <CLUSTER-IP>.

      • To obtain the list of tf-status pods and basic information about them:

        curl <CLUSTER-IP>/pod-list
        

        Example of system response:

        Name: tf-tool-status-2h4pq
        IP: 10.10.0.27
        UUID: a2feb3d1-62d5-11ea-b302-02420a0a010b
        HostIP: 10.10.0.27
        Host: ps-ws-w2myzajhovir-1-xskgrqlroqob-server-hqrfdu5kxhcc
        
      • To obtain an aggregated status of all TF services:

        # shows aggregated statuses sorted by host/pod
        curl <CLUSTER-IP>/status
        
        # output in JSON format by host/pod
        curl <CLUSTER-IP>/status/json
        
        # the same as /status
        curl <CLUSTER-IP>/status/node
        
        # shows aggregated statuses sorted by services, such as config-api, webui
        curl <CLUSTER-IP>/status/group
        

To verify the status of third-party services:

  1. Obtain the IP address and port of the tf-tool-status containers for third-party services:

    kubectl -n tf get pod -l tungstenfabric=status-party -o custom-columns=NAME:.metadata.name,IP:.status.podIP,PORT:.spec.containers[0].ports[0].containerPort
    

    Example of system response:

    NAME                                    IP              PORT
    tf-tool-status-party-7dfdb9dbb6-5pw8q   192.168.62.79   8080
    
  2. Request the third-party service status. For example:

    curl 192.168.62.79:8080