Verify the status of Tungsten Fabric services

Available since MOS Ussuri Update

This section describes how to verify the status of the Tungsten Fabric services using the TF Operator tf-status tool that resides in tf-tool-status containers among the services. Using this tool, you can also verify the status of the Cassandra, ZooKeeper, Kafka, Redis, and RabbitMQ third-party services supported by TF.

A tf-tool-status container provides access to the contrail-status Python script through HTTP. To verify the TF services status, use the following options:

  • Run directly from the tf-tool-status container console.

  • Send an HTTP request to the Kubernetes node IP, using the tf-tool-status container port, which is 8888 by default. The tf-status pod is deployed as a DaemonSet and does not expose any Kubernetes service. For third-party services, the tool uses the tf-tool-status-party pod.

  • Send an HTTP request to tf-tool-status-aggregator, a Kubernetes microservice that aggregates statuses from all hosts that handle at least one TF service or a third-party service.


    For third-party services, due to a limitation, tf-tool-status-aggregator may return an error. To obtain the status of a service pod, use direct access or send an HTTP request to the tf-tool-status container.

  • Verify the status of TF services

  • Verify the status of third-party services

To verify the status of the TF services:

Select from the following options:

  • Run directly from the console of the tf-tool-status container:

    kubectl -n tf exec <TF-TOOL-STATUS-POD> -- python /root/

    For the debug mode, use the --debug flag:

    kubectl -n tf exec <TF-TOOL-STATUS-POD> -- python /root/ --debug
  • Send an HTTP request to the tf-tool-status container:

    1. Obtain the list of nodes that handle tf-tool-status containers:

      kubectl -n tf get pod --selector app=tf-tool-status -o wide
    2. For the required host, obtain its port provided by the container:

      kubectl -n tf get pod <TF-TOOL-STATUS-POD> -o,HOST:.spec.nodeName,HOST-IP:.status.hostIP,PORT:.spec.containers[0].ports[0].hostPort

      Example of system response:

      NAME                  HOST        HOST-IP      PORT
      tf-tool-status-xxxx   host_name   8888
    3. Using the obtained IP and port, request tf-tool-status from any node in the environment. For example:

      # or
      curl   # for output in JSON format
      # or
      curl  # for the debug mode

      This process may take a certain amount of time depending on the services on the host.

  • Send an HTTP request to the tf-tool-status-aggregator Kubernetes service:

    1. Obtain the CLUSTER-IP address of the Kubernetes aggregator service:

      kubectl -n tf get svc --selector app=tf-tool-status-aggregator -o wide

      Example of system response:

      NAME                        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
      tf-tool-status-aggregator   ClusterIP   <none>        80/TCP    46h

      The service from the example above is provided on port 80.

    2. Obtain the status of the service. Substitute <CLUSTER-IP> with the obtained tf-tool-status-aggregator service CLUSTER-IP.


      For help with tf-tool-status-aggregator, use curl <CLUSTER-IP>.

      • To obtain the list of tf-status pods and basic information about them:

        curl <CLUSTER-IP>/pod-list

        Example of system response:

        Name: tf-tool-status-2h4pq
        UUID: a2feb3d1-62d5-11ea-b302-02420a0a010b
        Host: ps-ws-w2myzajhovir-1-xskgrqlroqob-server-hqrfdu5kxhcc
      • To obtain an aggregated status of all TF services:

        # shows aggregated statuses sorted by host/pod
        curl <CLUSTER-IP>/status
        # output in JSON format by host/pod
        curl <CLUSTER-IP>/status/json
        # the same as /status
        curl <CLUSTER-IP>/status/node
        # shows aggregated statuses sorted by services, such as config-api, webui
        curl <CLUSTER-IP>/status/group

To verify the status of third-party services:

  1. Obtain the IP address and port of the tf-tool-status containers for third-party services:

    kubectl -n tf get pod -l tungstenfabric=status-party -o,IP:.status.podIP,PORT:.spec.containers[0].ports[0].containerPort

    Example of system response:

    NAME                                    IP              PORT
    tf-tool-status-party-7dfdb9dbb6-5pw8q   8080
  2. Request the third-party service status. For example: