Datadog Monitoring Implementation Guide for Docker Enterprise¶

Git client command¶

The Git client command was used in this Solution Brief to clone the Kubernetes State Metrics GitHub Repository. It was run on the Docker Enterprise client machine as part of the setup to deploy Kubernetes State Metrics prior to deploying the Elastic stack with Kubernetes Monitoring Integration on Docker Enterprise 3.0. Refer to the Git Getting Started - Installing Git web page for details on how to install the Git client if you need to install it.

curl Command¶

Some of the commands in this Solution Brief use the curl command.

If the curl command is not installed you can install it using the instructions below for the Linux distributions:

• Debian/Ubuntu

  apt-get update -qq;apt-get install curl -y
  

• CentOS/RHEL

  yum makecache fast;yum install curl -y
  

• For other Linux distributions you can download and install it from https://curl.haxx.se/download.html.

jq command¶

Some of the commands in this Solution Brief use the jq command to format and display json output.

If the jq command is not installed you can install it using the instructions below for the Linux distributions:

• Debian/Ubuntu

  $ apt-get update -qq;apt-get install jq -y
  

• CentOS/RH

  $ yum makecache fast;yum install jq -y
  

• For other Linux distributions you can download and install it from https://github.com/stedolan/jq/wiki/Installation.

Generate and download a MKE client bundle using the MKE Rest API¶

You can download a sample bash script named get-docker-ee-ucp-client-bundle.sh from this GitHub Repository get-docker-ee-ucp-client-bundle which uses the Docker Enterprise MKE Rest API to generate and download a Docker Enterprise MKE client bundle. You can then run the script which will download a Docker Enterprise MKE client bundle.

1. Download the sample bash script get-docker-ee-ucp-client-bundle

2. Export your Docker Enterprise MKE user account and password as environment variables.

   Example:

   $ export DOCKER_USER="**your-docker-ee-ucp-user-account**"
   $ export DOCKER_PASSWORD="**your-docker-ee-ucp-password**"
   

3. Generate and download a Docker Enterprise MKE Client Bundle.

   Run the following command from the Docker Enterprise command shell. Replace manager.example.com with the hostname or IP address of your Docker Enterprise MKE manager.

   $ ./get-docker-ee-ucp-client-bundle.sh -d manager.example.com
   

   Example:

   

4. Configure your Docker Enterprise client command shell.

   Run the following command from the Docker Enterprise command shell.

   $ eval "$(<env.sh)"
   

   Example:

   

5. Test the Docker Enterprise MKE client bundle and configuration.

   Run the docker version command from the Docker Enterprise client command shell.

   $ docker version --format '{{println .Server.Platform.Name}}Client: {{.Client.Version}}{{range .Server.Components}}{{println}}{{.Name}}: {{.Version}}{{end}}'
   

   Example:

   

Generate and download a MKE client bundle from your MKE Web UI¶

If you prefer, you can generate and download a Docker Enterprise MKE client bundle from your Docker Enterprise MKE Web UI.

1. Login to your Docker Enterprise MKE Cluster Web UI using your Docker ID and password.

   

2. Click on your account name and then click on My Profile

   

3. Click on the New Client Bundle button then Generate Client Bundle button.

   

4. Locate the generated client bundle archive file and unzip it.

   Note

   The generated client bundle archive file will be downloaded to whatever folder your browser’s Download folder is configured for. You may have to move the generated client bundle archive file to the Docker Enterprise client machine if it is a different machine than the machine you downloaded it on.

   Run the following command from the Docker Enterprise client command shell to unzip the client bundle archive file. Substitute your generated client bundle archive file name.

   $ unzip ucp-bundle-admin.zip
   

   Example:

   

5. Configure your Docker Enterprise client command shell.

   Run the following command from the Docker Enterprise command shell.

   $ eval "$(<env.sh)"
   

   Example:

   

6. Test the Docker Enterprise MKE client bundle and configuration.

   Run the docker version command from the Docker Enterprise client command shell.

   $ docker version --format '{{println .Server.Platform.Name}}Client: {{.Client.Version}}{{range .Server.Components}}{{println}}{{.Name}}: {{.Version}}{{end}}'
   

   Example:

   

Deploy Kubernetes State Metrics¶

1. The Kubernetes Manifest files to deploy Kubernetes State Metrics are hosted and maintained on GitHub in the kube-state-metrics Repository. Run the following git command below from the Docker Enterprise client command shell to clone the kube-state-metrics GitHub repository.

   $ git clone https://github.com/kubernetes/kube-state-metrics.git
   

2. Deploy Kubernetes State Metrics.

   Run the following commands from the MKE client command shell to deploy Kubernetes State Metrics.

   $ cd kube-state-metrics
   $ kubectl apply -f examples/standard
   

3. Display the Kubernetes State Metrics Deployment.

   Run the following command from the MKE client command shell.

     $ kubectl get deployment kube-state-metrics --namespace kube-system -o wide
   
   Example:
   
   .. image:: ../_images/datadog/display_kube_state_metrics_deployment.png
      :width:
      :alt: Display Kubernetes State Metrics
   

4. Display the pod deployed by the Kubernetes State Metrics Deployment.

   Run the following command from the MKE client command shell.

     $ kubectl get pods --namespace kube-system -o wide
   
   Example:
   
   .. image:: ../_images/datadog/display_kube_state_metrics_deployment_pod.png
      :width: 100%
      :alt: Display Kubernetes State Metrics Pod
   

5. You can display detailed information on the Kubernetes State Metrics pod.

   Run the following kubectl describe pod command below from the MKE client command shell. Change the pod name to match the Kubernetes State Metrics pod which was deployed.

   $ kubectl describe pod --namespace kube-state-metrics-78f8b6786b-qg8wc
   

6. Check the Kubernetes State Metrics pod logs for any errors.

   Note

   A Kubernetes State Metrics pod contains 2 containers: kube-state-metrics and addon-resizer.

   Run the following kubectl logs commands below from the Docker MKE client command shell. Change the pod name to match the Kubernetes State Metrics pod which was deployed.

   $ kubectl logs --namespace kube-system kube-state-metrics-78f8b6786b-qg8wc --container kube-state-metrics | grep -i 'Error'
   $ kubectl logs --namespace kube-system kube-state-metrics-78f8b6786b-qg8wc --container addon-resizer | grep -i 'Error'
   

7. Create a Kubernetes secret named dd-api-key to contain the value of your Datadog API Key.

   Run the following command from the Docker Enterprise client command shell. Change the text **datadog-api-key** to your Datadog API Key.

   $ kubectl create secret generic dd-api-key --namespace kube-system --from-literal=api-key='**datadog-api-key**'
   

   Example:

   

8. Download the sample deploy-datadog-agent-with-kubernetes-monitoring-integration.yaml included with this solution brief which contains the Kubernetes DaemonSet manifest to deploy the Datadog agent at this link: deploy-datadog-agent-with-kubernetes-monitoring-integration.yaml

   Note

   The sample deploy-datadog-agent-with-kubernetes-monitoring-integration.yaml included with this solution brief has been customized for Docker Enterprise. Datadog provides sample Kubernetes daemonset yaml files at this link: https://github.com/DataDog/datadog-agent. For additional information on deploying the Datadog agent on Kubernetes refer to: Kubernetes DaemonSet Setup.

9. Deploy the Datadog agents with Kubernetes Monitoring Integration on all Docker Enterprise MKE Cluster nodes.

   Run the following command from the Docker Enterprise client command shell.

   $ kubectl apply -f deploy-datadog-agent-with-kubernetes-monitoring-integration.yaml
   

   Example:

   

10. Wait a few minutes for the Datadog environment to start up and become fully initialized.

11. Display the Datadog Agent DaemonSet.

    Run the following command from the Docker Enterprise client command shell.

    $ kubectl get daemonset --namespace kube-system datadog-agent -o wide
    

    Example:

    

12. Display the Datadog Agent DaemonSet Pods.

    Run the following command from the Docker Enterprise client command shell.

    $ kubectl get pods --namespace kube-system --selector app=datadog-agent -o wide
    

    Example:

    

13. You can display detailed information on a Datadog agent DaemonSet Pod.

    Run the following kubectl describe pod command below from the Docker Enterprise client command shell. Change the pod name to match a Datadog agent pod which was deployed.

    $ kubectl describe pod --namespace kube-system datadog-agent-fwgft
    

14. You can check a Datadog agent DaemonSet Pod logs for any errors.

    Run the following kubectl logs command below from the Docker Enterprise client command shell. Change the pod name to match a Datadog agent pod which was deployed.

    $ kubectl logs --namespace kube-system datadog-agent-fwgft | grep -i 'Error'
    

Install Datadog Windows Integration¶

1. Click on the Integrations link in the left frame, then click on the Integrations menu item.

   

2. Locate and click on the Windows Service integration icon/button to install it.

3. Click on the Configuration tab and then click on the Install Integration button. The install takes a few seconds.

4. After the installation completes, refresh the page and scroll to the top. The Windows Service Integration will appear as Installed.

   

Install the Windows Datadog agent on a Windows Docker Enterprise MKE node¶

1. Click on the Integrations link in the left frame, then click on the Agent menu item.

   

2. Right click the Download the Datadog Agent installer link and copy the link.

   

3. Run the following iwr command from a from a Windows Powershell command prompt on the Docker Enterprise Windows Node to download the Datadog Agent installer. Replace **datadog-windows-installer-download-link** with the link you copied in the previous step.

   > iwr **datadog-windows-installer-download-link** -outfile datadog-agent-installer-windows.msi

Example:

.. image:: ../_images/datadog/install_datadog_agent_on_windows4.png
   :width: 100%
   :alt: Install Datadog Agent on Windows

4. Run the following command from a Windows Powershell command prompt on the Docker Enterprise Windows Node. Change **datadog-api-key** to your Datadog API Key.

   Note: Refer to Datadog Basic Agent Usage for Windows documentation for more details on installing the Datadog Agent on Windows including the installation parameters.

    Start-Process -Wait msiexec -ArgumentList '/qn /i datadog-agent-installer-windows.msi APM_ENABLED="true" LOGS_ENABLED="true" PROCESS_ENABLED="true" APIKEY="**datadog-api-key**"'

Example:

.. image:: ../_images/datadog/install_datadog_agent_on_windows5.png
   :width: 100%
   :alt: Install Datadog Agent on Windows

.. Note::

   The Datadog Agent is being installed silently in the
   background. It may take a few minutes for the installation to
   complete.

5. Run the following command from a Windows Powershell command prompt on the Docker Enterprise Windows Node to display the Datadog Agent Windows services.

   > get-service -name "Datadog*"

Example:

.. image:: ../_images/datadog/display_datadog_agent_on_windows.png
   :width: 100%
   :alt: Display Datadog Agent on Windows

Datadog Events¶

Datadog Dashboards¶

Out of the box Datadog provides 3 system dashboards:

1. System - Disk I/O
2. System - Metrics
3. System - Networking

System - Disk I/O¶

System - Metrics¶

System - Networking¶

Integration with Docker¶

Integration with Kubernetes¶

Install the Kubernetes Integration (Dashboard)¶

Datadog provides extensive Kubernetes Integration which needs to be installed. To install the Kubernetes Integration:

1. Follow the same process as above to install Kubernetes Integration
2. After the installation completes, refresh the page and scroll to the top. The Kubernetes Integration will appear as Installed.

Infrastructure¶

View Docker Infrastructure¶

You can view the Docker Infrastructure dashboards from the Infrastructure link in the left frame:

Metrics¶

Search and graph any metrics collected by Datadog from your infrastructure environment.