Skip to content

Run etcdctlNew#

The etcd maintenance service provides an etcdctl subcommand that runs etcdctl against the etcd instance on the local controller node. The endpoint and the TLS certificates are located for you, so it is not necessary to supply them on the command line.

Use the etcd maintenance service to inspect etcd directly whenever the maintenance service endpoints do not give you what you need, such as when you read maintenance audit records or investigate a cluster issue.

Tip

In MKE 4, etcdctl is run through the etcd maintenance service on a controller node. This is in contrast to MKE 3, in which etcdctl is inside the ucp-kv container and requires that you manually supply the endpoint and TLS certificate paths.

Synopsis#

Run the etcdctl subcommand with the following syntax:

sudo /usr/local/bin/etcd-maintenance etcdctl [flags] -- <etcdctl args>

The -- separator splits the command into two parts:

  • Flags before -- configure the connection to etcd.
  • Everything after -- passes to etcdctl unchanged.

Use sudo when you run the etcdctl command, as reading the etcd certificates requires root privileges.

Important

Do not run defrag through the etcdctl command. Use the maintenance service instead, as detailed in etcd defragmentation. If you trigger a defragmentation here while a scheduled one is running, the two can collide and cost the cluster quorum.

The available flags for the etcdctl subcommand are detailed below:

Flag                                                       Description Default                                               
--k0s-data-dir The k0s data directory, used to locate the etcd certificates. Set this if your cluster uses a data directory other than the default. /var/lib/k0s
--local-etcd-endpoint The endpoint of the etcd instance on the local node. https://127.0.0.1:2379
--config The path to the etcd maintenance service configuration file. /var/lib/etcd-maintenance/config.yaml

Override the connection#

The command supplies the endpoint and the TLS certificates for you. To use different values, either pass the equivalent etcdctl flag after the -- separator or set the corresponding environment variable. If you supply a flag, that takes precedence.

Setting          etcdctl flag Environment variable
Endpoint --endpoints ETCDCTL_ENDPOINTS
CA certificate --cacert ETCDCTL_CACERT
Client certificate --cert ETCDCTL_CERT
Client key --key ETCDCTL_KEY

For example, to use a CA certificate from a different location:

sudo /usr/local/bin/etcd-maintenance etcdctl -- --cacert=/path/to/ca.crt endpoint health

Loopback-only client access

Each etcd instance accepts client connections on the loopback interface of its own node only. You therefore cannot reach the etcd of another controller node by overriding the endpoint. To inspect a different member, connect to that node with SSH and run the etcdctl command from there.

Environment variables and sudo

By default, sudo does not pass the environment variables of your shell to the etcdctl command. To use the environment variable form, either set the variable as part of the command with sudo env, or preserve your environment with sudo -E:

sudo env ETCDCTL_CACERT=/path/to/ca.crt /usr/local/bin/etcd-maintenance etcdctl -- endpoint health

Examples#

Each of the following examples is run from a controller node. To connect to a controller node, use SSH:

ssh -i <path_to_ssh_key> user@<controller_node_ip>

List the etcd members#

sudo /usr/local/bin/etcd-maintenance etcdctl -- member list -w table

Example response:

+------------------+---------+--------+------------------------+------------------------+------------+
|        ID        | STATUS  |  NAME  |       PEER ADDRS       |      CLIENT ADDRS      | IS LEARNER |
+------------------+---------+--------+------------------------+------------------------+------------+
| 50222da16c98c818 | started | node-a | https://10.0.0.11:2380 | https://127.0.0.1:2379 |      false |
| 929424b8f7828d6a | started | node-c | https://10.0.0.13:2380 | https://127.0.0.1:2379 |      false |
| ec77b45d4447fdda | started | node-b | https://10.0.0.12:2380 | https://127.0.0.1:2379 |      false |
+------------------+---------+--------+------------------------+------------------------+------------+

Every member reports the same client address, https://127.0.0.1:2379, because each etcd instance advertises its client API on the loopback interface of its own node. Use the peer address to tell members apart.

Check the health of the local etcd#

sudo /usr/local/bin/etcd-maintenance etcdctl -- endpoint health

Example response:

https://127.0.0.1:2379 is healthy: successfully committed proposal: took = 2.063042ms

This command reports on the local node only. Refer to etcd membership control-plane add/remove issues for how to assess the health of the cluster as a whole.

For detail on the complete set of etcdctl commands, refer to the official etcd documentation.