Kubernetes system

This section lists the alerts for the Kubernetes system.

• KubeNodeNotReady

• KubeStateMetricsTargetDown

• KubeVersionMismatch

• KubeletTargetDown

• KubeletTargetsOutage

• KubeClientErrors

• KubeContainerScrapeError

• KubeDNSTargetsOutage

• KubeletTooManyPods

• cAdvisorTargetDown

• cAdvisorTargetsOutage

• KubeAPIDown

• KubeAPIOutage

• KubeAPIErrorsHighMajor

• KubeAPIErrorsHighWarning

• KubeAPIResourceErrorsHighMajor

• KubeAPIResourceErrorsHighWarning

• KubeClientCertificateExpirationInSevenDays

• KubeClientCertificateExpirationInOneDay

• KubeAPICertExpirationMajor

• KubeAPICertExpirationWarning

• KubernetesApiserverTargetsOutage

• KubernetesMasterAPITargetsOutage

---

KubeNodeNotReady

Severity	Warning
Summary	Node {{ $labels.node }} is not ready.
Description	The {{ $labels.node }} Kubernetes has been unready for more than an hour.

---

KubeStateMetricsTargetDown

Severity	Critical
Summary	kube-state-metrics Prometheus target is down.
Description	Prometheus fails to scrape metrics from the kube-state-metrics endpoint(s) (more than 1/10 failed scrapes).

---

KubeVersionMismatch

Severity	Warning
Summary	Kubernetes components version mismatch.
Description	There are {{ $value }} different semantic versions of Kubernetes components running.

---

KubeletTargetDown

Severity	Critical
Summary	Kubelet Prometheus target is down.
Description	Prometheus fails to scrape metrics from kubelet on the {{ $labels.node }} node (more than 1/10 failed scrapes).

---

KubeletTargetsOutage

Severity	Critical
Summary	Kubelet Prometheus targets outage.
Description	Prometheus fails to scrape metrics from kubelet on all nodes (more than 1/10 failed scrapes).

---

KubeClientErrors

Severity	Warning
Summary	Kubernetes API client has more than 1% of error requests.
Description	The {{ $labels.instance }} Kubernetes API server client has {{ printf "%0.0f" $value }}% errors.

---

KubeContainerScrapeError

Severity	Warning
Summary	Failure to get Kubernetes container metrics.
Description	cAdvisor was not able to scrape metrics from some containers on the {{ $labels.node }} Kubernetes node.

---

KubeDNSTargetsOutage

Severity	Critical
Summary	CoreDNS Prometheus targets outage.
Description	Prometheus fails to scrape metrics from all CoreDNS endpoints (more than 1/10 failed scrapes).

---

KubeletTooManyPods

Severity	Warning
Summary	kubelet reached 90% of Pods limit.
Description	The kubelet container on the {{ $labels.node }} Node is running {{ $value }} Pods, nearly 90% of possible allocation.

---

cAdvisorTargetDown

Severity	Major
Summary	cAdvisor Prometheus target is down.
Description	Prometheus fails to scrape metrics from cAdvisor on the {{ $labels.node }} node (more than 1/10 failed scrapes).

---

cAdvisorTargetsOutage

Severity	Critical
Summary	cAdvisor Prometheus targets outage.
Description	Prometheus fails to scrape metrics from cAdvisor on all nodes (more than 1/10 failed scrapes).

---

KubeAPIDown

Severity	Critical
Summary	A Kubernetes API endpoint is down.
Description	The {{ $labels.node }} Kubernetes API endpoint is not accessible for the last 3 minutes.

---

KubeAPIOutage

Severity	Critical
Summary	Kubernetes API is down.
Description	The Kubernetes API is not accessible for the last 30 seconds.

---

KubeAPIErrorsHighMajor

Severity	Major
Summary	API server is returning errors for more than 3% of requests.
Description	The {{ $labels.instance }} API server is returning errors for {{ $value }}% of requests.

---

KubeAPIErrorsHighWarning

Severity	Warning
Summary	API server is returning errors for more than 1% of requests.
Description	The API server is returning errors for {{ $value }}% of requests.

---

KubeAPIResourceErrorsHighMajor

Severity	Major
Summary	API server is returning errors for 10% of requests.
Description	The {{ $labels.instance }} API server is returning errors for {{ $value }}% of requests for {{ $labels.resource }} {{ $labels.subresource }}.

---

KubeAPIResourceErrorsHighWarning

Severity	Warning
Summary	API server is returning errors for 5% of requests.
Description	The {{ $labels.instance }} API server is returning errors for {{ $value }}% of requests for {{ $labels.resource }} {{ $labels.subresource }}.

---

KubeClientCertificateExpirationInSevenDays

Severity	Warning
Summary	Client certificate expires in 7 days.
Description	The client certificate used to authenticate to the API server expires in less than 7 days.

---

KubeClientCertificateExpirationInOneDay

Severity	Critical
Summary	Client certificate expires in 24 hours.
Description	The client certificate used to authenticate to the API server expires in less than 24 hours.

---

KubeAPICertExpirationMajor

Severity	Major
Summary	Kubernetes API certificate expires in less than 10 days.
Description	The SSL certificate for Kubernetes API expires in less than 10 days.

---

KubeAPICertExpirationWarning

Severity	Warning
Summary	Kubernetes API certificate expires in less than 30 days.
Description	The SSL certificate for Kubernetes API expires in less than 30 days.

---

KubernetesApiserverTargetsOutage

Severity	Critical
Summary	Kubernetes API server Prometheus targets outage.
Description	Prometheus fails to scrape metrics from 2/3 of Kubernetes API server endpoints (more than 1/10 failed scrapes).

---

KubernetesMasterAPITargetsOutage

Severity	Critical
Summary	Kubernetes master API Prometheus targets outage.
Description	Prometheus fails to scrape metrics from 2/3 of Kubernetes master API nodes (more than 1/10 failed scrapes).