General

This section lists the general alerts for Kubernetes nodes.

• FileDescriptorUsageMajor

• FileDescriptorUsageWarning

• NodeDown

• NodeExporterCollectorFailure

• NodeExporterTargetDown

• NodeExporterTargetsOutage

• SystemCpuFullWarning

• SystemLoadTooHighWarning

• SystemLoadTooHighCritical

• SystemDiskFullInFourDays

• SystemDiskFullWarning

• SystemDiskFullCritical

• SystemMemoryFullWarning

• SystemMemoryFullMajor

• SystemDiskInodesFullWarning

• SystemDiskInodesFullMajor

• SystemUptimeCritical

• SystemUptimeWarning

• ZombieProcessesDetected

---

FileDescriptorUsageMajor

Severity	Major
Summary	90% of file descriptors are used.
Description	File descriptor usage reached 90% on the {{ $labels.node }} node ({{ $labels.machine_name }} machine).

FileDescriptorUsageWarning

Severity	Warning
Summary	80% of file descriptors are used.
Description	File descriptor usage reached 80% on the {{ $labels.node }} node ({{ $labels.machine_name }} machine).

NodeDown

Severity	Critical
Summary	{{ $labels.node }} node ({{ $labels.machine_name }} machine) is down.
Description	The {{ $labels.node }} node ({{ $labels.machine_name }} machine) is down. During the last 2 minutes Kubernetes treated the node as Not Ready or Unknown and kubelet was not accessible from Prometheus.

NodeExporterCollectorFailure

Severity	Warning
Summary	Node Exporter failure detected for {{ $labels.collector }} collector.
Description	The {{ $labels.collector }} collector has failed to scrape at least once in the last 20 minutes on {{ $value }} node(s).

NodeExporterTargetDown

Severity	Critical
Summary	Node Exporter Prometheus target is down.
Description	Prometheus fails to scrape metrics from the Node Exporter endpoint on the {{ $labels.node }} node ({{ $labels.machine_name }} machine).

NodeExporterTargetsOutage

Severity	Critical
Summary	Node Exporter Prometheus targets outage.
Description	Prometheus fails to scrape metrics from all Node Exporter endpoints.

SystemCpuFullWarning

Severity	Warning
Summary	High CPU usage.
Description	The average CPU usage on the {{ $labels.node }} node ({{ $labels.machine_name }} machine) is {{ $value }}% for 2 minutes.

SystemLoadTooHighWarning

Severity	Warning
Summary	System load is more than 1 per CPU.
Description	The system load per CPU on the {{ $labels.node }} node ({{ $labels.machine_name }} machine) is {{ $value }} for 5 minutes.

SystemLoadTooHighCritical

Severity	Critical
Summary	System load is more than 2 per CPU.
Description	The system load per CPU on the {{ $labels.node }} node ({{ $labels.machine_name }} machine) is {{ $value }} for 5 minutes.

SystemDiskFullInFourDays

Severity	Warning
Summary	Partition {{ $labels.device }} is expected to run out of space within 4 days.
Description	The {{ $labels.device }} disk partition on the {{ $labels.node }} node ({{ $labels.machine_name }} machine) is expected to run out of space within four days. Currently, {{ printf "%.2f" $value }}% of free space is available.

SystemDiskFullWarning

Severity	Warning
Summary	Partition {{ $labels.device }} usage is above 85% with free space under 100 GiB.
Description	The {{ $labels.device }} disk partition on the {{ $labels.node }} node ({{ $labels.machine_name }} machine) is {{ printf "%.1f" $value }}% full for 2 minutes.

SystemDiskFullCritical

Severity	Critical
Summary	Partition {{ $labels.device }} usage is above 95% with free space under 80 GiB.
Description	The {{ $labels.device }} disk partition on the {{ $labels.node }} node ({{ $labels.machine_name }} machine) is {{ printf "%.1f" $value }}% full for 2 minutes.

SystemMemoryFullWarning

Severity	Warning
Summary	{{ $labels.node }} ({{ $labels.machine_name }}) memory warning usage.
Description	The {{ $labels.node }} node ({{ $labels.machine_name }} machine) uses {{ $value }}% of memory for 10 minutes. More than 90% of memory is used and less than 8 GB of memory is available.

SystemMemoryFullMajor

Severity	Major
Summary	{{ $labels.node }} ({{ $labels.machine_name }}) memory major usage.
Description	The {{ $labels.node }} node ({{ $labels.machine_name }} machine) uses {{ $value }}% of memory for 10 minutes. More than 95% of memory is used and less than 4 GB of memory is available.

SystemDiskInodesFullWarning

Severity	Warning
Summary	85% of inodes for {{ $labels.device }} are used.
Description	The {{ $labels.device }} disk on the {{ $labels.node }} node ({{ $labels.machine_name }} machine) uses {{ printf "%.1f" $value }}% of disk inodes for 2 minutes.

SystemDiskInodesFullMajor

Severity	Major
Summary	95% of inodes for {{ $labels.device }} are used.
Description	The {{ $labels.device }} disk on the {{ $labels.node }} node ({{ $labels.machine_name }} machine) uses {{ printf "%.1f" $value }}% of disk inodes for 2 minutes.

SystemUptimeCritical

Severity	Critical
Summary	System with vulnerable AMD CPU (Erratum 1474) exceeds uptime of 1000 days.
Description	Node {{ $labels.node }} ({{ $labels.machine_name }} machine) has been running for {{ printf "%.0f" $value }} days without reboot. Installed CPU ({{ $labels.model_name }}) might be affected by Erratum 1474 that may cause system crashes after 1044 days of continuous operation.

SystemUptimeWarning

Severity	Warning
Summary	System with vulnerable AMD CPU (Erratum 1474) exceeds uptime of 800 days.
Description	Node {{ $labels.node }} ({{ $labels.machine_name }} machine) has been running for {{ printf "%.0f" $value }} days without reboot. Installed CPU ({{ $labels.model_name }}) might be affected by Erratum 1474 that may cause system crashes after 1044 days of continuous operation.

ZombieProcessesDetected

Severity	Warning
Summary	Zombie processes detected on {{ $labels.node }} ({{ $labels.machine_name }}).
Description	{{ $value }} zombie processes are detected on the {{ $labels.node }} node ({{ $labels.machine_name }} machine).