Alertmanager

Alertmanager

This section describes the alerts for the Alertmanager service.


AlertmanagerNotificationFailureWarning

Severity Warning
Summary Alertmanager has {{ $value }} failed notifications for {{$labels.integration }} on the {{ $labels.instance}} instance for 2 minutes.
Raise condition increase(alertmanager_notifications_failed_total[2m]) > 0
Description

Raises when Alertmanager fails to send a notification to a specific receiver or channel for the following exemplary reasons:

  • Slack: wrong API key or channel name
  • Email: authentication issues or the SMTP server is not available
  • Webhook: the server is not available
Troubleshooting Run docker service logs monitoring_alertmanager on any mon node and inspect the Alertmanager logs.
Tuning Not required

AlertmanagerAlertsInvalidWarning

Severity Warning
Summary An average of {{ $value }} Alertmanager {{$labels.integration }} alerts on the {{ $labels.instance }} instance are invalid for 2 minutes.
Raise condition increase(alertmanager_alerts_invalid_total[2m]) > 0
Description

Raises when Alertmanager receives an alert with errors, for example:

  • Missing start and end time
  • Empty labels
  • Wrong labels or annotations. The key or value must not be empty, must start with a char and must be alphanumeric.
Troubleshooting Run docker service logs monitoring_alertmanager on any mon node and inspect the Alertmanager logs.
Tuning Not required