Alertmanager
This section describes the alerts for the Alertmanager service.
AlertmanagerNotificationFailureWarning
Severity |
Warning |
Summary |
Alertmanager has {{ $value }} failed notifications for
{{$labels.integration }} on the {{ $labels.instance}} instance
for 2 minutes. |
Raise condition |
increase(alertmanager_notifications_failed_total[2m]) > 0
|
Description |
Raises when Alertmanager fails to send a notification to a specific
receiver or channel for the following exemplary reasons:
Slack: wrong API key or channel name
Email: authentication issues or the SMTP server is not available
Webhook: the server is not available
|
Troubleshooting |
Run docker service logs monitoring_alertmanager on any mon node
and inspect the Alertmanager logs. |
Tuning |
Not required |
AlertmanagerAlertsInvalidWarning
Severity |
Warning |
Summary |
An average of {{ $value }} Alertmanager {{$labels.integration }}
alerts on the {{ $labels.instance }} instance are invalid for 2
minutes. |
Raise condition |
increase(alertmanager_alerts_invalid_total[2m]) > 0
|
Description |
Raises when Alertmanager receives an alert with errors, for example:
Missing start and end time
Empty labels
Wrong labels or annotations. The key or value must not be empty,
must start with a char and must be alphanumeric.
|
Troubleshooting |
Run docker service logs monitoring_alertmanager on any mon node
and inspect the Alertmanager logs. |
Tuning |
Not required |