Alertmanager does not send resolve notifications for custom alerts

Due to the Alertmanager issue, Alertmanager loses the in-memory alerts during restart. As a result, StackLight does not send notifications for custom alerts in the following case:

  1. Adding a custom alert.

  2. Then removing the custom alert and at the same time changing the Alertmanager configuration such as adding or removing a receiver.

For a removed custom alert, Alertmanager does not send a resolve notification to any of the configured receivers. Therefore, until after the time set in repeat_interval (3 hours by default), the alert will be visible in all receivers but not in the Prometheus and Alertmanager web UIs.

When the alert is re-added, Alertmanager does not send a firing notification for it until after the time set in repeat_interval, but the alert will be visible in the Prometheus and Alertmanager web UIs.