Configure Alertmanager

Configure AlertmanagerΒΆ

The configuration of Alertmanager is stored in the prometheus:alertmanager section of the Reclass model. For available configuration settings, see the Alertmanager documentation.

To configure Alertmanager:

  1. Log in to the Salt Master node.

  2. Configure the prometheus:alertmanager section in the classes/cluster/cluster_name/stacklight/server.yml file of the Reclass model as required.

  3. Apply the Salt formula:

    salt -C 'I@docker:swarm:role:master and I@prometheus:server' state.sls prometheus.alertmanager
    

Example configuration:

prometheus:
  alertmanager:
    enabled: true
    bind:
      address: 0.0.0.0
      port: 9093
    config:
      global:
        resolve_timeout: 5m
      route:
        group_by: ['alertname', 'region', 'service']
        group_wait: 60s
        group_interval: 5m
        repeat_interval: 3h
        receiver: HTTP-notification
      inhibit_rules:
        - source_match:
            severity: 'down'
          target_match:
            severity: 'critical'
          equal: ['region', 'service']
        - source_match:
            severity: 'down'
          target_match:
            severity: 'warning'
          equal: ['region', 'service']
        - source_match:
            severity: 'critical'
         target_match:
            severity: 'warning'
          equal: ['alertname', 'region', 'service']
      receivers:
        - name: 'HTTP-notification'
          webhook_configs:
            - url: http://127.0.0.1
              send_resolved: true