Configure notifications subroutes

Configure notifications subroutesΒΆ

Note

This feature is available starting from the MCP 2019.2.9 maintenance update. Before using the feature, follow the steps described in Apply maintenance updates.

You can configure subroutes for the required notifications. For example, you can configure StackLight to send some notifications to email while also selecting the ones from these that have the CRITICAL severity and sending them to PagerDuty as well.

The changes performed within the following procedure are backward compatible with previous MCP releases and will not affect any of the already configured Alertmanager receivers and routes.

To configure notifications subroutes:

  1. Open the classes/cluster/<cluster_name>/stacklight/server.yml file of the Reclass model for editing.

  2. Configure the the subroute as required:

    parameters:
      prometheus:
        alertmanager:
          enabled: true
          config:
            route:
              routes:
                <route name>:
                  receiver: <receiver name>
                    [...]
                  routes:  # indicates the subroutes to configure
                    <subroute_name>:
                      receiver: '<subroute receiver name>'
                        [...]
                   [...]
    

    For example, to send the Apache service alerts to email but filter the ones with CRITICAL severity and send them to PagerDuty:

    parameters:
      prometheus:
        alertmanager:
          enabled: true
          config:
            route:
              routes:
                email_alerts:
                  receiver: team-X-email
                  match_re:
                    - label: service
                      value: apache
                  routes:
                    pager_alerts:
                      match:
                        severity: critical
                      receiver: team-X-pager
                  continue: true
    
  3. Log in to the Salt Master node.

  4. Verify the Reclass model:

    reclass -i
    

    The output should not include any errors.

  5. Verify that the changes render properly:

    salt '*mon*' state.sls prometheus.alertmanager test=True
    

    Example of system response for the configuration provided above:

    [...]
    route:
      receiver: default
    [...]
      routes:
      # email_alers
      - receiver: team-X-email
        continue: True
        match:
          service: apache
          routes:
          # pager_alerts
          - receiver: team-X-pager
            match:
              severity: critical
    [...]
    
  6. Apply the changes:

    salt '*mon*' state.sls prometheus.alertmanager