Configure notifications subroutes¶

You can configure subroutes for the required notifications. For example, you can configure StackLight to send some notifications to email while also selecting the ones from these that have the CRITICAL severity and sending them to PagerDuty as well.

The changes performed within the following procedure are backward compatible with previous MCP releases and will not affect any of the already configured Alertmanager receivers and routes.

To configure notifications subroutes:

1. Open the classes/cluster/<cluster_name>/stacklight/server.yml file of the Reclass model for editing.

2. Configure the the subroute as required:

   parameters:
     prometheus:
       alertmanager:
         enabled: true
         config:
           route:
             routes:
               <route name>:
                 receiver: <receiver name>
                   [...]
                 routes:  # indicates the subroutes to configure
                   <subroute_name>:
                     receiver: '<subroute receiver name>'
                       [...]
                  [...]
   

   For example, to send the Apache service alerts to email but filter the ones with CRITICAL severity and send them to PagerDuty:

   parameters:
     prometheus:
       alertmanager:
         enabled: true
         config:
           route:
             routes:
               email_alerts:
                 receiver: team-X-email
                 match_re:
                   - label: service
                     value: apache
                 routes:
                   pager_alerts:
                     match:
                       severity: critical
                     receiver: team-X-pager
                 continue: true
   

3. Log in to the Salt Master node.

4. Verify the Reclass model:

   reclass -i
   

   The output should not include any errors.

5. Verify that the changes render properly:

   salt '*mon*' state.sls prometheus.alertmanager test=True
   

   Example of system response for the configuration provided above:

   [...]
   route:
     receiver: default
   [...]
     routes:
     # email_alers
     - receiver: team-X-email
       continue: True
       match:
         service: apache
         routes:
         # pager_alerts
         - receiver: team-X-pager
           match:
             severity: critical
   [...]
   

6. Apply the changes:

   salt '*mon*' state.sls prometheus.alertmanager