Switch to Alertmanager-based notifications

Switch to Alertmanager-based notificationsΒΆ

This section describes how to switch from the Push Notification service to the Alertmanager-based notifications. For more information on the Alertmanager-based notifications, see MCP Reference Architecture: StackLight LMA components.

Caution

Before you start, perform the following prerequisite steps:

  1. Upgrade StackLight LMA to the newest version as described in Upgrade StackLight LMA to Build ID 2019.2.0.
  2. For the Salesforce notifications, verify that you have access to the Salesforce instance and have a customer account in Salesforce.

Warning

The Push Notification service uses the md5 hashing algorithm for creating alert IDs for the Salesforce notifications, whereas the Salesforce notifier service uses sha256 by default. Switching to sha256 without migration of Salesforce cases may lead to cases loss from the Salesforce notifier scope. If these services have different hashing set up, case duplication with the same subject and status but different IDs may occur. Therefore, Mirantis recommends explicitly setting the md5 hashing algorithm in your model configuration as described below. Migration of old cases is not supported.

To Switch to the Alertmanager-based notifications:

  1. Open your Git project repository with Reclass model on the cluster level.

  2. Set up the Alertmanager-based notifications:

    • For email notifications, follow the procedure described in Enable email or Slack notifications.

    • For Salesforce notifications:

      1. Set the md5 hashing algorithm in the <cluster>/stacklight/server.yml file:

        _params:
          sf_notifier_alert_id_hash_func: md5
        
      2. Set up the Salesforce notifier service as described in Enable Salesforce notifications.

  3. Disable the Push Notification service:

    1. Open the classes/cluster/<cluster_name>/stacklight/server.yml file for editing.

    2. Remove the following class:

      - system.prometheus.alertmanager.notification.pushkin
      
    3. Remove the following parameters:

      alertmanager_notification_pushkin_host: <host>
      alertmanager_notification_pushkin_port: <port>
      
  4. Log in to the Salt Master node.

  5. Refresh Salt pillars:

    salt '*' saltutil.refresh_pillar
    
  6. Apply the following state:

    salt -C 'I@docker:swarm and I@prometheus:server' state.sls prometheus.alertmanager -b 1
    
  7. (Optional) Remove the Push Notification service:

    1. Verify that only the Push Notification service uses the database in the Docker container.

    2. In the Git project repository with Reclass model on the cluster level, open the classes/cluster/<cluster_name>/stacklight/server.yml file for editing.

    3. Remove the following classes:

      - system.haproxy.proxy.listen.oss.pushkin
      - system.haproxy.proxy.listen.oss.postgresql
      - system.docker.swarm.stack.pushkin
      - system.docker.swarm.stack.postgresql
      - system.docker.swarm.network.oss_backend
      - system.postgresql.client.pushkin
      - system.postgresql.client.pushkin.sfdc
      - system.glusterfs.server.volume.postgresql
      - system.glusterfs.server.volume.pushkin
      - system.glusterfs.client.volume.postgresql
      - system.glusterfs.client.volume.pushkin
      
    4. Remove the following parameters:

      postgresql_client_user: 'postgres'
      postgresql_client_password: 'postgrespassword'
      
    5. Log in to the Salt Master node.

    6. Apply the changes:

      salt -C 'I@haproxy:proxy' state.sls haproxy.proxy
      salt -C 'I@docker:swarm and I@prometheus:server' state.sls docker.client
      
    7. Remove the Push Notification service from Docker Swarm:

      salt -C 'I@docker:swarm and I@prometheus:server' cmd.run 'docker stack rm pushkin'
      
    8. Remove the PostgreSQL service from Docker Swarm:

      salt -C 'I@docker:swarm and I@prometheus:server' cmd.run 'docker stack rm  postgresql'
      
    9. Remove the Docker images for the Push Notification service and PostgreSQL:

      salt -C 'I@docker:swarm and I@prometheus:server' cmd.run 'docker rmi <postgres_image> <pushkin_image>'
      
    10. Stop the Push Notification service and PostgreSQL:

      salt -C 'I@docker:swarm:role:master and I@prometheus:server' cmd.run 'echo "y" | gluster volume stop pushkin'
      salt -C 'I@docker:swarm:role:master and I@prometheus:server' cmd.run 'echo "y" | gluster volume stop postgresql'
      
    11. Delete the volumes:

      salt -C 'I@docker:swarm:role:master and I@prometheus:server' cmd.run 'echo "y" | gluster volume delete pushkin'
      salt -C 'I@docker:swarm:role:master and I@prometheus:server' cmd.run 'echo "y" | gluster volume delete postgresql'
      
    12. Remove the directories for GlusterFS bricks:

      salt -C 'I@docker:swarm and I@prometheus:server' cmd.run 'rm -rf /srv/glusterfs/pushkin'
      salt -C 'I@docker:swarm and I@prometheus:server' cmd.run 'rm -rf /srv/glusterfs/postgres'
      
    13. (Optional) Uninstall GlusterFS:

      sudo apt remove -y glusterfs-server