Alert dependencies

Using alert inhibition rules, Alertmanager decreases alert noise by suppressing dependent alerts notifications to provide a clearer view on the cloud status and simplify troubleshooting. Alert inhibition rules are enabled by default. The following table describes the dependency between the OpenStack-related alerts. For other alerts that can contain information about both OpenStack and MOSK clusters, see Mirantis Container Cloud Operations Guide: Alert dependencies.

Once an alert from the Alert column raises, the alert from the Inhibits and rules column will be suppressed with the Inhibited status in the Alertmanager web UI.

The Inhibits and rules column lists the labels and conditions, if any, for the inhibition to apply.

Alert

Inhibits and rules

CassandraTombstonesTooManyCritical

CassandraTombstonesTooManyMajor with the same cassandra_cluster, namespace, and pod labels

CassandraTombstonesTooManyMajor

CassandraTombstonesTooManyWarning with the same cassandra_cluster, namespace, and pod labels

CinderServiceOutage

CinderServiceDown with the same binary label

KubeDaemonSetOutage

  • LibvirtExporterTargetsOutage

  • TungstenFabricControllerOutage

  • TungstenFabricControllerTargetsOutage

  • TungstenFabricVrouterOutage

  • TungstenFabricVrouterTargetsOutage

And other alerts described in Mirantis Container Cloud Operations Guide: Alert dependencies.

KubeDeploymentOutage

  • RabbitMQExporterTargetDown for the particular OpenStack service

  • RabbitMQOperatorTargetDown

  • TelegrafOpenstackTargetDown

And other alerts described in Mirantis Container Cloud Operations Guide: Alert dependencies.

KubeStatefulSetOutage

  • CassandraClusterTargetDown Since 23.3

  • CassandraClusterTargetsOutage Before 23.3

  • KafkaClusterTargetDown Since 23.3

  • KafkaClusterTargetsOutage Before 23.3

  • MariadbClusterDown

  • MariadbExporterTargetDown Since 23.3

  • MariadbExporterClusterTargetsOutage Before 23.3

  • MemcachedClusterDown

  • MemcachedExporterTargetDown Since 23.3

  • MemcachedExporterClusterTargetsOutage Before 23.3

  • RabbitMQDown for the particular OpenStack service

  • ZooKeeperClusterTargetDown Since 23.3

  • ZooKeeperClusterTargetsOutage Before 23.3

And other alerts described in Mirantis Container Cloud Operations Guide: Alert dependencies.

LibvirtExporterTargetsOutage

LibvirtExporterTargetDown

MemcachedConnectionsNoneMajor

MemcachedConnectionsNoneWarning with the same namespace label

NeutronAgentOutage

NeutronAgentDown with the same binary label

NodeDown

Alerts with the same node label:

  • LibvirtExporterTargetDown

Since 23.3:

  • CassandraClusterTargetDown

  • KafkaClusterTargetDown

  • MariadbExporterTargetDown

  • MemcachedExporterTargetDown

  • OpenstackCloudproberTargetDown

  • RabbitMQOperatorTargetDown

  • RabbitMQExporterTargetDown

  • RedisClusterTargetDown

  • ZooKeeperClusterTargetDown

And other alerts described in Mirantis Container Cloud Operations Guide: Alert dependencies.

NovaServiceOutage

NovaServiceDown with the same binary label

OpenstackSSLCertExpirationHigh

OpenstackSSLCertExpirationMedium with the same namespace and service_name labels

OsDplSSLCertExpirationHigh

OsDplSSLCertExpirationMedium with the same identifier label

TungstenFabricControllerOutage

TungstenFabricControllerDown

TungstenFabricVrouterOutage

TungstenFabricVrouterDown

TungstenFabricVrouterTargetsOutage

TungstenFabricVrouterTargetDown