OpenStack message bus¶
The internal components of Mirantis OpenStack for Kubernetes (MOSK) coordinate their operations and exchange status information using the cluster’s message bus (RabbitMQ).
Exposable OpenStack notifications¶
Available since MOSK 22.5
MOSK enables you to configure OpenStack services to emit
notification messages to the MOSK cluster messaging bus
(RabbitMQ) every time an OpenStack resource, for example, an instance, image,
and so on, changes its state due to a cloud user action or through its
lifecycle. For example, MOSK Compute service (OpenStack
Nova) can publish the instance.create.end
notification once a newly created
instance is up and running.
Sample of an instance.create.end
notification
{
"event_type": "instance.create.end",
"payload": {
"nova_object.data": {
"action_initiator_project": "6f70656e737461636b20342065766572",
"action_initiator_user": "fake",
"architecture": "x86_64",
"auto_disk_config": "MANUAL",
"availability_zone": "nova",
"block_devices": [],
"created_at": "2012-10-29T13:42:11Z",
"deleted_at": null,
"display_description": "some-server",
"display_name": "some-server",
"fault": null,
"flavor": {
"nova_object.data": {
"description": null,
"disabled": false,
"ephemeral_gb": 0,
"extra_specs": {
"hw:watchdog_action": "disabled"
},
"flavorid": "a22d5517-147c-4147-a0d1-e698df5cd4e3",
"is_public": true,
"memory_mb": 512,
"name": "test_flavor",
"projects": null,
"root_gb": 1,
"rxtx_factor": 1.0,
"swap": 0,
"vcpu_weight": 0,
"vcpus": 1
},
"nova_object.name": "FlavorPayload",
"nova_object.namespace": "nova",
"nova_object.version": "1.4"
},
"host": "compute",
"host_name": "some-server",
"image_uuid": "155d900f-4e14-4e4c-a73d-069cbf4541e6",
"instance_name": "instance-00000001",
"ip_addresses": [
{
"nova_object.data": {
"address": "192.168.1.3",
"device_name": "tapce531f90-19",
"label": "private",
"mac": "fa:16:3e:4c:2c:30",
"meta": {},
"port_uuid": "ce531f90-199f-48c0-816c-13e38010b442",
"version": 4
},
"nova_object.name": "IpPayload",
"nova_object.namespace": "nova",
"nova_object.version": "1.0"
}
],
"kernel_id": "",
"key_name": "my-key",
"keypairs": [
{
"nova_object.data": {
"fingerprint": "1e:2c:9b:56:79:4b:45:77:f9:ca:7a:98:2c:b0:d5:3c",
"name": "my-key",
"public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgQDx8nkQv/zgGgB4rMYmIf+6A4l6Rr+o/6lHBQdW5aYd44bd8JttDCE/F/pNRr0lRE+PiqSPO8nDPHw0010JeMH9gYgnnFlyY3/OcJ02RhIPyyxYpv9FhY+2YiUkpwFOcLImyrxEsYXpD/0d3ac30bNH6Sw9JD9UZHYcpSxsIbECHw== Generated-by-Nova",
"type": "ssh",
"user_id": "fake"
},
"nova_object.name": "KeypairPayload",
"nova_object.namespace": "nova",
"nova_object.version": "1.0"
}
],
"launched_at": "2012-10-29T13:42:11Z",
"locked": false,
"locked_reason": null,
"metadata": {},
"node": "fake-mini",
"os_type": null,
"power_state": "running",
"progress": 0,
"ramdisk_id": "",
"request_id": "req-5b6c791d-5709-4f36-8fbe-c3e02869e35d",
"reservation_id": "r-npxv0e40",
"state": "active",
"tags": [
"tag"
],
"task_state": null,
"tenant_id": "6f70656e737461636b20342065766572",
"terminated_at": null,
"trusted_image_certificates": [
"cert-id-1",
"cert-id-2"
],
"updated_at": "2012-10-29T13:42:11Z",
"user_id": "fake",
"uuid": "178b0921-8f85-4257-88b6-2e743b5a975c"
},
"nova_object.name": "InstanceCreatePayload",
"nova_object.namespace": "nova",
"nova_object.version": "1.12"
},
"priority": "INFO",
"publisher_id": "nova-compute:compute"
}
OpenStack notification messages can be consumed and processed by various corporate systems to integrate MOSK clouds into the company’s infrastructure and business processes.
The list of the most common use cases includes:
Using notification history for retrospective security audit
Using the real-time aggregation of notification messages to gather statistics on cloud resource consumption for further capacity planning
Cloud billing considerations
Notifications alone should not be considered as a source of data for any kind of financial reporting. The delivery of the messages can not be guaranteed due to various technical reasons. For example, messages can be lost if an external consumer is not fetching them from the queue fast enough.
Mirantis strongly recommends that your cloud billing solutions rely on the combination of the following data sources:
Periodic polling of the OpenStack API as a reliable source of information about allocated resources
Subscription to notifications to receive timely updates about the resource status change
If you are looking for a ready-to-use billing solution for your cloud, contact Mirantis or one of our partners.
A cloud administrator can securely expose part of a MOSK cluster message bus to the outside world. This enables an external consumer to subscribe to the notification messages emitted by the cluster services.
Important
The latest OpenStack release available in MOSK supports notifications from the following services:
Block storage (OpenStack Cinder)
DNS (OpenStack Designate)
Image (OpenStack Glance)
Orchestration (OpenStack Heat)
Bare Metal (OpenStack Ironic)
Identity (OpenStack Keystone)
Shared Filesystems (OpenStack Manila)
Instance High Avalability (OpenStack Masakari)
Networking (OpenStack Neutron)
Compute (OpenStack Nova)
To enable the external notification endpoint, add the following structure
to the cluster’s OpenStackDeployment
custom resource. For example:
spec:
features:
messaging:
notifications:
external:
enabled: true
topics:
- external-consumer-A
- external-consumer-2
For each topic name specified in the topics
field, MOSK
creates a topic exchange in its RabbitMQ cluster together with a set of queues
bound to this topic. All enabled MOSK services will
publish their notification messages to all configured topics so that
multiple consumers can receive the same messages in parallel.
A topic name must follow Kubernetes standard format for object names and IDs
that is only lowercase alphanumeric characters, -
, or .
The topic
name notifications
is reserved for the internal use.
Each topic exchange is protected by automatically generated authentication
credentials that are stored as a secret in the openstack-external
namespace
of a MOSK underlying Kubernetes cluster. A secret
is identified by the name of the topic. The list of attributes for the secret
object includes:
hosts
The IP addresses which an external notification endpoint is available on
ports
The TCP ports which external notification endpoint is available on
vhost
The name of the RabbitMQ virtual host which the topic queues are created on
username
,password
Authentication data
For the configuration example above, the following objects will be created:
kubectl -n openstack-external get secret
NAME TYPE DATA AGE
openstack-external-consumer-A-notifications Opaque 4 4m51s
openstack-external-consumer-2-notifications Opaque 4 4m51s