Upgrade OpenStack

This section provides instructions on how to upgrade OpenStack to a major version on a MOSK cluster.

Note

The update of the OpenStack components within the same major OpenStack version is performed seamlessly as part of the MOSK cluster update.

Prerequisites

  1. Verify that your OpenStack cloud is running on the latest MOSK release. See Compatibility Matrix for the release matrix and supported upgrade paths.

  2. Just before the upgrade, back up your OpenStack databases. See the following documentation for details:

  3. Verify that OpenStack is healthy and operational. All OpenStack components in the health group in the OpenStackDeploymentStatus CR should be in the Ready state. See OpenStackDeploymentStatus custom resource for details.

  4. Verify the workability of your OpenStack deployment by running Tempest against the OpenStack cluster as described in Run Tempest tests. Verification of the testing pass rate before upgrading will help you measure your cloud quality before and after upgrade.

  5. Read carefully through the Release Notes of your MOSK version paying attention to the Known issues section and the OpenStack upstream release notes for the target OpenStack version.

  6. Calculate the maintenance window and notify users accordingly.

Migrate Tenant Telemetry from Gnocchi

If the metric service (Gnocchi) is enabled in your cloud, remove metric and add metric-storage in the spec:features:services field of the OpenStackDeployment object in the same change that switches spec:openstack_version from epoxy to gazpacho.

Data impact

The upgrade to Gazpacho removes Redis, the key-value store that Gnocchi used to hold metrics, regardless of whether the metric service is enabled. As a result, all metrics currently stored in Gnocchi are lost. In autoscaling mode, this amounts to the last hour of metrics.

User impact

Users of the OpenStack Alarming (Aodh) service must delete and re-create their alarms against the new metrics and queries.

Users of the OpenStack Heat autoscaling feature must additionally update their Heat templates and Heat stacks:

  • Gnocchi-specific alarm resources no longer work and must be replaced with the OS::Aodh::PrometheusAlarm resource. The list of the affected resources includes:

    • OS::Aodh::GnocchiAggregationByMetricsAlarm

    • OS::Aodh::GnocchiAggregationByResourcesAlarm

    • OS::Aodh::GnocchiResourcesAlarm

    • OS::Aodh::CompositeAlarm

  • Gnocchi-specific alarm options, such as metric, query, threshold, granularity, and aggregation method, must be replaced with an equivalent PromQL query parameter of the OS::Aodh::PrometheusAlarm stack resource that reproduces the intended triggering logic. Refer to Access Tenant Telemetry services on how to access the metric-storage service to find available metric names.

See also

Tenant Telemetry service

Perform the upgrade from Epoxy to Gazpacho

To start the OpenStack upgrade, change the value of the spec:openstack_version parameter in the OpenStackDeployment object from epoxy to gazpacho.

After you change the value of the spec:openstack_version parameter, the OpenStack Controller initializes the upgrade process.

To verify the upgrade status, use:

  • Logs from the osdpl container in the OpenStack Controller rockoon pod.

  • The OpenStackDeploymentStatus object.

    When upgrade starts, the OPENSTACK VERSION field content changes to the target OpenStack version, and STATE displays APPLYING:

    kubectl -n openstack get osdplst

    Example of system output:

    NAME      OPENSTACK VERSION   CONTROLLER VERSION   STATE
osh-dev   gazpacho            1.3.8                APPLYING

    When upgrade finishes, the STATE field should display APPLIED:

    kubectl -n openstack get osdplst

    Example of system output:

    NAME      OPENSTACK VERSION   CONTROLLER VERSION   STATE
osh-dev   gazpacho            1.3.8                APPLIED

The maintenance window for the OpenStack upgrade usually takes from two to four hours, depending on the cloud size.

Verify the upgrade

  1. Verify that OpenStack is healthy and operational. All OpenStack components in the health group in the OpenStackDeploymentStatus CR should be in the Ready state. See OpenStackDeploymentStatus custom resource for details.

  2. Verify the workability of your OpenStack deployment by running Tempest against the OpenStack cluster as described in Run Tempest tests.