Rotate OpenStack credentials

The credential rotation procedure is designed to minimize the impact on service availability and workload downtime. It depends on the credential type and is based on the following principles:

  • Credentials for OpenStack admin database and messaging are immediately changed during one rotation cycle, without a transition period.

  • Credentials for OpenStack admin identity are rotated with a transition period of one extra rotation cycle. This means that the credentials become invalid after two rotations. MOSK exposes the latest valid credentials to the openstack-external namespace. For details, refer to Access OpenStack through CLI from your local machine.

  • Credentials for OpenStack service users, including those for messaging, identity, and database, undergo a transition period of one extra rotation cycle during rotation.

Note

If immediate inactivation of credentials is required, initiate the rotation procedure twice.

Impact on workloads availability

The restarts of the Networking service may cause workload downtimes. The exact lengths of these downtimes depend on the cloud density and scale.

Impact on APIs availability

Rotating both administrator and service credentials can potentially result in certain API operations failing.

Rotation prerequisites

  • Verify that the current state of the LCM action in OpenstackDeploymentStatus is APPLIED:

    kubectl -n openstack get osdplst -o yaml

    Example of an expected system response:

     1 kind: OpenStackDeploymentStatus
 2 metadata:
 3   name: osh-dev
 4   namespace: openstack
 5 spec: {}
 6 status:
 7   ...
 8   osdpl:
 9     cause: update
10     changes: '((''add'', (''status'',), None, {''watched'': {''ceph'': {''secret'':
11       {''hash'': ''0fc01c5e2593bc6569562b451b28e300517ec670809f72016ff29b8cbaf3e729''}}}}),)'
12     controller_version: 0.5.3.dev12
13     fingerprint: a112a4a7d00c0b5b79e69a2c78c3b50b0caca76a15fe7d79a6ad1305b19ee5ec
14     openstack_version: ussuri
15     state: APPLIED
16     timestamp: "2021-09-08 17:01:45.633143"

  • Verify that there are no other LCM operations running on the OpenStack cluster.

  • Thoroughly plan the maintenance window taking into account the following considerations:

    • All OpenStack control plane services, components of the Networking service (OpenStack Neutron) responsible for the data plane and messaging services are restarted during service credentials rotation.

    • OpenStack database and OpenStack messaging services are restarted during administrator credentials rotation, as well as some of the Openstack control plane services, including the Instance High Availability service (OpenStack Masakari), Dashboard (OpenStack Horizon), and Identity service (OpenStack Keystone).

    For approximate maintenance window duration, refer to Calculate a maintenance window duration for update.

Rotate the credentials

  1. Log in to the osdpl container in the openstack-controller pod:

    kubectl -n osh-system exec -it <openstack-controller-pod> -c osdpl -- bash

  2. Use the osctl utility to trigger credentials rotation:

    osctl credentials rotate --osdpl <osdpl-object-name> --type <credentials-type>

    Where the <credentials-type> value is either admin or service.

    Note

    Mirantis recommends rotating both admin and service credentials simultaneously to decrease the duration of the maintenance window and number of service restarts. You can do this by passing the --type argument twice:

    osctl credentials rotate --osdpl <osdpl-object-name> --type service --type admin

  3. Wait until the OpenStackDeploymentStatus object has state APPLIED and all OpenStack components in the health group in the OpenStackDeploymentStatus custom resource are in the Ready state.

    Alternatively, you can launch the rotation command with the --wait flag.

Now, the latest admin password for your OpenStack environment is available in the openstack-identity-credentials secret in the openstack-external namespace.

See also