OpenStack known issues

This section lists the OpenStack known issues with workarounds for the Mirantis OpenStack for Kubernetes release 24.1.

[31186,34132] Pods get stuck during MariaDB operations

Due to the upstream MariaDB issue, during MariaDB operations on a management cluster, Pods may get stuck in continuous restarts with the following example error:

[ERROR] WSREP: Corrupt buffer header: \
addr: 0x7faec6f8e518, \
seqno: 3185219421952815104, \
size: 909455917, \
ctx: 0x557094f65038, \
flags: 11577. store: 49, \
type: 49

Workaround:

  1. Create a backup of the /var/lib/mysql directory on the mariadb-server Pod.

  2. Verify that other replicas are up and ready.

  3. Remove the galera.cache file for the affected mariadb-server Pod.

  4. Remove the affected mariadb-server Pod or wait until it is automatically restarted.

After Kubernetes restarts the Pod, the Pod clones the database in 1-2 minutes and restores the quorum.

[36524] etcd enters a panic state after replacement of the controller node

After provisioning the controller node, the etcd pod initiates before the Kubernetes networking is fully operational. As a result, the pod encounters difficulties resolving DNS and establishing connections with other members, ultimately leading to a panic state for the etcd service.

Workaround:

  1. Delete the PVC related to the replaced controller node:

    kubectl -n openstack delete pvc <PVC-NAME>
    
  2. Delete pods related to the crashing etcd service on the replaced controller node:

    kubectl -n openstack delete pods <ETCD-POD-NAME>
    

[39768] OpenStack controller exporter fails to start

Fixed in 24.1.2

On large (500+ compute nodes) clusters, openstack-controller-exporter may fail to initialize within the default timeout.

As a workaround, define OSCTL_EXPORTER_MAX_POLL_TIMEOUT in the cluster object:

spec:
  providerSpec:
    value:
      helmReleases:
        - name: openstack-operator
          values:
            exporter:
              settings:
                raw:
                  OSCTL_EXPORTER_MAX_POLL_TIMEOUT: 900