If you update or upgrade your MCP cluster, the kvm
nodes that usually
host the GlusterFS cluster require a reboot for the updates to be applied.
In such cases, you have to verify the GlusterFS cluster status after reboot
of every node before rebooting the next kvm
node.
To verify a GlusterFS cluster status:
Log in to the Salt Master node.
Verify that the GlusterFS server status is healthy:
salt -C 'I@glusterfs:server' cmd.run "gluster peer status"
Example of system response:
kvm01.cookied-cicd-bm-os-contrail40-maas.local:
Number of Peers: 2
Hostname: 10.167.8.243
Uuid: 5ac8ee70-40a3-44c4-8ec8-967a0584a0d4
State: Peer in Cluster (Connected)
Hostname: 10.167.8.242
Uuid: 8ce5fd88-2cae-47b5-a397-447874686406
State: Peer in Cluster (Connected)
kvm02.cookied-cicd-bm-os-contrail40-maas.local:
Number of Peers: 2
Hostname: 10.167.8.243
Uuid: 5ac8ee70-40a3-44c4-8ec8-967a0584a0d4
State: Peer in Cluster (Connected)
Hostname: kvm01.cookied-cicd-bm-os-contrail40-maas.local
Uuid: 6bf7eccb-69a2-42bd-b033-50d683120130
State: Peer in Cluster (Connected)
kvm03.cookied-cicd-bm-os-contrail40-maas.local:
Number of Peers: 2
Hostname: 10.167.8.242
Uuid: 8ce5fd88-2cae-47b5-a397-447874686406
State: Peer in Cluster (Connected)
Hostname: kvm01.cookied-cicd-bm-os-contrail40-maas.local
Uuid: 6bf7eccb-69a2-42bd-b033-50d683120130
State: Peer in Cluster (Connected)
Verify that the GlusterFS client status is healthy:
salt -C 'I@glusterfs:client and not I@glusterfs:server' test.ping
Example of system response:
cid03.cookied-cicd-bm-os-contrail40-maas.local:
True
cid02.cookied-cicd-bm-os-contrail40-maas.local:
True
cid01.cookied-cicd-bm-os-contrail40-maas.local:
True
ctl03.cookied-cicd-bm-os-contrail40-maas.local:
True
prx02.cookied-cicd-bm-os-contrail40-maas.local:
True
prx01.cookied-cicd-bm-os-contrail40-maas.local:
True
ctl01.cookied-cicd-bm-os-contrail40-maas.local:
True
ctl02.cookied-cicd-bm-os-contrail40-maas.local:
True
If any of the above commands fail, refer to the GlusterFS official documentation to troubleshoot the required services.
Verify the GlusterFS volumes status as described in Verify the GlusterFS volumes status.