Frequent node disconnections with mcc-keepalived forcing new election
After cluster update, some nodes may remain in an unstable Ready state
with mcc-keepalived constantly reelecting the leader and failing to
acquire the VIP address, which produces forcing new election messages in
logs.
To resolve the issue:
Identify the leader node that owns the VIP:
On any control plane node, run the following command:
cat /etc/keepalived/keepalived.confIn the system response, capture the VIP used for the cluster.
Using the VIP, identify the leader node:
ip a| grep <VIP>
If the VIP is not found, run the command on another control plane node until you find the leader.
Connect to the non-leader control plane nodes and change the priority on these nodes in
keepalived.conf:vi /etc/keepalived/keepalived.confFor example, change the priority on each node to 150 and 200 respectively:
vrrp_instance VRRP1 { state MASTER garp_master_delay 15 interface k8s-lcm virtual_router_id 154 priority 100 # Change it on one node to 150 and on the other node to 200 virtual_ipaddress { 10.205.88.181 }Restart the
mcc-keepalivedservice on the control plane nodes where the priority was changed:systemctl restart mcc-keepalived
In 10-15 minutes, verify the logs of the node identified in step 1:
journalctl -u mcc-keepalived -f | grep election
You should no longer see the
forcing new electionmessages, and the flapping node status should be resolved.