Cloverleaf HA on a Redhat Cluster "Fencing" Questi

[Mike Shoemaker]

Hi. I’m running CIS5.8 with HA on a 2 node cluster in VMWare.  We had Goutham come out and set this thing up and he enabled the “fencing” feature for VMWare (specifically VMware Fencing (SOAP Interface)).  Does anyone know if the fencing is a requirement for the cluster or just an added bonus “watch dog”.  I believe I have a situation where the “safety equipment is bringing down the plane”.  Our VM environment appears to become stressed at or around the same time each day.  The cpu use spikes to heart stopping levels for a few moments and then everything calms down. The queues in cloverleaf start backing up, alerts are generated, etc etc but it calms down within 20mins and everything is back to normal. Yesterday, the passive node in the cluster decided to reboot the active node and we failed over. The new active node then had some trouble starting up the sites and we experienced all sorts of database corruption on various sites.  We recovered with minimal problems but now I’d like to stop the fence feature for a little while to at least not have a fail over due to the CPU spikes.  I feel like the active node, due to the cpu failed to check in with the passive node and the fencing kicked in?

Thanks!

Mike

Topic

[Max Drown (Infor)]

I’m no expert with HA, but have been around it a little while installing Cloverleaf and the HA scripts. I would suggest sending Goutham an email with your questions. That being said, it is my un-expert understanding that fencing is highly configurable. For example, we had servers rolling back and forth due to high I/O which was obviously unwanted behavior. The admin was able to adjust the settings to prevent this from happening.

-- Max Drown (Infor)

[Mike Shoemaker]

Thanks Max. I will revisit the details once the dust settles. I just got off the phone with Goutham and he was pretty helpful as usual. For my current situation, I am simply going to remove the standby node from the cluster and just run the single vm with cloverleaf during the network swap tonight. It’s basically just stopping the cman/rgman services on the inactive node. Go figure.

[Author]

Replies