MSI Statistics Lost During Failover

[Bob Richardson]

Greetings,

We have recently upgraded to new hardware – IBM model 8203 with Power 6 processors – to support a failover configuration using two servers:

a primary and a secondary.  We are running CIS 5.6 Rev2 on AIX5.3 TL8 SP3. We developed a procedure that includes unmounting and mounting a SAN pool of disks that contains all of our sites from the root.  We use symbolic links from the root filesystem to each site that also resides on its own filesystem.  Historically, this was done to reduce disk I/O demands and aid in site management in general.

We have observed that once we unmount our SAN from our primary box and then mount this SAN on our secondary box, we lose the message statistics.  We assumed (!) that these statistics would be preserved across mount events.  In doing some research, we know that a pipe file – monitorShmemfile – allows the monitor daemon to communicate with the processes/threads in the site and the “msi” region for the site.   The identification of exactly where this region exists escapes us.  Again, we do a totally manual failover for maintenance purposes mostly but also as an option in the event of a primary hardware failure.  Note that for this failover, we did not perform any engine initializations or memory scrubs.

We have checked the posts in this forum regarding the MSI but have not come up with any answers.

Ok… short question: is it possible to preserve the message statistics during a failover that involves an unmount/mount event?

Thanks in advance for any information on this question.

Topic

[Rob Abbott]

MSI statistics are stored in shared memory.  the monitorShmemFile is simply a pointer to the shared memory.  Since the data are not stored on disk, it will not survive a failover to another physical machine.

Rob Abbott
Cloverleaf Emeritus

[Bob Richardson]

Thank you Rob.

I take it that this shared memory resides at the OS level and not within any Cloverleaf process?  That would explain why they persist even when the engine (site) is down?

Is it possible to get more detailed information on how this works

without compromising any Cloverleaf secrets?

Just interested in understanding the beast that I support.

Again, thanks!

[Rob Abbott]

Hi Bob – Nothing secret about it 🙂

There is a shared memory segment that is reserved for statistics.

Rob Abbott
Cloverleaf Emeritus

[Bob Richardson]

Thank you again, Rob!

It is details like this that shed light in the otherwise dark recesses of my brain when it comes to how the Cloverleaf engine does its work.

Too bad we can’t somehow drain or offload these statistics and restore them via some sort of system utility.

I will share your response with our group – hopefully, others in this forum will have gained new knowledge as well.

BobR

[Rob Abbott]

Quote:

Too bad we can’t somehow drain or offload these statistics and restore them via some sort of system utility.

good idea – submitted as enhancement request #12209 “Request for statistics backup/restore utliity”

Rob Abbott
Cloverleaf Emeritus

[Author]

Replies