Reply To: to SMAT or not to SMAT…

The good news is that I believe your hardware is adequate to handle your load because your are practicaly describing what we are running on with a similar number of threads with about a 13 million a day message flow.

If the SAN which is probably out of your control is the cause of the disk I/O bottle neck then a new server using the same SAN group might not be a significant improvement.

This was a serious concern I had and ran into when we were first forced to move from local serial disk to SAN.

I spent the next 3 years suffering while the SAN group finally got it runniong fast enough and reliably.

We have a very similar setup of the hardware, OS, cloverleaf, thread count.

We do have more site granualaity but have SMAT on just about everything and keep up nicely with about a 13 milliong message flow per day with a comfortable amount of horse power to spare.

My previous platform did force me to get very creative to stay afloat and I quickly discovered disk I/O to be the single biggest bottleneck which typically is always true of any platform or application.

Since I’ve witnessed first hand the capacity of your hardware is likely adequate, I like Jim suspect your home grown tracing might be suspect and would take a look at it to see if it is opening and closing the file with every message.

You are welcome to call me if you want to leverage my experience since I’ve already made the journey you are talking about embarking on.

One quick thought that pays big returns towards reducing wasted use of resources, is to filter unecessary message as far upstream as possible.

In fact, we don’t allow the use of static raw route and require a route for each message type to help in this effort.

Also, make sure EO config logging is not set to all except during times of troubleshooting.

Okay enough or I will get going and find it hard to stop with so many things that all add up.

I’m curious with such similar hardware what is the most queue dethp of messages you’ve seen without the process panicing?

Our old server tanked at about 8,000 queue depth but on our current server we hit over 150,000 and kept working fine, which is a much nicer cushion, but leaves me wondering what my upper limit might be.