SUN Sol Unix and Cloverleaf V5.6 R2(Thread Startup Slownes)

[Yong Ping]

Dear All,

Would need any assistance on this particular – SUN Solaris Unix and Cloverleaf V5.6 R2.  We are doing some proof of concept right now, migration from Wintel to Sun Solaris Unix.   In term of message throughput performance, it is really impressive however we encountered slowness when starting up.   Approximately it took around 5 – 7 minutes to start up a thread.   Stopping a thread is never a problem.

We’ve consulted both our cloverleaf support locally and SUN vendor but so far no concrete answered given yet.

Appreciate if anyone could share with us the experience and perhaps those fine tunning process.

Topic

[John Mercogliano]

One thing that has been reported to cause a slowdown is using dns names instead of IP addresses.  Resolving the DNS name can slow things.  You might want to have them look at that.  If you replace the dns name with the IP address that would tell you if it’s the problem.  

If you are using dns names, you can improve things by using the local host file to resolve things instead of a dns server.

Hope this helps,

John Mercogliano
Sentara Healthcare
Hampton Roads, VA

[Rob Abbott]

It’s probably DNS as John says.

You might also turn on full EO for your process and do a “tail -f” on the process log after you start the process.  Watch the log, when it pauses, it’ll help narrow down where the issue is.

Rob Abbott
Cloverleaf Emeritus

[Yong Ping]

Dear John and Rob,

Thanks for the information first at all. In fact, we have tried the tail -f on the process log, nothing abnormal and significant noticed.

We did tried out with the IP and DNS name, the differences is around 19 seconds slowness if we used DNS instead. Anyway, we are using iP address.

As part to isolate the problem, we also installed QDX/Cloverleaf locally but same slowness is encountered with TCP/IP protocol threads

We are running out of ideas to resolve or workaround this slowness and we even sought assistance from our partner whom had tested this and he has no ideas the causes else well.

Do you have any other things that we need to take note.  🙁

[David Harrison]

I

[John Mercogliano]

Just for further clarification.  Is this only happening when you first start up the process or if you stop and restart the connection also?

You didn’t mention, so I’m curious if you have set enable all on the connection and/or process before starting it in the netconfig?

Is your disk storage on a detached raid array?  Between the smat files/log files and raima database if there is a hardware problem between the server and raid, that could be a potential culprit.

When you say installed it locally what do you mean?

Does this slowdown on startup happen if you try to connect to a port on the local host?  Not sure if this is what you mean when you talked about installing locally.

It’s been a while since I had my solaris admin hat on, but have your administrators sat with you and run sar and lockstat to monitor what the system is doing while the connection is starting up?  This could point to some type of resource contention on startup that isn’t present once it’s up.  Also, run netstat -i to make sure that the Ierrs, Oerrs, and Collision columns are 0 or near zero.  High numbers in any column would indicate problems with the nic card.

That’s all I can think of for now but hopefully something one of us throws at the wall sticks for you 😯

John Mercogliano
Sentara Healthcare
Hampton Roads, VA

[Yong Ping]

Dear John and David,

Thanks you once again for the promptly replied and sharing on information.

Here is my answer to your question

1. Have you checked you have the latest Solaris patches and Cloverleaf kernel updates?

[YANTO] Yes both are running with latest update for cloverleaf and kernel patches.

2. Just for further clarification.  Is this only happening when you first start up the process or if you stop and restart the connection also?

[YANTO] The slowness is only happening during startup the thread.

3. Is your disk storage on a detached raid array?  Between the smat files/log files and raima database if there is a hardware problem between the server and raid, that could be a potential culprit.

[YANTO] We are using SAN Storage – and it is the similiar SAN storage that we are using now to running our QDX/Cloverleaf on Wintel Servers.  

4. When you say installed it locally what do you mean?

[YANTO] We installed QDX/Cloverleaf on local drive instead – under local storage, we tried to eliminate the possibility or potential SAN storage resource contention problem.  It is proven that it is not relate to SAN storage.

Our team would be walking thru with the hardware vendor again to make sure that we have look thru all the parameter or potential resource contention indicator.

[Author]

Replies