process startup questions

[Jeff Dinsmore]

Running CL 5.6 on Linux.

I’m trying to determine why the starting of my interfaces is slow and I’m trying to make sure I understand what goes on when starting a process.

Let’s say I have a process p2. This process has 13 threads.

When I start p2, the command thread (hciengine) starts and then, apparently, fires up the individual threads.

The process startup results in 14 hcitcl processes that look like this:

hcitcl /quovadx/qdx5.6/integrator/bin/hcixlatenotify xrel_post

The logs of p2 – and all other processes – receive 14 xrel_post commands like this:

[cmd :cmd :INFO/0:       p2_cmd:04/28/2011 10:31:07] Received command: ‘p2_xlate xrel_post’

I’m assuming one for each thread in p2? (and one for extra measure?)

Perhaps I’m not understanding correctly, but it would seem that ONE of these for each process would suffice, wouldn’t it?

I’ve also stepped through the hcixlatenotify code and it seems by far that the largest time consumer is the netcfgLoad command – nearly 40 seconds for my test netConfig – about 24 seconds for live.

I have 25 processes and 121 threads in my test site – 19 and 101 respectively for prod. The engines don’t have any problem keeping up with the message load, starting the processes is very slow.

So… have any of you attempted to rewrite hcixlatenotify? I’m certain it can be made MUCH faster.

Looking for a little sage advice.

Jeff Dinsmore
Chesapeake Regional Healthcare

Topic

[Jeff Dinsmore]

After a little experimentation:

1) Replacing netcfgLoad in hcixlatenotify can definitely make it much faster. Down from 40 seconds to almost nothing on my test site.

2) Unfortunately, it makes no difference in thread startup time since, it would seem, hcixlatenotify runs in the background after threads are started.

3) The number of threads in a given process has little, if any, difference in total process startup time.

Therefore, while an interesting exercise, I’m still stuck with the problem of long process start times – apparently a function of the hciengine executable – and, presumably, total number of processes and/or threads in the site.

I am assuming that the smaller the number of processes/threads per site, the quicker processes will stop/start? So, my expectation is that breaking my single site into multiple smaller sites would have a positive effect.

What are other factors that might affect process start/stop time?

Jeff Dinsmore
Chesapeake Regional Healthcare

[Jim Kosloskey]

Jeff,

If you have messages in the Recovery/Error DBs this will have an effect.

email: jim.kosloskey@jim-kosloskey.com 30+ years Cloverleaf, 61 years IT – old fart.

[Jeff Dinsmore]

Not many messages in recovery or error databases, but they are relatively large – should probably run hcidbinit to trim file sizes.

Jeff Dinsmore
Chesapeake Regional Healthcare

[Jim Kosloskey]

Jeff,

I don’t think the allocated size matters – just the number of messages at time of startup.

We utilize multi-site here and as long as our DBs are clear our startups are rather rapid.

Besides multi-site we have a limit on the number of processes per site and the number of threads per process.

Those threshold get evaluated from time to time (certainly whenever we change hardware).

We also try to not use cross-process routing.

I personally maintained since 1997 it is not whether a Cloverleaf shop will be multi-site but when.

With 5.6 the Master Site simplifies matters and with 5.8 the cross-site routing adds even more potential simplification as far as multi-site is concerned.

Hopefully in future releases we will see some additional multi-site enhancements which have been and will be requested.

email: jim.kosloskey@jim-kosloskey.com 30+ years Cloverleaf, 61 years IT – old fart.

[Author]

Replies