Thread limitations – capacity planning

[Peg Schwartz]

We are planning on adding several more ADT threads in the near future and are wondering if there is a limit to the number of threads or translations to a site?

We have two sites (prod and test); We currently have 65 threads on production site and 65 on test.

Tech info:

Cloverleaf 5.7

Windows Server 2003

Topic

[Robert Kersemakers]

Hi Peg,

We also have 2 sites (prod and dev/test). We currently have 125 threads in the prod site in 15 processes and about 125 translations; dev/test will be a bit smaller.

In my opinion there is no real restriction on the number of threads/processes/translations. The real restriction is whether the hardware/server can keep up.

In the past we have had some troubles when we put too many thread in one process though, so we try to limit the number of threads in a process to about 12 or 15. But maybe this is outdated too in the newer CL versions (we are on CL5.5 currently).

Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands

[Peg Schwartz]

Thanks Robert.  

Is it required to switch to a Linux box once the volume of messages reach a certain amount?

We are currently on Win 2003 R2 w/ CL 5.7 and will eventually upgrade to CL 5.8.  We are trying to determine whether or not to stay with Windows.

[Bob Richardson]

Greetings,

Our standard is no more than 15 outbound threads in a process cluster with no more than one or two inbounds.  No more than 15 processes in a site though our development/test box may have up to 20 processes and 122 threads in a site but (!) volumes are low in our dev/test box.

Keep in mind that the Integrator is billed as a 32 bit application regardless of whether or not you run it in a 64 bit addressable host.  Rumor has it that some modules are only 16 bit addressable.

Ok… with that said you can experiment with the 5.8 sub-threading feature (we have yet to do that) which may gain you performance and throughput but (!) beware of global usage in your configuration if you do sub-thread.

And, the underlying hardware and OS need to be robust if you have complex implementatons and high volume.  Recoverablity and stability are key with the OS.  We run on AIX which is where the Integrator was born and raised for most of its life.  The core is C or C++.  We have had extremely reliable service from our AIX platforms.   And currently running in a virtualized setting.

Hope this helps in your research.

[Luke Anderson]

I used to work at a site that had around 500 threads broken into six or seven sites and it was running on a virtualized Windows platform with Cloverleaf 5.8.  The main problem we ran into was performance issues when a Windows task went to do something where it had to look at a folder with lots of files.  It didn’t so much seem to be a probelm with the size of the files but the number of files in the folder.  Something about how Windows works when it has to index a lot of files in a folder to do something like archiving – copying/deleting files from a folder with lots of files in it can cause it to consume high amounts of CPU resulting in problems with Cloverleaf processes not having enough CPU bandwidth to there job properly.  It even caused a site to freeze or panic sometimes.  We had to end up limiting the number of days we kept files around in our Cloverleaf archive folders in order to keep the file counts down.

[Robert Kersemakers]

Right now we have CL5.5 running on HP-UX ia64 hardware. We also have a 5.7 test site on there, but we never got to migrating to 5.7. Now as servers are end-of-life, we are looking to migrate to a Linux platform with CL6.0.

We have thought about Windows servers (preferred by our technical department) but we are going for Linux anyway. It will be easier to migrate all the HP-UX scripts/commands to Linux and we want a stable system.

Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands

[Author]

Replies