- This topic has 18 replies, 5 voices, and was last updated 6 years ago by
.
-
Topic
-
Hi folks, Another for the X-files.
I have Cloverleaf 5.8.7 on Solaris 10 running smoothly since early Dec last year. There’s no firewall (yet) for the primary server that Cloverleaf is currently running on.
[Reason for the “host_server_default_port=34502” entry is for a node in another location with firewall]
Last Thursday, out of the sudden, IDE client access from machines everywhere was extremely slow. Engine and messaging performance were not affected. SSH telnet access is ok.
Besides the GUI, only other noticeable difference is the host server.
– running ‘hciss’ command takes about 30 seconds to return output. Not normal.
– in the hostserver logs, saw this entry:
-
Replies
-
-
Greetings,
I notice that your jvm args is set rather low.
We have the following setting:
jvm_args=-Xmx1024m
You might want to bump up this setting and see if that eliminates the overflow errors for the hostserver. You will have to cycle the hostserver to pickup the new value.
Otherwise, INFOR support can supply you the syntax to enable full debug mode for the hostserver log files to gather more information. You could then post a case at their support website and work the problem with them.
Another quick thought: you may just need to periodically cycle the hostsever (maintenance) to recover memory and clear its registers – so to speak.
Good luck!
-
Thanks Bob. Will keep that in mind, but the fact that things have been running fine since Dec. Weird.
Latest findings. A forum post on memory leak led me to shutdown all the test regions that are also running on that same box. Things seem more normal after that, other than the fact that it was 2 am.
I then did a dbinit on the test regions and brought one of the test regions up. Slowness again! Since a dbinit was done, there’s no messages to invoke any script, so this should rule out memory leak, right?
Still trying.
Thanks folks.
-
David,
Here is another thought for you to investigate.
I would suggest reviewing what user limits you have set on the engine user “hci” – you might be starving it (so to speak). The Release notes
usually have a section on adjusting system parameters like how many files can user hci open, how much memory can it use, and so on.
For us AIX Unix folks (we run AIX 6.1 TL 7) we run >ulimit -a
and get this output:
alin1hub:/healthvision/cis5.8/integrator/allina_prod >ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) unlimited
stack(kbytes) 2097152
memory(kbytes) 2097152
coredump(blocks) 2097151
nofiles(descriptors) 10000
threads(per process) unlimited
processes(per user) unlimited
Just another avenue to investigate.
Hope this helps you.
-
-
Greetings,
I have attached a docx document that we had received from INFOR back in 2012 that discusses site limits for threads, processes, etc.
Note: had to save as a “doc” extension – docx not supported for attachments on this forum.
I am not aware of any later revisions – you may want to check on that with INFOR.
Hope this proves useful.
-
-
-
Hi folks,
I’ve confirmed that 2 scripts have memory leak.
Both test and production sites are on the same server. We’ve made changes to the test copy of the scripts and end-to-end testings are proceeding. Test processes have been started and things seems to be better, but still not as fast as it used to be prior to when things started to slow down.
I am of course not concluding that those 2 scripts are the culprits yet.
Here’s what puzzling me:
1. Those 2 scripts have been around for more than 5 years. Was there any changes in the TCL library etc that may cause some issues in 5.8.7?
2. We upgraded to 5.8.7 since Dec 2013. Per day, 10-20k of messages will activate that script. Which translates to 10-20k of leaked handles per day. Odd that we got hit only after 4 months (assuming this is the root cause)?
-
-
-
Check to make sure none of your interfaces in any cloverleaf site are using ports in the ephemeral range.
On our AIX server that means any port above 32K.
I learned the hard way that using ports above 32K in my case, can result in extreme IDE slowness and sometimes confuses the host server enough that it will automatically launch multiple instances of itself until you reach complete lock up with the IDE.
If you find new interfaces have been assigned port numbers in the ephemeral range at the same time your slowness showed up unexpectedly, then I can relate because I lived thru the same thing and did not come to understand the dynamics very easily either.
Russ Ross
RussRoss318@gmail.com -
-
-
-
-
-
I think we were using cloverleaf 5.2 at that time running under AIX 5.? and what we saw was the following:
– IDE slowness and sometimes complete freeze for all IDEs
– multiple instances of the hostserver would launch on its own
– interface port conflicts, would not connect because port was being used by someting else
Russ Ross
RussRoss318@gmail.com -
-
- The forum ‘Cloverleaf’ is closed to new topics and replies.
