Running out of semephores

This happen to my test environment.  I brought down all processes and sites then rebooted our HP/UX Unix Server, then brought up all sites and processes.  Try rebooting your HP/UX Unix Server.

Since I have a huge number of sites, threads and message flow, I was concerned about understanding semaphores a bit better.

Fortunately, I’ve never had a single semaphore issue during my 8+ years here.

I searched old postings related to your semaphore problem as a start.

Here is one that might be of interest:

https://usspvlclovertch2.infor.com/viewtopic.php?t=115&highlight=semaphore” class=”bbcode_url”>https://usspvlclovertch2.infor.com/viewtopic.php?t=115&highlight=semaphore

One of the old posts states:

The problem surfaces when someone starts the process with a user name other then hci.  This is because the semaphore and the shared memory segment cannot be accessed.  When you see this, you can do an ipcs -a and you will see the semaphore and shared memory segment information.

Since we do everything as hci this may have kept us out of semaphore trouble.

Also, a couple of old posts I located indicated this problem is more likely to occur on a Windows platform.

I’m on AIX and have not located where the limit of semaphores is set but did see a reference to increasing the number of files in the /etc/security/limits file – mine is set to 2048 at this time.

Here is a quote window showing my /etc/security/limits settings for the hci users (2 of them are for QDX support):

hci:

       fsize = 2097151

       core = 2048

       cpu = -1

       data = 786432

       rss = 196608

       stack = 196608

       nofiles = 2048

hcitest:

       fsize = 2097151

       core = 2048

       cpu = -1

       data = 786432

       rss = 196608

       stack = 196608

       nofiles = 2048

hcispt1:

       fsize = 2097151

       core = 2048

       cpu = -1

       data = 786432

       rss = 196608

       stack = 196608

       nofiles = 2048

A couple of semaphore related commads whose output I’ve yet to figure out are:

ipcs -a

sar -m 5 3

I also found posts suggesting to keep process and thread names short:

We’ve had this alot, and I think it is because of long thread names. Threads shouldn’t be more than 15 characters, and processes 9. But after you change them, you need to delete the /stats directory, which is under /exec – probably why that fix works, but it just works temporarily.

Do you have long thread or process names?

We have exceeded these limits alot but do not have semaphore problems.

Other posts also talked about the use of

hcimsiutil -R

hcimsiutil -rs

Put it all together and I’m not really comfortable with my current understanding of semaphores

My gut tells me that being I have such a high probablity of a semaphore error due to huge number of threads and sites and message flow, I suspect either running on AIX or just using the hci user ID has contributed to our stable success.

Since security is pushing us to seperate logons, I will pay attention to see if any semaphore problems arise at that juncture.

Does HP/UX use the /etc/system settings? On Solaris, you have to define the amount of available semaphores in that file. Look for stuff like this:

* Cloverleaf Interface Engine Requirements

set semsys:seminfo_semmap=2050

set semsys:seminfo_semaem=16384

set semsys:seminfo_semmnu=2048

set pt_cnt = 1024

set semsys:seminfo_semmsl=8192

John:

Here is what I see using

ipcs -s

IPC status from /dev/mem as of Wed Dec 27 13:30:42 CST 2006
T [code]IPC status from /dev/mem as of Wed Dec 27 13:30:42 CST 2006
T

Does HP/UX use the /etc/system settings? On Solaris, you have to define the amount of available semaphores in that file. Look for stuff like this:

* Cloverleaf Interface Engine Requirements

set semsys:seminfo_semmap=2050

set semsys:seminfo_semaem=16384

set semsys:seminfo_semmnu=2048

set pt_cnt = 1024

set semsys:seminfo_semmsl=8192

Anyone can help to verify this applies for Solaris 10 systems?

For Solaris 10 and Cloverleaf 5.6 you just need the following (according to the installation instructions):

set semsys:seminfo_semmni = 512

I’m guessing the rest is self tuning.

Thanks David.

I’ve hit the exact same error when starting the 2nd test site monitor daemon (1st site monitor daemon started without any errors)

I switch the site and try, yet the same error occurs.

Nothing else was done but just adding the extra setting as listed by David after the 1st testing was done that shows the error. Rebooted Solaris and got the same thing.

What else I’ve miss out??  🙁