slow getting out of state 7

Homepage Clovertech Forums Read Only Archives Cloverleaf Cloverleaf slow getting out of state 7

  • Creator
  • #49937
    Kevin Scantlan

    We have a puzzling situation here.  We have a number of threads that if we get an increase in volume of messages, will get a buildup of messages that queueing in the post xlate queue (state 7).  However, others with approximately the same volume do not.  Here’s an example:

    Inbound thread A has 2 destination routes to what I’ll call thread B and thead C, both outbound threads.  They have similar amounts of traffic and similar processing.  But if the volume of messages coming inbound to A increases, you’ll see the post-xlate queue to C increase while it does not to B.  I have placed a script in the post-xlate route and the pre-outbound TPS of C to measure times.  There are times with it’s almost instantaneous, but there are times when it increases as much as 30 seconds.  Is this an ICL problem?  But if it were one would think it would be an across the site problem, which it is not.  

    Any experience out there with something similar or any suggestions?  Maybe someone from Healthvision can shed some light on how the ICL works.

Viewing 8 reply threads
  • Author
    • #64171
      Jim Kosloskey


      Are all of the threads in the same process?

      Jim Kosloskey


    • #64172
      Michael Hertel

      If I were a betting man, I would say this is related to your previous post.

      I am running 5.4.1 (special rev before rev1) on AIX ML6.

      Here’s something from a previous post of mine you can look at.

      Also, have you used topas to identify the offening process?

      Try ulimit -a and see what your settings are. Here are mine.

      hci rs6kg% ulimit -a

      time(seconds)        unlimited

      file(blocks)         2097151

      data(kbytes)         1572864

      stack(kbytes)        262144

      memory(kbytes)       262144

      coredump(blocks)     2097151

      nofiles(descriptors) 10000

      Here’s the info on maxuser processes from my Unix administrator:

      root@rs6kg:/root>lsattr -El sys0 -a maxuproc

      maxuproc 2048 Maximum number of PROCESSES allowed per user True

      Hopefully your admin person can check this for you.

      Here is info from our /etc/security/limits file:


            fsize = 2097151

            core = 2097151

            cpu = -1

            data = 3145728

            rss = 524288

            stack = 524288

            nofiles = 10000

      Lastly, check to see what mode you are running in.

      64 bit or 32 bit

      The box, the unix o/s and Cloverleaf need to be running in the same mode. I don’t know if 5.4.1 supports 32 bit mode anymore.

      Hope this helps…

    • #64173
      Michael Hertel

      Another question, is FTP or SNA involved?

    • #64174
      Kevin Scantlan


      No, the threads are in different processes.  In fact, both the threads A, B, and C are each in processes by themselves.


      Neither FTP nor SNA are involved.  I will check out the settings you gave me.

    • #64175
      Kevin Scantlan

      Here’s what our sys admin had to say:

      Kevin,  All of our settings are the same or higher.  The machine is running the 64 bit kernel.

      its-pluto# ulimit -a

      time(seconds) unlimited

      file(blocks) unlimited

      data(kbytes) 1572864

      stack(kbytes) 262144

      memory(kbytes) 393216

      coredump(blocks) 2097151

      nofiles(descriptors) 10240

      its-pluto# lsattr -El sys0 -a maxuproc

      maxuproc 2048 Maximum number of PROCESSES allowed per user True

    • #64176
      Michael Hertel

      Did you have the same issue putting all the threads in one process?

    • #64177
      Kevin Scantlan

      We did not attempt to place all the threads in the same process.  Thread A and Thread B are in different processes, but have no problem when going from A to B.  Yet we have a problem with going from Thread A  to Thread C, which are also in different processes.  We did not want to overload one proces with it having both the inbound and the outbound threads.

    • #64178
      Michael Hertel

      Try putting them in one process.

      My understanding is that there is additional overhead for going cross process. Cloverleaf works in a round robin fashion and is single threaded where it can only work on one thing at a time. Unless you have throttling involved, it will try to finish the first task before moving to the second. The since thread A delivers to thread B first and then to thread C, it may go A/B, A/C, A/B, A/B, A/B, A/C, etc. Remember, this is my opinon and theory.

      Further, my opinion is that by putting them all in one process, the routing task will be smoother and more even since there is no cross processing involved.

      Try it and see what happens. If it doesn’t work, put them back in separate processes.

    • #64179
      Jim Kosloskey


      I agree with Michael.

      We avoid cross process here.

      Sometimes for other reasons you need to have the engaged threads in different processes.

      You can still avoid cross process in the same site by sending the messages from the inbound thread to an intermediate client thread which is tcp/ip or pdl tcp localhost. in the other process you would have a server thread same protocol as the intermediate client, same host (localhost) same port.

      Now you have sent messages cross process without using the internal Cloverleaf(R) cross process handling.

      Depending on the situation, this may be an indication it is time to embrace a multi-site architecture.

      Of course on one of your other posts you indicated some system consumption issues. These could be related and it is also possible you might see some of the system consumption relieved after bypassing cross process. Then again, the system resource issue might simply be exacerbating the cross process situation.

      Any way we have never regretted trying to eliminate cross process message delivery.

      Jim Kosloskey


Viewing 8 reply threads
  • The forum ‘Cloverleaf’ is closed to new topics and replies.

Forum Statistics

Registered Users
Topic Tags