suggestions for reducing thread count

Clovertech Forums Cloverleaf suggestions for reducing thread count

  • Creator
    Topic
  • #119059
    Nancy McDaniel
    Participant

      We are getting closer to our license thread count for prod and are looking at additional quite a few threads to support a new application that will go live in Fall 2022.   We are requesting an increase in our licensed thread count but also looking at areas where we can trim unneeded thread connections.

      What are folks doing when they get their inbound ADT (we get from Epic) that goes to multiple threads in that same process that then connect to a listening thread in multiple processes on the ADT site?   example thread is sent to 5 client tcp threads that than have a server tcp thread in different processes on the site.  Every time we want to send data from that inbound ADT thread to different processes, we use 2 threads (client/server) for each processes that we want to send ADT to.

      I have seen this same setup at other places I have worked at.

      Sometime ago I tried playing around with the intra-site routing but that only works if routing to a different site.

      I have tried testing routing directly to the inbound thread  on each process and setting the inbound hread (setup as /dev/null file thread) so that each message routed to it is treated as inbound but have had the sending process crash in the test environment.

      Appreciate any recommendations that others have done.  I understand the need for client and server tcp threads especially for connecting with external systems but when it is used purely for routing to multiple inbound threads on multiple processes, seems like there has to be a better way.

    Viewing 3 reply threads
    • Author
      Replies
      • #119060
        Jim Kosloskey
        Participant

          Perhaps some history is in order here (at least as I recall it).

          In earlier releases Cross Process routing presented issues and many shops (mine included) opted instead to route via TCP/IP Client/Server threads as you have observed. We experienced those issues and it was painful until it was understood what was happening and we started using TCP/IP thread pairs.

          I think even today it is not recommended to use Cross Process Routing in high volume situations. I do not believe I have seen a quantification of the term ‘high volume’.

          It is my understanding the ‘Intra-site’ routing utilizes Cross Process Routing under the covers. Hence I believe that is the reason Intra-site routing is not recommended for high volume situations (again whatever that means).

          Personally I think the use of TCP/IP thread pairs is the best solution available at  least until the Cross Process functionality is improved upon.

          Unfortunately thread limited licenses place institutions at a disadvantage in the modern integration world but budgets rule.

          email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

        • #119061
          Peter Heggie
          Participant

            We have the same configuration. There does not seem to be a way to reduce the number of threads as part of an ADT distribution methodology. Other than cramming all your ancillary threads in a single site, you will need to connect from your “EMR” site to your ancillary site(s), thus the “extra” tcp/ip connection thread pair for each downstream site.

            We took advantage of our EMR replacement project to move from a thread count license to an enterprise license. This was a good move because our thread count has increased by almost 40% over the last five years.

            Peter

            Peter Heggie
            PeterHeggie@crouse.org

          • #119062
            Jeff Dawson
            Participant

              Agree with what has been stated we follow the same configuration setup, one way you may be able to reduce threads is utilizing Multi-Server.  It may require more Trx ID formatting but its helped us reduce threads.  For example we have individual Powerscribe interfaces per hospital that send back to a single Epic Bridges interface dedicated to text based reports.  Instead of having a client/server for each interface pair we have all client threads sending back to a single multi-server thread in our ToEpic site.

              It’s been a long time since I’ve been to any of the QDX training classes, I remember back in the day version CIS 5.* there was a heavy emphasis on avoiding cross process communication.

              Jeff

              • This reply was modified 3 years, 5 months ago by Jeff Dawson.
            • #119066
              David Barr
              Participant

                We use inter-site routing threads, and we also use the other idea that you tried: “I have tried testing routing directly to the inbound thread  on each process and setting the inbound hread (setup as /dev/null file thread) so that each message routed to it is treated as inbound but have had the sending process crash in the test environment.” We do that by running a proc on the outbound TPS to send the message back to the inbound queue. The relevant part of that proc looks like this:

                run {
                # ‘run’ mode always has a MSGID; fetch and process it
                keylget args MSGID mh
                msgmetaset $mh SOURCECONN $HciConnName
                lappend dispList “OVER $mh”
                }

                This doesn’t crash for us, so I’m not sure why your process is crashing.

                • #119069
                  Nancy McDaniel
                  Participant

                    David,

                    This might be something for us to look at.  I have done some updating destination in metadata.  But it looks like you are talking about changing the SOURCECONN to a different thread so that the message will process the routing from another inbound thread.  Definitely something to pursue.  We are on CL 19.1 in prod and testing with CL 20.1.1 on our test cloverleaf server.  When I say we had the main ADT process crash in test, it occurred of times after I had tried to switch to the direct routing.  I had just rolled it back rather than troubleshooting the exact issue.

                    Question – on the original ADT inbound thread are you routing to a stub i.e. file thread that has the OB tcl proc that will re-route it over to a different IB thread on a different process?

                    run {
                    # ‘run’ mode always has a MSGID; fetch and process it
                    keylget args MSGID mh
                    msgmetaset $mh SOURCECONN $HciConnName
                    lappend dispList “OVER $mh”
                    }

                     

                    thanks,

                    Nancy

                  • #119070
                    David Barr
                    Participant

                      Yes, we’re routing to a stub, but that code runs in the stub outbound TPS, so $HciConnName refers to itself. This puts it back on the inbound queue of the stub thread, and you can set up routes from that thread to your final outbound destination threads.

                    • #119071
                      Nancy McDaniel
                      Participant

                        David – one more question, did you have to check the box on the outbound stub that say treat messages routed to this connection as inbound.

                        or was the tcl proc on the outbound tps for the stub enough?

                      • #119072
                        David Barr
                        Participant

                          No, we probably set this up before that was an option.

                    Viewing 3 reply threads
                    • You must be logged in to reply to this topic.