Transactions Per Second Alert

Clovertech Forums Read Only Archives Cloverleaf Cloverleaf Transactions Per Second Alert

  • Creator
    Topic
  • #53687
    Bob Richardson
    Participant

      Greetings,

      Running 5.8.6 on AIX 6.1 TL 7 Virtualized.

      Image attached.

      Looking for how to compute the alert value for a thread that can get prolific (Epic Systems ADTs) and flood the Integrator creating havoc for downstream applications.

      We had a developer configure a “Transactions Per Second” alert to email the support group when the number of messages exceeded 4 a second over a minute time period.

      The online help gives a formula but to me at least not enough explanation on how to crunch the magic number for the alert threshold.

      Looking for your experience on how to figure out the values for this alert type.  The people getting the emails want us to up the threshold.

      Thank you.

    Viewing 6 reply threads
    • Author
      Replies
      • #78542
        Russ Ross
        Participant

          I haven’t setup this type of an alert either but taking a guess at what I see in your screen shots that you say are working, I would try adjusting the “compare >= 12”.

          My guess is that 60/12=5 messages per second is when your alert gets triggered the way it is right now, which is greater then 4 messages per second.

          So if that is true, then setting “compare >= 10” might cause the alert to be triggered at 60/10=6 messages per second average over a 1 minute period.

          Since we are going to Epic ourselves it is good to hear they can do better than our minimum requirement of 3 messages per second.

          Our fastest interfaces have hit as high as 150 messages per second and half that is not uncommon on a steady basis during peak hours.

          I can’t imagine having to deal with a situation where 4 messages per second is too fast because that is where I start to think of it as barely being fast enough.

          I’m curious about what action you take when you get a cloverleaf alert that messages from Epic are being generated at a rate of more than your desired threshold of 4 messages per second?

          In other words, how do you respond to an alert like this?

          Russ Ross
          RussRoss318@gmail.com

        • #78543
          Bob Richardson
          Participant

            Russ,

            Epic Systems will generate an A08 just about anytime a user looks at the application – it is prolific.  Also when batches are released you can get flooded with traffic.

            We notify the Epic Systems support folks using an email notice to them with a copy to us in Cloverleaf support.  We also have the Global Monitor product which our Operations staff watch so we stack a notify on top of

            the email action (you can add multiple actions to an alert).  The notify displays an alert in the Alert pane of the Global Monitor and they open up

            the thread notes to get instructions on what to do next.

            I do have a recent response from INFOR support on how to interpret

            the online documentation for adjusting this type of alert and will share that

            to this Forum once I get some time today to do fun stuff like that!

            Let me know if this helps you guys out.  Thanks.

          • #78544
            Russ Ross
            Participant

              Sorry to hear Epic sends a batch of message via real-time interface.

              As we know real-time is just that and is challenged when sent a large burst of messages.

              I have that probem with pharmacy system (Centricity) that sends a huge batch burst of messages around 5 AM each morning and it takes 20 minutes for downstream Pyxis system to catch up.

              Knowing this we just adjusted our alerts to be less reesponsive during this known window, but it does put stress on the integration during that time window.

              I’m also sad to hear that looking at a patient in EPIC wihtout making any change creates A08.

              We have that problem with our current HIS system and is a tremendous souce of overconsuming cloverleaf unecessarily.

              Do the A08’s generated by EPIC at least populate all the fields for every A08 which could let us do some filtering?

              With our currnet HIS it only popluates A08 fields that might of changed and most of the fields are empty, preventing us from doing any filtering.

              I will at least know to talk to the EPIC folks when we implement to ask if nothing changes don’t send an A08 and cross my fingers they can accomodate.

              Russ Ross
              RussRoss318@gmail.com

            • #78545
              Bob Richardson
              Participant

                Russ,

                There are profile entries that can be made to suppress A08s but

                you will need the Epic trained staff (your employees) who work at your facility to identify conditions that can be filtered to eliminate a lot of the traffic.

                We have staff here certified on the product and have created TCL based filters to get rid of some of that traffic.

                The application specifics are beyond my skill level so we rely on these

                Epic staff to specify what we can kill in the engine.

                Some of the batch stuff we were able to route out a different connection

                which helps as some downstream applications complained about

                siginificant latency in getting current messages to do their work.

                Definitely ask Epic Systems about actions that can reduce the traffic

                and about batch releases.

              • #78546
                Bob Richardson
                Participant

                  One and All,

                  The final puzzle piece was found regarding configuring a transactions per second alert.  The online documentation speaks about transactions per specified threads but to me at least that was not clear.

                  What I needed to know was that we are looking at translations per specified threads.  That is how many translations is the engine doing at the time.  

                  So to compute the alert value, that is, transactions per second to monitor:

                  number of messages per second translated X number of outbound threads

                  (destination threads if you will).  For example, we want no more than 4 messages per second and have one inbound and 5 outbounds.

                  Alert value:  4 X 5 = 20 for the alert value.

                  It is unclear at this time how (if any) impact setting multi-threading has on the computation of the alert value.  I have a question to INFOR support on that question pending.  

                  Hope this makes sense – see my earlier attachment for an example alert.

                • #78547
                  Jim Kosloskey
                  Participant

                    Bob,

                    Thanks for the clarification.

                    email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

                  • #78548
                    Robert Milfajt
                    Participant

                      Russ Ross wrote:

                      Do the A08’s generated by EPIC at least populate all the fields for every A08 which could let us do some filtering?

                      Not sure if this is controlled by the interface (profile variables) or a standard part of the interface, but the EVN segment has two pieces of data that may help in filtering.

                      In our implementation, EVN-3 contains the trigger that generated the message.  The batches, may have their own trigger ID for you to filter on, or the ability to set this trigger ID.  EVN-5 contains the user who triggered the event.  If your batch is run by a designated user, i.e., EMP record #1, EVN-7 might look like 1^EPIC^USER, again something you can filter on.

                      I would like to point out that any inbound ADT message also triggers outbound message(s), so that may be another source you can filter out.

                      Hope this helps,

                      Robert Milfajt
                      Northwestern Medicine
                      Chicago, IL

                  Viewing 6 reply threads
                  • The forum ‘Cloverleaf’ is closed to new topics and replies.