LARGE VOLUME OF PENDING MESSAGES

Homepage Clovertech Forums Cloverleaf LARGE VOLUME OF PENDING MESSAGES

  • Creator
    Topic
  • #115747
    Stewart
    Participant

    We have a vitals feed coming from 2 different vendors into cloverleaf and then being sent to the EMR.  There continues to be a large amount of pending messages in the OB thread to the EMR.  Is this most likely due to the EMR not being able to process the incoming messages fast enough or is it due to a limit with Cloverleaf?

Viewing 4 reply threads
  • Author
    Replies
    • #115748
      Tipu Razaq
      Participant

      How many messages are coming inbound from the two feeds? Are the inbound/outbound feeds in different processes? We have ~1000 Philips devices here, sending a message every 5 seconds. We filter using a TCL script to only accept every 6th message which may help.

      We’ve ran into instances where a bolus of inbound messages will cause the outbound thread to not send messages out, but that usually clears up once the inbound queue clears. If there are too many messages on the inbound thread, you may just have to delete them from the recovery database of the site that’s hosting the thread so the outbound thread can get going. Cloverleaf seems to dedicate resources where it’s needed and the inbound feed will consume those resources if there are many messages coming through.

      Our Cloverleaf instance is running on Linux if that matters at all.

    • #115749
      Keith McLeod
      Participant

      If you are receiving an ACK back from the EMR, then the EMR is regulating the flow of messages.  This can be caused by how high the debug level is set on your EMR as one possibility.  Generally cloverleaf has no problems keeping up with other interfaces, however, another possibility is the handling of the ACK on cloverleaf side.  I have experienced on McKesson STAR a while back where the ACK message the engine sent back did not match the phrase expected on their side, so they would wait for the timeout before sending the next message.  This gave me a message flow of approx. 1 message every 2 seconds.  Both sides claiming to be configured correctly.  Only this one feed was painfully slow.  Finally got a programmer involved who said I was missing a carriage return at the end of the ACK message for this particular interface. All others didn’t require it.  They in fact needed 3 CR’s and it broke the dam….

    • #115750
      Peter Heggie
      Participant

      We are looking at building a results feed from a vital signs system, and although it is not built yet, we are thinking that we will not to send every result, mostly because our EMR cannot handle such high volume. The EMR is not built to be a continuous feed display/capture tool. All of the results sent into it are stored in the patients clinical record.

      I have not put much thought into it yet but am thinking about building a little piece of code that only passes a subset of readings, perhaps every 5 minutes or more. We need clinical staff buy-in for that methodology, because we don’t know if important outlier data could be discarded if they only presented in-between the interval we use. This could be a show-stopper for the feed.

      As far as your original question, I don’t think the issue is Cloverleaf – it is incredibly fast, even if using the Recovery DB. Perhaps with clinical data, you are not allowed to run without the Recovery DB enabled, but you could ask the question. And since the pending messages are on the OB thread, the likelihood is much greater that the issue is the EMR. I assume the OB thread is tcp/ip, that you are using Auto-Reconnect and not using Close After Write?

      Maybe you could create another interface pair of threads that fits in-between your translation interface and the connection to the EMR, just for the purpose of having another place where the pending messages would build up. It could be in a different process or even a different site, so that a large volume of messages does not impact performance in other processes or sites.

      Peter Heggie

    • #115754
      Vince Angulo
      Participant

      We have Cerner Millennium as our EMR and we experience this type of queuing from time to time.

      Not sure if it’s slow ACKs or the complex server chain that Cerner uses to process inbound traffic.

    • #115759
      David Barr
      Participant

      What is the message state of the pending messages? If they are all in state 7 (for example), that could indicate a Cloverleaf problem, but if they are in state 11, that’s probably either a network issue or a problem with the EMR.

Viewing 4 reply threads
  • You must be logged in to reply to this topic.

Forum Statistics

Registered Users
5,129
Forums
28
Topics
9,301
Replies
34,448
Topic Tags
288
Empty Topic Tags
10