Needing to append several messages into one

[Dianna Braden]

Because of a 65K message limitation in the sending system, I am needing to append several smaller messages into one into QDX – these will be identified with a special code in an OBR field – ones which is the start of the append, the appending ones, and the end message which needs to be appending and those which do not need to go through this process. I figure I would route via a Trxid for this and setup a thread to ftp some to local server those which needs to go through this process and ftp back into the engine when receiving the end one so the entire big message can be then processed like all the others.

Question is: Is this the best approach? – or is there better options out there, any examples would be appreciated.

Topic

[Robert Milfajt]

What communication protocol is the vendor using to send you these messages?  I cannot see why the vendor cannot make this work with batch FTP or TCP/IP MLLP protocols.

Can you get them to change their code or are you working within the confines of what they are sending?

Bob

Robert Milfajt
Northwestern Medicine
Chicago, IL

[Dianna Braden]

They are sending HL7 via TCPIP – and yes they are changing their code – but it will be next year before a release like that will come out. So, in the meanwhile I am writing something to handle this iin cloverleaf engine, because they cirrently have the ability to make the messages smaller chunks – but I have to put them back together in the engine.

[Robert Milfajt]

If they are changing there code for you in the future, then make sure they send it all as one message.  MLLP protocol will guarantee Cloverleaf reads it as one message, even if they have to break it up into several socket sends, just as long as they keep the HL7 format of the message intact across the multiple sends.  In other words, Cloverleaf actually does the work you are going to have to do in the interim.

Since you need a solution now, and there is no possibility of change, then your approach seems reasonable.  I would want to make sure that there is some link between the messages, i.e., the field in OBR refers you to the original message somehow, so that you are appending the correct messages together, similar to how DSC segments in Query and Display messages work.

Hope this helps,

Robert Milfajt
Northwestern Medicine
Chicago, IL

[Tom Rioux]

How about having a line in your xlate be a conditional Tcl Call based on the special code in the OBR field.  Your code can check the incoming messages for the special code and determine whether it is a first message, appending message, or end message.  In the tcl code you can format the messages into one message by appending the messages to a file.  If the incoming message from the Tcl call is a first or appending message, then return a value to the xlate that will suppress those messages.  If it is an end message then return the message back to the xlate and continue the message.  

I’m not sure if that can even been done.  If not, you should be able to something like that in an inbound tcl or pre-xlate tcl code.

Just a thought….

Tom Rioux

[Rob Abbott]

Tom is on the right track.  You don’t even need to append the messages in a file.  FTPing a file around would be messy I think.

You can “hold” the messages in Inbound TPS until you get the entire batch.

This has the added advantage of being recoverable.  If the engine goes down it will reprocess the messages that are being held.

Asuume the message will have an “append_flag” in it which you will extract.

Assume the last message in the batch will have an “append_flag” that is false.

Assume the messages that need to be appended will come back-to-back.

Here’s some logic for you:

Code:

global msg_list
set dispList “”

keylget args MODE mode

switch -exact — $mode {
start { set msg_list “” }

run {
keylget args MSGID mh

# Extract append_flag from the OBR
# … extract logic goes here …

# Check the append flag
if $append_flag {
  # Save this message handle in a global list
  lappend msg_list $mh

  # Hold the message (return nothing) until we get the entire batch
  set dispList “”
} else {
  # If we have a msg_list, this is the last message in the batch
  if { “$msg_list” ne “” } {
      # Append the messages into one – create a new msg handle to hold the batch
     set batch_mh [msgcopy $mh]
     msgset $batch_mh “”
     lappend dispList “CONTINUE $batch_mh”

     # Loop over the list and append messages as we go
     foreach msg $msg_list {
        set heldMsgData [msgget $msg]
        # you may need to modify heldMsgData here to get rid of MSH, etc

        # Append the data to our batch msg and kill the held msg
        msgappend $batch_mh $heldMsgData
        lappend dispList “KILL $mh”
     }

     # Clear our global list
     set msg_list “”
  } else {
        # No batch, just continue this message
        lappend dispList “CONTINUE $mh”
  }
}
}

time {}
shutdown {}
}

return $dispList

Rob Abbott
Cloverleaf Emeritus

[Dianna Braden]

Very good suggestions.  I am not confident the messages needed will be back to back, so I have the fileset -local setup and it is building the files as needed and I have them named so they are complete with what is needed per big message.

Last thing I am trying to figure out which is probably easy – is once I see the last message with a code of E – I need to send the big message to the starting thread to process through the xlate as other normal messages do – any suggestions?

[Tom Rioux]

I would think you can still probably do it with Rob’s code, even if the messages aren’t sent back to back.  Is the sending system sending the Control ID in MSH-10?  You may be able to modify Rob’s code to add an additional comparison to see the the Control ID’s are the same.

Just a thought….

Tom Rioux

[Dianna Braden]

Yes MSH-10 is there but not unique – I so have different elements which I use to build the filename – but what happens to this code when this code is work on 2 or 3 different sets of building messages?

[Author]

Replies