identifying duplicate messages

[kiran tummala]

We are running into issues with one of the downstream systems.  The downstream system is sending the same message again, sometimes in succession or some times after some time.  How can we configure Cloverleaf to use the Message control id, to determine, if the message is already processed and make it not process the 2nd iteration of the message.

Please help.

Topic

[David Barr]

If the message that is repeated is not always the last message received, then it looks like you need to keep track of all previous message IDs and compare the message with those IDs. Usually a SQLite database would be used for this purpose. I assume that you would also want to periodically purge old IDs from the database so that it doesn’t get too big.

[kiran tummala]

Can Cloverleaf store a certain value in memory, like the way java hash table and use it for the lookups.  instead of making a call out to database and maintaining a connection pool for quicker response.  

If we want to go towards the database route, what would be the best way to handle it, using TCL process for database lookup or Xlate.

Please suggest.

David Barr wrote:

If the message that is repeated is not always the last message received, then it looks like you need to keep track of all previous message IDs and compare the message with those IDs. Usually a SQLite database would be used for this purpose. I assume that you would also want to periodically purge old IDs from the database so that it doesn’t get too big.

[bill bearden]

Kiran,

Yes, Cloverleaf/Tcl/tpsproc can store a value in a global variable that is there from message to message. You could store either the last message id or a Tcl list of the last x message ids. We have tps Tcl procs that do both of these things. You can probably use global variables in Tcl fragments in xlates. But I haven’t done (or tested) that so you would be on your own.

A couple of things to consider… While the value in the global variable is persistent between messages, it gets wiped out when you stop and start the process that this tpsproc is running in. A duplicate message might not be killed/blocked if it comes in right after the process has been started. Also, global variables are tricky. I believe the scope of a global variable in a tps proc is the entire process. So, you need some fancy footwork if you want to use the same tps proc more than once in one process.

Here are snippets of our code, some of which I copied in manually, so use with caution. Also, this is just how we have done it. There is very possibly a better way.

Outside of and before the switch statement (so, outside of the standard start/run/time/shutdown/default sections of a tps proc), we put 2 global statements.

Code:

global msgIdList
global dupCtr

In the start section, we initialize the globals.

Code:

set msgIdList {}
set dupCtr 0

In the run section, I am assuming you have the message handle in a variable called mh. I am also assuming the message id is in a variable called msgId. First, search the list to see if your new message id is in it. If it is, we “ERROR” the message so we the interface team can look at it. You don’t have to ERROR it, you could SUPPRESS it. If it isn’t already in the list, you need to add it to the list and possibly delete the oldest element of the list.

set loc [lsearch $msgIdList $msgId]
if {$loc == -1} {
[code]set loc [lsearch $msgIdList $msgId]
if {$loc == -1} {

[Charlie Bursell]

I agree with David’s post above.  As Bill stated, there is always a possibility that the engine could be shut down between duplicates and you would lose your global values (I prefer namespace variables for this purpose).  Remember that anything that can happen probably will happen.   😀

If your sqlite database is setup properly, it should be almost as fast as setting and checking global variables.  I would store the IDs along with a time stamp and periodically flush the database of those values over a certain age.    You could do this with a timer thread that runs at night.

The only other option I can think of would be to store your global list to a file with each run.  Use the write option that will over-write previous.  Then when the thread starts,  if file exists, populate global with contents of the file.

Even with global or namespace values you would need some routine to periodically flush old values.  Perhaps an array with message ID and time stamp.

Never easy is it?   😉

[Charlie Bursell]

One option I forgot.  A number 10 boot in the vendor’s rear to make him stop doing that!  😆

[David Coffey]

We do have logic in place for is very function.

The current HL7 control id is saved in a global variable.  If the new message control id matches the saved control id the message is KILLed and a counter is updated and the event is logged. If they are not the same the previous is updated with the current and the message is CONTINUEd. If the duplicate counter exceeds the threshold the thread is reset and the counter set back to zero.

Why reset the thread?  We have found that sometimes the two systems get out of sync in the processing of the original message and it’s associated ack.  Be resetting the connection this often will cause the two systems to get back in sync and the duplicate issue is resolved.  

This is not complex.

[Robert Milfajt]

Charlie,

So many vendors, so little time…

Robert Milfajt
Northwestern Medicine
Chicago, IL

[Jeff Dinsmore]

We do this same type of thing to prevent charge duplication.

We morph results from Xcelera into charges for Epic, and if a result is re-sent from Xcelera, we don’t want to trigger a second charge.

We use a SQLite DB to track sent charges in a routine called by Tcl interface code.

Here’s the SQLite DB code:

Code:

namespace eval crmcChargeDb {

variable nsDbName “UNKNOWN”

 variable dbName chargeDb

}

proc crmcChargeDb::dbInit { dbName initErrsName } {

variable nsDbName
 
upvar $initErrsName initErrs
 
set nsDbName $dbName

set retVal 1

set initErrs “”

if { ! [llength [$dbName eval “PRAGMA table_info(sentCharges)”]] } {
crmcSqliteUtils::log $dbName “First access – creating sentCharges table” 0
if { [catch {$dbName eval “create table sentCharges(chargeUid TEXT, sourceSystem TEXT, lastUpdateTclSec INTEGER)”} initErrs] } {
set retVal 0
}
}

return $retVal

}

proc crmcChargeDb::openWait { a } {

variable nsDbName

 set waitMsec 2000
 
 crmcSqliteUtils::log $nsDbName “Attempt $a – Database $nsDbName is locked – waiting $waitMsec milliseconds” 1
 
 after $waitMsec
 
 return 0
 
}

proc crmcChargeDb::insertCharge { chargeUid sourceSystem } {

variable dbName

 set retVal 1

 if { ! [crmcChargeDb::alreadyCharged $chargeUid $sourceSystem] } {
if { [crmcSqliteUtils::openDb 0 $dbName crmcChargeDb::dbInit crmcChargeDb::openWait] } {
if { [catch {$dbName eval “insert into sentCharges (chargeUid,sourceSystem,lastUpdateTclSec) values(’$chargeUid’,’$sourceSystem’,’[clock seconds]’)”} errs] } {
crmcSqliteUtils::log $dbName “Failed to insert chargeUid= $chargeUid, sourceSystem= $sourceSystem” 1
set retVal 0
}
} else {
set retVal 0
}
}

crmcSqliteUtils::closeDb $dbName

 return $retVal

}

proc crmcChargeDb::alreadyCharged { chargeUid sourceSystem } {
 
 global env

 variable dbName

 if { [info exists env(debugChargeDb)] } {
   crmcSqliteUtils::log $dbName “chargeUid= $chargeUid, sourceSystem= $sourceSystem” 0
 }
 
 if { ! [crmcSqliteUtils::openDb 0 $dbName crmcChargeDb::dbInit crmcChargeDb::openWait] } {
   return 0
 }
 
 set chgExists [$dbName eval “select count(chargeUid) from sentCharges where chargeUid = ‘$chargeUid’ AND sourceSystem = ‘$sourceSystem'”]

 if { [info exists env(debugChargeDb)] } {
   crmcSqliteUtils::log $dbName “chargeExists= $chargeExists, chargeUid= $chargeUid, sourceSystem= $sourceSystem” 0
 }
 
 crmcSqliteUtils::closeDb $dbName
 
 return $chgExists
 
}

When a message arrives, we check to see if it’s already been sent (keyed to document UID from Xcelera).  If it has, we kill the message and write a warning that is later emailed.  Othewise, we morph it into a DFT message and send it on to Epic – like this:

Code:

# Check that this Xcelera document number has not been charged before
if { [crmcChargeDb::alreadyCharged $xceleraDocNum XCELERA] } {
set killMsg 1
append warnings “Duplicate charge discarded – (Procedure= [crmcHL7::readSegFieldComponent msgArray OBR 4 3] ([crmcHL7::readSegFieldComponent msgArray OBR 4 1]), MRN= [crmcHL7utils::validMrn msgArray EPIC], Encounter= [crmcHL7utils::validEncounter msgArray EPIC])nn”
} else {

# manipulate message here

# record the charge message in the DB indicating it has been sent
crmcChargeDb::insertCharge $xceleraDocNum XCELERA

}

Jeff Dinsmore
Chesapeake Regional Healthcare

[kiran tummala]

Thanks a lot sir,  I would try the global variable idea first.  Can you please email the process to me at kiran.tummala@hoag.org.  I am very new to the TCL and would take days to complete the process myself.

Thanks a lot.

[kiran tummala]

I tried modifying the script and the variables set in the START process are not showing up in the run mode.  What would be the right way to get that fields.  

If I set the variables in the run, they are available only in that process and are showing that the variable always has 0 elements.

[Author]

Replies