Perl over SMAT – The search begins

[Tony Benitz]

I have a problem:

I have an inbound inteface that is receiving messages (results) and is supposed to be routing to the outbound interface to another system.

Back in December, 6 messages went missing (MIA).  When the message was resent, it worked.  All the testing tools showed all was OK.  We could find no reason for why 6 identified messages went AWOL in the engine.

Now we have no faith in the routing of these specific result types in the interface engine.  Our analysts have not had any reports of missing results, but still the faith in the delivery of the message has been broken.

I am looking for a tool (a perl script?) that can read the SMAT .msg file and parse out certain fields based on a condition.

If the OBR.20 has the value CD, pull the MRN, FIN, and DOS

This would be stuffed (written) to another file.

Then run the tool on the outbound SMAT file.

At this point I would have two files that would give me a picture of the inbound (received message) and the outbound (routed messages) for this result type.

I could then run a diff on the two files and if I have a difference between the two files, I got a MIA message and further investigation is needed.

I am new to perl (read REALLY new) so I am looking for something that I can steal (re-use).

Thanks to all who took the time to read this post and more thanks to the people that have the tool that I search for.

Topic

[Richard Hart]

Tony.

I have a Perl script that parses SMAT files with usage below.

If you can code Perl, this can easily be changed.

Usage: /hci/InfoHEALTH/bin/smatList.sh  [-v] [-h] [-p] [-fp] [-a] [-fa] [-m] [-fm] -o [file2] …

Where -h          help!

     -v          verbose mode

     -o    the output file. See Note below

     -p  the Patient Code (MRN). This can be repeated

     -fp   a file containing Patient Codes (MRN), one per line

     -a  the Account Number. This can be repeated

     -fa   a file containing Account Numbers, one per line

     -m  the Medical Staff ID or GP Code (for Master File Updates). This can be repeated

     -fm   a file containing Medical Staff codes, one per line

     file1       is a Cloverleaf SMAT file, compressed or uncompressed

    The output file supplied will have a suffix _xxx.dat, where

    xxx is the file number in the range 1 through 999.

    Each file will contain  a maximum of 1000 records.

[James Cobane]

Tony,

Was the message in the inbound saved message file?

[Tony Benitz]

Richard, sounds like this is what I need, Can you forward me the source file?

Richard Hart wrote:

Tony.

I have a Perl script that parses SMAT files with usage below.

If you can code Perl, this can easily be changed.

Usage: /hci/InfoHEALTH/bin/smatList.sh

[Jim Kosloskey]

Tony,

I would be intersted to find out if these are TCP/IP connections AND if you are using application level acknowledgment.

If you are not using any acknowledgment protocol, then I find no surprise you might be missing messages. A resend could very well work.

If you are using an acknowledgment protocol, then check the SAMT for the acknowledgments and the foreign system to insure acks were properly processed.

Like Jim Cobane said if they are not in the inbound SMAT, then Cloverleaf did not get them.

Jim Kosloskey

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Tony Benitz]

Jim;

They are TCP/IP connections and we are using system level acks.

I can find the inbound message in the SMAT file (associated to the receiving thread), but NOT in the outbound SMAT file.  That is the strange thing.

Even stranger is when I run the route test, all is good, it shows that the message will be delivered as it should be.

At this point I have 6 messages out of over 15 thousand a day that went MIA at the end of December.  We have not had any other reports of missing data.  I am more and more inclined to say that there was a change in the engine code, but yet I can’t find a change and why did only some of the messages go MIA?  Very strange indeed.

Thus the need to externally audit what I get and what I send.

I am currently working on building a TCL proc that will produce a file with key information if one of the suspect messages is received on the inbound thread.  Then on the outbound thread I will again examine the message for the suspect message type and if found, write to another file (MRN,ACCT,DOS).  Run a diff at the end and the day and if NOT good, I can at least find them before my users do and be able to diagnosis real time instead of a month later.

The perl over SMAT is so I can look at the files produced by the engine and not effect the processing within the engine. (reading every message twice)  Only problem there is that I can write TCL way faster then I can perl.

Jim Kosloskey wrote:

Tony,

I would be intersted to find out if these are TCP/IP connections AND if you are using application level acknowledgment.

If you are not using any acknowledgment protocol, then I find no surprise you might be missing messages. A resend could very well work.

If you are using an acknowledgment protocol, then check the SAMT for the acknowledgments and the foreign system to insure acks were properly processed.

Like Jim Cobane said if they are not in the inbound SMAT, then Cloverleaf did not get them.

Jim Kosloskey

[Daniel Lee]

I agree with James.  The engine should never loose a message unless something is going on with ACK’s as Jim mentioned.  But if you’re going to do this you may want to simply just capture the Message Control ID from the MSH-10 and then diff the files.  That should be good enough to tell you if you have a 1:1 match.

[Richard Hart]

Hopefully, the attached is a copy of the Perl script smatList.sh

The ouput is 1000 records per file that, at the time, was the maxmum number of message a thread would accept before a ‘timeout’ error.

Here is an example function that we used within a copy of smatList.sh (for our LAB messages) to produce an index ..

sub GetData {

  local($myMsg) =  @_;

  local(@myMsg) = split(‘^M’, $myMsg);

  local($myRet, @myField, $myMrn, $myAcct, $myAccession, $myName, $myDob, $myGender);

#print STDERR “Msg @myMsgn”;

  $myAccession = ”;

  $gMsg = ”;

  for $mySeg (@myMsg) {

      if (substr($mySeg, 0, 3) eq ‘PID’) {

          @myField = split(/|/, $mySeg, 20);

          $myMrn = $myField[3];

          $myName = $myField[5];

          $myDob = $myField[7];

          $myGender = $myField[8];

          $myAcct = $myField[18];

          if(!defined($myAcct)) {

              $myAcct = ”;

          }

      }

      if (substr($mySeg, 0, 3) eq ‘PV1’) {

          @myField = split(/|/, $mySeg, 21);

          if($myAcct eq ”) {

              $myAcct = $myField[19];

          }

      }

      if (substr($mySeg, 0, 3) eq ‘OBR’) {

          @myField = split(/|/, $mySeg, 9);

          $myDate = $myField[7];

          if(!isLabGen($myAcct) && substr($myDate, 0, Cool lt ‘20050905’) {

              @myField = split(‘^’, $myField[3], 2);

              $myAccession = $myField[0];

              $gMsg = “$myMrn,$myName,$myDob,$myGender,$myAcct,$myAccession,$myDate,”;

#print STDERR “GetData [$myAccession] [$gMsg]n”;

          }

          last;

      }

  }

#print STDERR “Ret $myRetn”;

  return $myAccession;

}

[Author]

Replies