Useful HL7 Scripts

Homepage Clovertech Forums Read Only Archives Cloverleaf Cloverleaf Useful HL7 Scripts

  • Creator
    Topic
  • #50373

    As requested, here are the scripts I use for working with Cloverleaf smat (.msg) files, HL7 files, and the engine from the command line. I’ll describe how I use them with examples below.

    Scripts:

    -- Max Drown (Infor)

Viewing 34 reply threads
  • Author
    Replies
    • #65806

      msgExtract Examples

      msgExtract bnd01ms4p_in.tin.msg > test.hl7

      -- Max Drown (Infor)

    • #65807

      msgParser Examples

      Note01: msgParser splits HL7 messages into fields. I use this script more than any other script. It pains me to see people looking at an HL7 message and counting fields with their fingers. Don’t do that anymore!

      Note02: You can use this script on smat .msg files, files generated from msgExtract, and you can pipe data to it from all sorts of places. It’s very flexible.

      Note03: The script correctly counts MSH fields starting with the first pipe (|) as MSH.01. So, MSH.02 would normally be “^~&”. For us, 99.99% of the time, the pipe is the field delimiter, so I just hard coded it that way. It’s a simple matter to alter the script to work on messages where there is a different delimiter.

      Note04: The script left pads 1-digit number with a zero? Why? so that grep can be more accurate and easier to use, and so the fields line up nicely on the screen. I’ll show some examples.

      msgExtract bnd01ms4_in.tin.0809062315.msg | grep ‘TEST^’ > test.hl7

      -- Max Drown (Infor)

    • #65808

      There you go. I hope some of you will find this helpful. Your feedback is welcome.

      -- Max Drown (Infor)

    • #65809
      Amol Kulkarni
      Participant

      Max,

      Thank you so very much for the scripts and the examples. I find your scripts very helpful and quick to work with when the suits want immediate information. New members of the interface group get a useful education with your example on how to manipulate the output thus empowering them.

      Again thank you very much.

    • #65810
      Troy Morton
      Participant

      Hey Max,

      These scripts are great!

      Have you thought about possibly creating an HL7 2.2 or 2.3 field list table and use it to print the names of the fields along with the values for those people who are new to HL7 and may not know the exact location of the field they are looking for?

      Just a thought.

      Again, all these scripts are excellent and a must have for any Integration deveopler workign in a UNIX platform.

    • #65811

      I hadn’t, but I’ll add it to my To-do’s.

      -- Max Drown (Infor)

    • #65812
      Troy Morton
      Participant

      Hi Max,

      I’m not sure why but I had to modify the following lines in msgParser in order to get each field to print on a different line.  Maybe a difference with our shell or something.  We are using ksh with Perl 5.6 as delivered with QDX 5.4.

                 print “nMessage $msgnumn“;

                 print “MSH.01: |n“;

      .

                 print “$name.0$msh_n: $_n” if $_;

      .

                 print “$name.$msh_n: $_n” if $_;

      .

                 print “$name.0$n: $_n” if $_;

      .

                 print “$name.$n: $_n” if $_;

      Not sure if anyone else had to make this same change.

      Thanks again for the scripts!

    • #65813

      Troy, good catch. The reason you had to add the /n’s was because I forgot to add the -l switch to the header. The -l switch tells perl to always add a newline to the print statements. Sometime this is useful, and sometimes it’s not.

      It should be …

      Code:

      #!/usr/bin/perl -lw

      I have fixed that above, too.

      -- Max Drown (Infor)

    • #65814
      Charlie Bursell
      Participant

      What is wrong with using hcihl7test with -n option for field names or even -i option in 5.6+ for filed numbers.

      Max:  Are you re-inventing the wheel  😀

    • #65815

      Charlie Bursell wrote:

      What is wrong with using hcihl7test with -n option for field names or even -i option in 5.6+ for filed numbers.

      Max:

      -- Max Drown (Infor)

    • #65816

      Charlie, that’s a handy utility, but it is not exactly what we’re looking for in this context. To get a clean output, you’d need to scrub the results. For example, single digit numbers are not right padded with a zero which makes grepping results trickier. Searching for PV1(0).#1 would match on PV1(0).#11, too.

      -- Max Drown (Infor)

    • #65817
      Russ Ross
      Participant

      msgExtract seems to be tailored made for HL7 messages.

      Check out the following 3 scripts that will convert any smat file to newline even if messages aren’t HL7

          ebcdic_smat_to_nl.ksh

          len10_to_nl.pl

          ascii_smat_to_nl.ksh

      that I previously posted at the following URL

      <a href="http://clovertech.infor.com/viewtopic.php?t=1539&&#8221; class=”bbcode_url”>http://clovertech.infor.com/viewtopic.php?t=1539&amp;

      Russ Ross
      RussRoss318@gmail.com

    • #65818

      I made a small change to msgExtract. I noticed that some vendors do not always send a trailing carriage return on the last segment in the message.

      Code:

      #    s/rMSH/rnMSH/g;
         s/MSH|/nMSH|/g;

      -- Max Drown (Infor)

    • #65819
      David Barr
      Participant

      Max Drown wrote:

      I made a small change to msgExtract. I noticed that some vendors do not always send a trailing carriage return on the last segment in the message.

      # [code]#

    • #65820

      David Barr wrote:

      Max Drown wrote:

      I made a small change to msgExtract. I noticed that some vendors do not always send a trailing carriage return on the last segment in the message.

      # [code]#

      -- Max Drown (Infor)

    • #65821

      I updated the code for msgExtract. I’ve updated the post with the code above.

      -- Max Drown (Infor)

    • #65822
      Alice Kazin
      Participant

      I have been using your msgParser script.  We would like to propose some changes.  Ability to use with different encoding characters (colon instead of pipe).  Ability to show “repeating fields” on separate lines.

    • #65823

      For repeating fields, one quick option is to simply pipe your results to tr: msgParser test.hl7 | tr “~” “n”.

      I’ll see what I can do with the field delimiter.

      -- Max Drown (Infor)

    • #65824
      herm ernst
      Participant

      Max,

      Would you have a windows versions of your msgParser, msgReader and msgExtract scripts?

      Been using them on *nix and they’ve been great!

      Now a client is moving to windows 2003server  and it would be great to have these available.

    • #65825

      Attached are versions of the scripts for windows. They aren’t perfect, so feel free to alter them for your needs.

      -- Max Drown (Infor)

    • #65826
      Brad Gollifer
      Participant

      awk ‘{gsub(/MSH/,”nMSH”);print}’  some_file >temp

      puts each message on its own line and stores in file temp

      then search or something else

      awk ‘/some big reg ex here/’ temp

      or to find a field

      strings temp >temp1

      grep PID temp1 |cut -f 6,20 -d “|”

      returns patient_name and ssn from each message in the file

    • #65827

      I re-wrote the msgParser and msgReader perl scripts into one tcl script that does mostly the same thing.

      You may need to tweak the regsub lines depending on the format of your HL7 files. While I was developing this script, my files were nl (one HL7 message per line).

      By default, msgParser will print HL7 fields. For example, “msgParser *.hl7”.

      To get it to print segments instead, use the -seg argument like this: “msgParser -seg *.hl7”.

      #!/qdx/qdx5.4/integrator/bin/tcl

      if {[lindex $argv 0] == “-seg”} {
      [code]#!/qdx/qdx5.4/integrator/bin/tcl

      if {[lindex $argv 0] == “-seg”} {

      -- Max Drown (Infor)

    • #65828

      #### UPDATE 04/08/2011 ####

      I added support for reading message from a pipe/stdin.

      I re-wrote the msgParser and msgReader perl scripts into one tcl script that does mostly the same thing.

      This script makes the assumption that the input will be one message per line (nl).

      By default, msgParser will print HL7 fields. For example, “msgParser *.hl7”.

      To get it to print segments instead, use the -seg argument like this: “msgParser -seg *.hl7”.

      #!/qdx/qdx5.4/integrator/bin/tcl

      # Script: msgParser
      # Author: Max Drown
      # Usage:
      #
      [code]#!/qdx/qdx5.4/integrator/bin/tcl

      # Script: msgParser
      # Author: Max Drown
      # Usage:
      #

      -- Max Drown (Infor)

    • #65829

      Zac Votrain submitted a msgExtract code update that adds support for Cloverleaf 5.8+ smat files. I have included the code in the first post of this thread.

      -- Max Drown (Infor)

    • #65830
      Keith McLeod
      Participant

      I noticed that when I performed a msgExtract on a series of SMAT files, that the messages came out of the files from how they the files were sorted by name.

    • #65831

      Sorry, Kieth, these scripts provide no sorting options. You will have to use the unix tools and/or shell script to extend the functionality to sort the files.

      -- Max Drown (Infor)

    • #65832
      David Barr
      Participant

      You can use “ls” to do the sorting:

      ls -rt thread_name*.msg | xargs msgExtract | grep something > somefile

    • #65833
      Alice Kazin
      Participant

      I tried using the msgExtract script in Cloverleaf 5.8 for outbound thread and the outbound thread name is still part of the outbound message.   This script works very well for an inbound thread.

    • #65834
      glen goldsmith
      Participant

      A Table for HL7 2.2/2.3 names are already in the $HCIROOT/formats/hl7/2.3/etc directory……… so getting them is easy as pie.

      I have for years, used some sed functions to do the same things for your scripts…………  I’ve had to update it to  perl for 5.8…..

      readhl7msg () {

        cat $1 | perl -pe ‘s/{CONNID.*?}MSH/x0dx0aMSH/g; s/x0d/x0dx0a/g’

      }

      converthl7msg () {

        cat $1 | perl -pe ‘s/{CONNID.*?}MSH/x0dx0aMSH/g; s/{CONNID.*}//g’

      }

    • #65835

      I updated msgExtract and msgParser (see the original post containing the code) to dynamically parse fields on the MSH.01 value instead of hard coding to “|”.

      -- Max Drown (Infor)

    • #65836

      I added a new msgExtract script to the original post. This new script uses the .idx file to parse out the messages from the .msg file. There is no consistent way to programmatically remove the metadata at the end of each message in the .msg file, and so using the .idx file gives accurate results. I have placed an enhancement request with R&D to change the way that this meta data is added so that it can be removed predictably by externals scripts.

      -- Max Drown (Infor)

    • #65837
      David Barr
      Participant

      Max Drown wrote:

      There is no consistent way to programmatically remove the metadata at the end of each message in the .msg file, and so using the .idx file gives accurate results. I have placed an enhancement request with R&D to change the way that this meta data is added so that it can be removed predictably by externals scripts.

      I think the .idx files have always been required to figure out where the messages in the .msg file stop and start. With some message formats (like HL7) it was easy to figure out where the messages start and stop before they added the metadata. Is your request that they change the format of the .msg file so that it can be read without a .idx file?

      Edit: I’ve added a script that shows how I got around this problem. It looks like you used the OFFSET and LENGTH fields to figure out where the extra metadata is. I explicitly look for all of the metadata fields that can be present in the file. Either method should work. I think my script works for pre and post-5.8 saved message formats.

    • #65838
      Leanne Blank
      Participant

      I’m a newbie to this so I think this is GREAT!!! I will be trying these out.

      Thank you so much!

    • #65839
      herm ernst
      Participant

      Max,

      Love your scripts for SMAT files! Do you have equivalents for SMATDBs?

      We are migrating to SMATDBs from SMATs when we install Cloverleaf 6.1.2 and were wondering if there’s similar scripts for use from the command line to process messages stored in SMATDBs?

    • #65840

      I have not written a script for SMAT DB yet. However, do a search of hcismatdb on Clovertech. You can use that script to extract the messages from SMAT DB and then run them through msgParser.

      -- Max Drown (Infor)

Viewing 34 reply threads
  • The forum ‘Cloverleaf’ is closed to new topics and replies.

Forum Statistics

Registered Users
4,966
Forums
28
Topics
9,104
Replies
33,617
Topic Tags
248