Useful HL7 Scripts

[Max Drown (Infor)]

As requested, here are the scripts I use for working with Cloverleaf smat (.msg) files, HL7 files, and the engine from the command line. I’ll describe how I use them with examples below.

Scripts:

-- Max Drown (Infor)

Topic

[Max Drown (Infor)]

msgExtract Examples

msgExtract bnd01ms4p_in.tin.msg > test.hl7

-- Max Drown (Infor)

[Max Drown (Infor)]

msgParser Examples

Note01: msgParser splits HL7 messages into fields. I use this script more than any other script. It pains me to see people looking at an HL7 message and counting fields with their fingers. Don’t do that anymore!

Note02: You can use this script on smat .msg files, files generated from msgExtract, and you can pipe data to it from all sorts of places. It’s very flexible.

Note03: The script correctly counts MSH fields starting with the first pipe (|) as MSH.01. So, MSH.02 would normally be “^~&”. For us, 99.99% of the time, the pipe is the field delimiter, so I just hard coded it that way. It’s a simple matter to alter the script to work on messages where there is a different delimiter.

Note04: The script left pads 1-digit number with a zero? Why? so that grep can be more accurate and easier to use, and so the fields line up nicely on the screen. I’ll show some examples.

msgExtract bnd01ms4_in.tin.0809062315.msg | grep ‘TEST^’ > test.hl7

-- Max Drown (Infor)

[Max Drown (Infor)]

There you go. I hope some of you will find this helpful. Your feedback is welcome.

-- Max Drown (Infor)

[Amol Kulkarni]

Max,

Thank you so very much for the scripts and the examples. I find your scripts very helpful and quick to work with when the suits want immediate information. New members of the interface group get a useful education with your example on how to manipulate the output thus empowering them.

Again thank you very much.

[Troy Morton]

Hey Max,

These scripts are great!

Have you thought about possibly creating an HL7 2.2 or 2.3 field list table and use it to print the names of the fields along with the values for those people who are new to HL7 and may not know the exact location of the field they are looking for?

Just a thought.

Again, all these scripts are excellent and a must have for any Integration deveopler workign in a UNIX platform.

[Max Drown (Infor)]

I hadn’t, but I’ll add it to my To-do’s.

-- Max Drown (Infor)

[Troy Morton]

Hi Max,

I’m not sure why but I had to modify the following lines in msgParser in order to get each field to print on a different line.  Maybe a difference with our shell or something.  We are using ksh with Perl 5.6 as delivered with QDX 5.4.

           print “nMessage $msgnumn“;

           print “MSH.01: |n“;

.

           print “$name.0$msh_n: $_n” if $_;

.

           print “$name.$msh_n: $_n” if $_;

.

           print “$name.0$n: $_n” if $_;

.

           print “$name.$n: $_n” if $_;

Not sure if anyone else had to make this same change.

Thanks again for the scripts!

[Max Drown (Infor)]

Troy, good catch. The reason you had to add the /n’s was because I forgot to add the -l switch to the header. The -l switch tells perl to always add a newline to the print statements. Sometime this is useful, and sometimes it’s not.

It should be …

Code:

#!/usr/bin/perl -lw

I have fixed that above, too.

-- Max Drown (Infor)

[Charlie Bursell]

What is wrong with using hcihl7test with -n option for field names or even -i option in 5.6+ for filed numbers.

Max:  Are you re-inventing the wheel  😀

[Max Drown (Infor)]

Charlie Bursell wrote:

What is wrong with using hcihl7test with -n option for field names or even -i option in 5.6+ for filed numbers.

Max:

-- Max Drown (Infor)

[Max Drown (Infor)]

Charlie, that’s a handy utility, but it is not exactly what we’re looking for in this context. To get a clean output, you’d need to scrub the results. For example, single digit numbers are not right padded with a zero which makes grepping results trickier. Searching for PV1(0).#1 would match on PV1(0).#11, too.

-- Max Drown (Infor)

[Russ Ross]

msgExtract seems to be tailored made for HL7 messages.

Check out the following 3 scripts that will convert any smat file to newline even if messages aren’t HL7

    ebcdic_smat_to_nl.ksh

    len10_to_nl.pl

    ascii_smat_to_nl.ksh

that I previously posted at the following URL

<a href="https://usspvlclovertch2.infor.com/viewtopic.php?t=1539&&#8221; class=”bbcode_url”>https://usspvlclovertch2.infor.com/viewtopic.php?t=1539&amp;

Russ Ross
RussRoss318@gmail.com

[Max Drown (Infor)]

I made a small change to msgExtract. I noticed that some vendors do not always send a trailing carriage return on the last segment in the message.

Code:

#    s/rMSH/rnMSH/g;
   s/MSH|/nMSH|/g;

-- Max Drown (Infor)

[David Barr]

Max Drown wrote:

I made a small change to msgExtract. I noticed that some vendors do not always send a trailing carriage return on the last segment in the message.

# [code]#

[Max Drown (Infor)]

David Barr wrote:

Max Drown wrote:

I made a small change to msgExtract. I noticed that some vendors do not always send a trailing carriage return on the last segment in the message.

# [code]#

-- Max Drown (Infor)

[Max Drown (Infor)]

I updated the code for msgExtract. I’ve updated the post with the code above.

-- Max Drown (Infor)

[Alice Kazin]

I have been using your msgParser script.  We would like to propose some changes.  Ability to use with different encoding characters (colon instead of pipe).  Ability to show “repeating fields” on separate lines.

[Max Drown (Infor)]

For repeating fields, one quick option is to simply pipe your results to tr: msgParser test.hl7 | tr “~” “n”.

I’ll see what I can do with the field delimiter.

-- Max Drown (Infor)

[herm ernst]

Max,

Would you have a windows versions of your msgParser, msgReader and msgExtract scripts?

Been using them on *nix and they’ve been great!

Now a client is moving to windows 2003server  and it would be great to have these available.

[Max Drown (Infor)]

Attached are versions of the scripts for windows. They aren’t perfect, so feel free to alter them for your needs.

-- Max Drown (Infor)

[Brad Gollifer]

awk ‘{gsub(/MSH/,”nMSH”);print}’  some_file >temp

puts each message on its own line and stores in file temp

then search or something else

awk ‘/some big reg ex here/’ temp

or to find a field

strings temp >temp1

grep PID temp1 |cut -f 6,20 -d “|”

returns patient_name and ssn from each message in the file

[Max Drown (Infor)]

I re-wrote the msgParser and msgReader perl scripts into one tcl script that does mostly the same thing.

You may need to tweak the regsub lines depending on the format of your HL7 files. While I was developing this script, my files were nl (one HL7 message per line).

By default, msgParser will print HL7 fields. For example, “msgParser *.hl7”.

To get it to print segments instead, use the -seg argument like this: “msgParser -seg *.hl7”.

#!/qdx/qdx5.4/integrator/bin/tcl

if {[lindex $argv 0] == “-seg”} {
[code]#!/qdx/qdx5.4/integrator/bin/tcl

if {[lindex $argv 0] == “-seg”} {

-- Max Drown (Infor)

[Max Drown (Infor)]

#### UPDATE 04/08/2011 ####

I added support for reading message from a pipe/stdin.

I re-wrote the msgParser and msgReader perl scripts into one tcl script that does mostly the same thing.

This script makes the assumption that the input will be one message per line (nl).

By default, msgParser will print HL7 fields. For example, “msgParser *.hl7”.

To get it to print segments instead, use the -seg argument like this: “msgParser -seg *.hl7”.

#!/qdx/qdx5.4/integrator/bin/tcl

# Script: msgParser
# Author: Max Drown
# Usage:
# [code]#!/qdx/qdx5.4/integrator/bin/tcl

# Script: msgParser
# Author: Max Drown
# Usage:
#

-- Max Drown (Infor)

[Max Drown (Infor)]

Zac Votrain submitted a msgExtract code update that adds support for Cloverleaf 5.8+ smat files. I have included the code in the first post of this thread.

-- Max Drown (Infor)

[Keith McLeod]

I noticed that when I performed a msgExtract on a series of SMAT files, that the messages came out of the files from how they the files were sorted by name.

[Max Drown (Infor)]

Sorry, Kieth, these scripts provide no sorting options. You will have to use the unix tools and/or shell script to extend the functionality to sort the files.

-- Max Drown (Infor)

[David Barr]

You can use “ls” to do the sorting:

ls -rt thread_name*.msg | xargs msgExtract | grep something > somefile

[Alice Kazin]

I tried using the msgExtract script in Cloverleaf 5.8 for outbound thread and the outbound thread name is still part of the outbound message.   This script works very well for an inbound thread.

[glen goldsmith]

A Table for HL7 2.2/2.3 names are already in the $HCIROOT/formats/hl7/2.3/etc directory……… so getting them is easy as pie.

I have for years, used some sed functions to do the same things for your scripts…………  I’ve had to update it to  perl for 5.8…..

readhl7msg () {

  cat $1 | perl -pe ‘s/{CONNID.*?}MSH/x0dx0aMSH/g; s/x0d/x0dx0a/g’

}

converthl7msg () {

  cat $1 | perl -pe ‘s/{CONNID.*?}MSH/x0dx0aMSH/g; s/{CONNID.*}//g’

}

[Max Drown (Infor)]

I updated msgExtract and msgParser (see the original post containing the code) to dynamically parse fields on the MSH.01 value instead of hard coding to “|”.

-- Max Drown (Infor)

[Max Drown (Infor)]

I added a new msgExtract script to the original post. This new script uses the .idx file to parse out the messages from the .msg file. There is no consistent way to programmatically remove the metadata at the end of each message in the .msg file, and so using the .idx file gives accurate results. I have placed an enhancement request with R&D to change the way that this meta data is added so that it can be removed predictably by externals scripts.

-- Max Drown (Infor)

[David Barr]

Max Drown wrote:

There is no consistent way to programmatically remove the metadata at the end of each message in the .msg file, and so using the .idx file gives accurate results. I have placed an enhancement request with R&D to change the way that this meta data is added so that it can be removed predictably by externals scripts.

I think the .idx files have always been required to figure out where the messages in the .msg file stop and start. With some message formats (like HL7) it was easy to figure out where the messages start and stop before they added the metadata. Is your request that they change the format of the .msg file so that it can be read without a .idx file?

Edit: I’ve added a script that shows how I got around this problem. It looks like you used the OFFSET and LENGTH fields to figure out where the extra metadata is. I explicitly look for all of the metadata fields that can be present in the file. Either method should work. I think my script works for pre and post-5.8 saved message formats.

[Leanne Blank]

I’m a newbie to this so I think this is GREAT!!! I will be trying these out.

Thank you so much!

[herm ernst]

Max,

Love your scripts for SMAT files! Do you have equivalents for SMATDBs?

We are migrating to SMATDBs from SMATs when we install Cloverleaf 6.1.2 and were wondering if there’s similar scripts for use from the command line to process messages stored in SMATDBs?

[Max Drown (Infor)]

I have not written a script for SMAT DB yet. However, do a search of hcismatdb on Clovertech. You can use that script to extract the messages from SMAT DB and then run them through msgParser.

-- Max Drown (Infor)

[Author]

Replies