Message parsing utility for tcl?

[John Hervatin]

Hi!

I’ve been away from Cloverleaf development assignments for quite a while, and have now been asked to try a heavy-duty rearranging of lab result messages to help an unsophisticated EMR post results the way we want.

I think there’s a utility that can parse up messages so that a tcl proc can access pieces of the message the same way an Xlate can.  Is there one?  And if so, how can I learn how to use it?

Thanks for your help!

Topic

[Charlie Bursell]

Boy, that is an open-ended question  😀

Remember Xlates has a configuration file to determine the type of messages and how to parse.

What sort of message are you parsing?  If HL7 all would start with some thing like:

keylget ARGS MSGID mh

set msg [msgget $mh]

set fldSep [string index $msg 3]     ;# Field Sep (|)

set subSep [string index $msg 4]    ;3 Sub-field sep (^)

# List of Segments

set SEGMENTS [split $msg r]

# If message type were important

# Set Message Type like: ADT_A01, etc

set MSH [split [lindex $SEGMENTS 0] $fldSep]

set msgType [string map “$subSep _” [lindex $MSH 8]]

From here on down, it depends on what you want to do

[John Hervatin]

Hey Charlie!  I met you in Dallas a couple of years ago, but I’m not that memorable.

Yes, I was talking about HL7.  The technique you showed is exatly how we go fish out segments and fields we need.

What I was wondering was if there is some available proc that takes an HL7 message and builds a tree structure like what becomes available to Xlates (and referenced with indexes like “1(0).0(0).PV1(0).00194(0)” etc.  But something that a tcl proc could use so it wouldn’t have to parse out with split and lindex.

Then I could traverse different branches of the tree in all different ways for the logic I need to apply.

I thought maybe whatever parses the message into the tree structure for Xlate might also be usable outside Xlate.  Something like that could be quite handy.

[Charlie Bursell]

No.  As I said you would first have define the message configuration to Tcl ala HL7 variants.  That would be more effort than simply writing the proc

Remember there can be many variations of HL7 messages

There are a lot of people on this board that have written procs such that if you pass in enough variables it will do everything but cook your lunch.  😀

I am not a proponent of these “one proc do all” methods, but to each his own.  😛

[Jim Kosloskey]

John,

I am not sure why you don’t want to use Xlate but there are also the GRM functions which you can reference from Tcl.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Scott Folley]

Yeah, you can use GRM and it will do basically what you are looking for but it is a huge pain.  A lot of work for little or no apparent gain.

[Charlie Bursell]

Agree Scott.

With just a little knowledge you can knock out a Tcl script to filter what you need in just a few minutes.

Seems we all want to make the big bucks as a “Computer Scientist”  However we come along with a simple, easy to lean, public domain, *SCRIPTING* language and we have problems with it.

What is wrong with this picture?  😛

On my soap box again, sorry!  🙂

[Chris Williams]

My $ 0.02 worth:

I have to agree with Charlie. Tcl is so easy to learn and easy to use, I don’t understand why there is so much resistance to it. Is it fear of the unknown?

Many times on CloverTech we’ve heard “I have so much work to do, I don’t have time to learn it.” How many postings do we see here that are the result of not understanding Tcl syntax? Spend a few hours learning it, and save your self weeks of struggling without it.

I’m a programmer. I realize that not everyone here has a programming background, but your interface work will be so much easier if you spend a little time learning how to use the tools. Tcl gives you incredible flexibility in manipulating the data you’re working with, and there is certainly nothing wrong with increasing your skill set and your value to your organization.

[Bob Richardson]

Greetings,

I have worked with languages like COBOL and IBM Assembler and the only way to test that stuff was to compile it and run it using JCL.   Ok… I was very resistant to some “up start” hacker language like TCL but once I got into using TCL shells to test “conceptual” pieces of logic with instantaneous feedback, my productivity went up significantly.   My efforts to those in our own shop who whine about how “hard” it is to learn TCL can greatly alleviate their pain by playing with it interactively in a TCL shell

on the engine.  Then “compose” your logic as a program to run in the engine AFTER using the provided hcitpstesting tools.

Sometimes I think some people just want that instant coffee now and not apply some effort to gain knowledge and understanding.

My soap box…

[Robert Gordon]

Everyone has been told they need a tcl/tk class for years.  Everyone believes that it will cost them thousands.  The secret tcl/tk is free, in google type “Tcl Pro”, click on the first link, look for “Free Keys” on the page, install the software and use the help file.  Over a lifetime it will still cost you thousands, but it will be spread out over time.  Take that to your boss the next time you have a review or they try to fire you, tell them “I saved this organization millions and you don’t realize it; I’m one of your best resources because I know the business, and I have done everything intentionally for the benefit for the business.  What have you done that saves this business one cent besides recommending cutting benefits.”

[John Hervatin]

Thank you all for your feedback.  Jim’s mention of GRM functions reminded me of what it was that I had heard that led me to ask the question in the first place.  Seeing that it would be “a huge pain”, I will probably avoid that route.

Don’t know why some took my question to mean I was having trouble with Tcl.  I’m not a pro, but that’s not why I asked.  I just see a lot of our Tcl scripts needing to access a lot of the same HL7 fields in ways that would benefit from having some pre-written modules that allow accessing specified fields and components more directly.  In other languages that I’m more familiar with, I could do something like  obsId = getHL7value(“OBX-3.1″,”2.3”, 1, 1, y)  after having developed the getHL7value function and made it availabe to any other script that wants to use it. However easy it is to get to a field component using split, lindex, etc., it’s still a lot of busy work if you have to reference a great many fields.

So, I will turn to Google and figure out how to develop and use functions in tcl.  😀

[David Barr]

I’m willing to working on an open source HL7 library if I can get some other people to help. I haven’t found any existing libraries that I’m excited about.

We’ve got a nice library where I work, but it isn’t open source.

Charlie and others have found that the “hl7_get_field” library which some people have access to is not of the highest quality.

[Scott Folley]

Actually, what you are looking for is how to create a TCL package and/or library.  The library is a bit more complicated on the front-end but will auto-load if it is in your TCL library path.  The package has to be imported with the package require command.

One interesting caveat with the TCL library, careful that you have it completely developed and working before you put it in the library path with a .tlib extension because the auto-loader will grab it and index it automagically and you will have things failing all over the place before you know it.

If you want to know how the libraries work, you can simply refer to $HCIROOT/tcl/lib/cloverleaf.  The .tlib files are the ones that you create and the .tndx files are what the auto-loader creates when it sees a tlib file.  Libraries are a lot of fun to play with but you will need to understand namespaces and packages first.

If you want a headstart you can look at the hl7msg.tlib file in the aforementioned location and that actually does much of what you are looking for already.  In fact there are quite a number of useful gadgets in the libraries under $HCIROOT/tcl/lib

Have fun!

[John Hervatin]

Sounds like a great lead, Scott.  Thanks!

[Charlie Bursell]

I don’t know how anyone does this sort of work without understanding and using namespaces and packages.

With that said, I don’t understabd the need to build a utility that will, at most, save me one or two lines of code.

What do you want from an HL7 library?  It certainly gets convoluted when you write something that takes longer to use than do your self.  By that I mean those “LEGO” routines that will cook your breakfast for you if you pass enough arguments  😀

If all you need is a segment or a field, it takes maybe two statments using lsearch, etc.  It takes me maybe 10 minutes to write most HL7 filters unless they get very complex.  Just use one you have already written.  The part down to where you have extracted the segments and separators will probably not change.

Now you have a simple, easy to maintain procedure instead of the maintenance nightmare I see at so many places.

Everyone has an opinion and like certain parts of your anatomy most of them stink  😀   Mine is no different

[Bob Richardson]

Greeti

ngs,

I agree with Charlie in this case based on my observations at our own shop.  We have inherited several general purpose utilities that do everything except make coffee and unfortunately they have turned out to be maintenance nuisances in several instances.

My personal campaign has been to consolidate the many stacks of “TCL cordwood” where a message has to run several gauntlets to get filtered and/or massaged until it hits the translate or outbound queues.  By consolidating the filters and/or manipulations into ONE TCL procedure where the message is handled ONCE overall throughput and efficiency is achieved and future maintenance/debugging is minimized.  This is not to say that general utilities are bad design if we are only dealing with a single filter requirement.  But as we all know those pesky users keep coming back with more stuff for Cloverleaf to do (and believe me we have rewritten our Lab application on Cloverleaf!!) or problems that allegedly Cloverleaf created (95% of the time not!!).   It is easier to zero in on debugging/maintenance when logic is encapsulated into one (or few) logical locations.

My two cents worth.

Have a great day!

[David Barr]

I don’t think I was suggesting a library that could cook breakfast or coffee.  I think it’s helpful to have a library that can easily retrieve a field by segment and field number and is also able to deal with repeating segments, repeating fields, components and sub-components.  And I’ll repeat my offer to help develop such a library if there are other people who are interested and could participate in developing specifications.

[Scott Folley]

I don’t know that I see the value in it but that doesn’t mean there isn’t any.  It does sound like it might be an entertaining diversion and I am up for something challenging, count me in.

[Scott Folley]

And if it makes coffee, all the better…LOL

[Michael Hertel]

Reposting from a previous thread:

Attached are two procs that we’ve created for doing what you want.

We use them all the time. They cut development time down to ‘next to nothing’.

Charlie probably won’t agree with this method but it works very well for us.

Usage is:

vm_getHL7

and

vm_setHL7

The only caveat is when dealing with the MSH segment. Adjust the field count by -1 due to the field separator counting as a field.

Examples:

set event [vm_getHL7 $msg MSH 0 8 0 1] ;(Looking for MSH-9.1, see caveat)

set MRN [vm_getHL7 $msg PID 0 3] ;(get MRN)

set MRN2 [vm_getHL7 $msg PID 1 3] ;(get MRN from 2nd instance of PID (usually A17 messages))

set OBX5 [vm_getHL7 $msg OBX $count 5] ;(part of a foreach that increments $count)

and

set msg [vm_setHL7 $msg [vm_getHL7 $msg PV1 0 19] PID 0 18] ;(copies value in PV1-19 to PID-18)

set msg [vm_setHL7 $msg {} PV1 0 19] ;(blank out PV1-19)

set msg [vm_setHL7 $msg VMMC MSH 0 3] ;(hard code VMMC in MSH-4, see caveat about MSH)

set msg [vm_setHL7 $msg 03 PV1 0 3 0 2] ;(hard code bed value to 03)

For additional info you can email me at michael.hertel@vmmc.org

[David Barr]

In general, I like the functionality of this library.  The performance probably could be improved.  Maybe you could store the message as lists of fields so that you don’t have to re-split the message each time you call setHL7 or getHL7.

[Michael Hertel]

You’re right. I wrote this about 6 years ago.

I’m sure it could be improved.

At least it’s bullet proof for our needs.

😛

[Jeff Dinsmore]

John,

I’ve just started with Cloverleaf, but I’ve been a Tcl-er for a long time.

In an effort to save time and improve clarity/quality, I’ve written a library like what you’ve described to tear HL7 2.x apart, shuffle it around, and put it back together.

There’s a standard message parser that reads the message into an array, procedures to read/write fields/components/sub-components, move/create/delete/reorder segments, etc.

Here’s a simple example snippet

Code:

set msg [msgget $mh]

# parse message string into array for processing
crmcHL7::parseMsg $msg msgArray

if { [crmcHL7::readSegFieldComponent msgArray MSH 9 1] == “ADT” } {
 
 if { [crmcHL7::supportedTriggerEvent msgArray [list A01 A03 A04]] } {

   # Trim down the message to only those segments we need
   crmcHL7::filterSegments msgArray [list MSH PID PV1 PV2 IN1] keep
   
   # clear out some fields
   crmcHL7::clearSegFields msgArray MSH [list 4-6 8 10 11 12-end]  

   # write a field
   crmcHL7::writeSegField msgArray PID 4 “000000000”
 
   # write a component
   crmcHL7::writeSegField msgArray PID 5 2 “BOB”

   # read a field
   set pid_4 [crmcHL7::readSegField msgArray PID 4]
   
   # reassemble message
   crmcHL7::assembleMsg msgArray outMsg

} else {
   # may want to kill message here
 }
} else {
 # may want to kill message here
}

msgset $mh $outMsg

Is this something like you were thinking?

Jeff Dinsmore
Chesapeake Regional Healthcare

[Author]

Replies