How do I test tps code?

Dennis,

If I understand correctly you are trying to execute tps type Tcl procs in an Xlate.

If that is correct, then you need to know tps procs are by definition not designed to be invoked from inside an Xlate.

Xltp type procs can be executed from inside an Xlate and require a particular construct in order to converse with the Xlate.

As an aside, I will recommend you do message filtering (killing messages) before the Xlate. For maximum flexibility I like to do my filtering in message routing.

Jim Kosloskey

Dennis,

I’ll echo Jim’s suggestion that you kill unwanted messages before the xlate. It’s really quite easy in a simple tcl proc. Much easier than killing during translation. The only downside is your tcl code has to split the message to find the right segment(s) and field(s).

We received HL7 from a large number of client hospitals. Many of them will send spurious message types that we don’t want to process. So a collegue and I created a tcl message filter. Not quite the same thing you need to do but pretty close.

I call the message filter proc in the TPS Inbound Data box on the Inbound tab on the Thread Configuration.

If you want the code, let me know and I’ll post or email it to you.

Steve Robertson

TeamHealth, Inc.

Ah ha! We do something similar here. We have some clients that are groups of hospitals that all use a single HL7 feed to us. But we typically have multiple hospital groups sending us HL7 over different ports on the same Cloverleaf site. Because we use a common thread to write translated message data to a database, we need to assign a hospital identifier to each message. While each hospital in a group has a unique value for each hospital, those values might be the same as another group.

So we have multiple inbound threads, each using overlapping hospital identifiers and a single outbound thread that writes to a database.

So what we do is have a translations (or set of translations) per inbound thread (hospital group). Each inbound thread has a Cloverleaf lookup table. Each translation does a table lookup on it’s table to translate the facility ID assigned by the hospital group to the enterprise-wide ID that we want.

It’s very simple. I suspect you will be able to do the same thing for your provider IDs. You can even assign a default value in a lookup table such that if the input value isn’t found, the output value will be the default. You could assign a default value that indicates that manual intervention is needed.

While you certainly can change message data in a TPS inbound tcl proc, it’s easier to do in the translation, provided all you need to do is a Cloverleaf table lookup. We typically only change inbound (pre-translation) message data in tcl when we need to remove or replace spurious characters that baffle the Cloverleaf xlate thread.

Steve Robertson

TeamHealth, Inc.

If I could insert my $0.02 worth here.  Probably worth less  🙂

When dealing with dynamic lookups like doctor IDs, I do not agree that Cloverleaf Tables are your best bet.  These tables are designed more for static data.   And, I wholeheartly agree with the others that, if possible, you *ALWAYS* KILL oR ERROR messages prior to entering the Xlate engine!

Here is what I do.

I ask that these tables be maintained by a clerk or whoever in an Excel spreadsheet.  Whenever the table changes – which can be often – save a copy in CSV format to a known file name on the Cloverleaf box.

Then, in my Tcl procedure, I read this CSV file into a structure (I prefer an array) upon script startup and save the mtime of the file in a global or namespace variable.  Then for each message, simply stat the file.  If the file has changed, re-read it else using existing table in memory.  I also can add code to error the message if I cannot translate it and send e-mail if required.

FWIW, I have several implementations like that out there and running now.  So far, none of the software has exceeded it’s shelf life  😛

BTW, there is an excellent package for parsing CSV in tcllib.  You can download the latest version of tcllib (tcllib1.9 is the latest) at http://www.tcl.tk/software/tcllib/

Charlie, is there a reason for not using GDBM as the “in memory version” of the table instead of an array?

We used a GBDM based setup at one time that worked quite well.  In it we used a web CGI interface to update the GDBM “database” (using simple HTML tables and forms) and used a little retry loop in the tcl proc to work around the “open from write” collisions when updates were being written and an open for read was executed.  We then let the “users” do the table maintainance via the web page, and as soon as they hit submit the change was there for the next read.

GDBM is now my last choice for non-volitile storage!  It is so finiky to deal with and very limited in what you can do with it.  My preference for non-volitile store now is sqlite and  the Tcl package to access it.  It can be accessed with or without ODBC.  It gives most of the functionality of a full-fledged relational database.  IT IS FREE!!  If you compile the Tcl library, it can run on most any platform.  There are pre-compiled libraries for Windows and Linux.

With that said, if dealing with a lookup table, I prefer, for ergonomic reasons, to have it reside in memory.  This used to be a problem but in the modern day computers I never run into a problem with it.

There are as many methods of tackling any problem as there are people doing it.  Most of them are good.  Just because we do it different does not make one right and one wrong.  But, on the other hand, just because it works does not make it right!

There are only two people I know of that do it correctly, you and me.  And, sometimes I wonder about you!  😆

Charlie,

This (using sqlite and  the Tcl package) sounds like a great topic  to present as a webinar or at the User Group conference.

My $.02 worth.

Jim Cobane

Henry Ford Health

We are designing interfaces to an EMR.  The EMR will be populated from various sources, but the common filter will be whether or not the doc has signed up.

I am planning a small tcl that will pull the doctor numbers from the various HL7 messages and look into a SQLite table to decide whether to kill or continue the message to the EMR.

1) This Tcl will be called from multiple threads, processes and sites.  Will I be affected by locking if I use a single file? The calls will all be reads.

2) The table would be rebuilt daily from a system generated flat file, should I create multiple copies, one per site or thread?  

3) Will I have to shutdown all threads using the script when I update the table?

4) There will literally be tens of thousands of messages checked against this filter daily.  What about the speed and reliability of SQLite in your experience?

I am brainstorming the idea, so suggestions or comments are welcome.  Thanks.

Felix:

In my usage sqlite is pretty fast.  I haven’t used in an environment like you intend to.  My suggestion would be to experiment with it.

I do know if you will be accessing it from several sources it will not be a problem unless some of the sources are writing to the database.  In that case, like any other database, it will lock the record it is updating.  If you attempt to access the record during this time you will get an error that it is busy.

What I do is add logic to my code to look for the busy error and, if seen, wait for a second and try again.  I retry 3 or 4 times and if still unsuccessful, bail out with an error message.

The sorts of problems you will encounter with sqlite are the same sorts of probles you would encounter with any database.

I have since read more of the posts.  

I like the idea of reading the file into memory at startup.  That way I don’t have the overhead of opening an closing the database for each message.  Is it safe?

I also like the idea of using the modification date of the file.  Do you think it could be used to avoid starting and stopping the threads, when updating the database?  

Scenario: I overwrite/update the database file, the mod date changes, all threads would check the mod date of the file notice it is different from the date it has stored in the global when the thread started and reload the table.

I wonder if five threads tried to reload the table at the same time if there would be an issue.

I will have a lot of testing ahead, I appreciate any and all of the suggestions you have made so far.  Thanks again for your help.

That is why I use the mod time so as not to start/stop.  As for several threads trying to update at once, you would have to add code to orchestrate that unless they were all in the same process.  It is similar to the problem of several applications trying to update a file at the same time.  You would have to apply some sort of lock while updating then have all check for the lock.  Perhaps a companion file that could be opened and the flock’ed?

That’s why you get the big bucks  😀   If it were that easy, we would’nt need you  😉