Auditing transactions

I can see the value in this kind of database. I’m forever trawling smat files to answer the question “What happened to my message?”, usually from 3rd party vendors.

I would suggest, rather than collecting the data in real-time as you are proposing, it may be better to build a system which updates the database from smat files after you have archived them. This way you could match up messages for a set of patients together with their respective HL7 Acks. It would also be completely non-invasive.

Dave

Pete,

First of all I am not sure why you wouldn’t just utilize the already available SMAT files (which contain every message sent and received – including acknowledgments if you like) instead of adding overhead to capture the messages (or some portion thereof) again.

However, I think the answer to your question regarding capturing the outbound messages is that you need to make a copy of the message in Tcl and set the disposition of the copied message to OVER. That will place the copied message on the inbound side of the outbound thread and thus look like a message which can then be routed.

Pete,

Use the idx and msg files of the SMAT as indicated (via Tcl for example) and then build your mysql db from that information if you like.

Then the search could be done with the mysql tools you will have (you were going to need them anyway – right?).

Or — you could merge all of your SMAT files into a gigantic SMAT file (I am not recommending this just saying it could be done) and use the SMAT tool to query that huge SMAT file.

Or — read through the SMAT files as indicated and maintain an Oracle or SQL Server or other proprietary DBMS and use the plethora of tools in the market for those.

As for SMAT command line go to the help documents in the Reference Book, Engine commands and you will see hcismat.

Hey Pete,  I’m not sure if this is what you have in mind, but one of my “extracurricular activities” here has been a message archiver for all of our engines (we have both Cloverleaf and Openlink).  We wrote a Java app that receives source messages (untranslated) via tcp/ip (with hl7 ack/nak) and archives them to a mysql database. The app lives on its own server and is 3 different modules, a tcp/ip port listener, an “archiver / indexer” that parses the data and inserts the appropriate fields, and a fancy little viewer utility that will also read cloverleaf Smat files.  We are NOT dealing with engine meta-data because I feel it doesn’t really provide any value. We are pulling the data from the MSH segment and able to provide the appropriate context to retrieve messages that way. I have 1 single thread on cloverleaf that basically forwards feeds from all of my ancillary feeds. Things like ADT, Orders, Results, etc etc are combined into a single feed which is received by the Java app. Our next step is to build a bigger index from the stored data the archiver produced.  So what I’m trying to get out of the indexer is a 3rd party app that basically will display the data for a particular patient, message by message across all sending systems. The patient data is directly linked to an HL7 message which can then be resent if needed.  If you or anyone is interested in specifics I can go further.

Mike

Hey Pete,  No tcl needed in my solution. I’m simply adding another ancillary application to the mix, but in terms of netconfig, i am splitting the receiving thread at the point at which it is received. It might take a little rewiring in another setting, but I am able to take the 1 single point of input and split it off, 1 going through the interface and 1 going to an “archive” site. This site does only 1 thing, receives feeds of hl7 messages and combines them to a single tcp/ip connection.  These are all raw messages.  I can use then query on visit number, ssn or account number, name or whatever field (we even built it so we can easily add fields of interest and reprocess old data) to produce a list of hl7 message and then resend then do whatever I want with them. I’m trying to avoid needing to use cloverleaf since we also use Openlink. I can tell from the MSH segment where the message came from and from that point I’ll know the source engine as well.

Mike

I’ve been toying with something similar for searching across multiple days of SMAT files. It’s a cumbersome task to find messages when you don’t know exactly when they were processed.

The indexing of .idx files is a nice idea. I’d recommend storing most of the info in the .idx files. There’s a datestamp, current and source message ID’s, source/destination connections, and offset/length for pulling messages from the .msg files. Plus you can store select info from the messages – like message type, MRN, encounter, patient name, etc. – anything you may want to search by.

Depending on your message volume, this database could get large quickly.

The other option is to work with the .idx/.msg files directly. A method I used in the past was to start with “now” and work backward through logs – displaying messages that match search criteria as they’re found – so you can cancel when it uncovers what you’re looking for. A date range speeds the search if you have an idea of approximate date/time. Depending on how frequently you need to find messages – and assuming that most of what you’re looking for is fairly recent – this may be an acceptable approach.

Does anyone know if there are any locking concerns with reading the .idx/.msg files the engine is currently using?

It seems like you are saving messages via smat for trouble shooting? I’m not sure thats the intended purpose. Smat is an archive of messages. Sure you can save both inbound and outbound and plainly see where you are missing messages, but thats kind of only part of the picture. My troubleshooting always begins with the question “What happened with Patient XYZ?” Or “What happened to lab result from this patient on this date?” So to deal with those issues, I’ve always used smat as a way to reproduce the questioned message (or messages), a source message that hasn’t been translated or filtered, and then use Cloverleaf’s testing tool to see whats happening.  I never save outbound messages because its redundant if you have the source. You can always resend the source message or run it through the route tester tool or translation tester to see what’s happening with that particular message. Also, as it stands now for me, those smat files become too big to deal with if I don’t cycle save routinely. I chose to do so every hour and then time-date stamp the files and store outside of the /hci filesystem. Also, each individual “interface” has it’s own threads and saves in different Smat files, so I never get to see “the big picture” without lots of work pooling all the messages. The other problem I have is dealing with a patient that spans several hours or even days. Pulling those files make my stomach churn sometimes 😉 So with my little utility, I can basically see all the messages in a 1 stop-shop time lapsed feed so to speak.  Think of it like a patient profile in facebook format for Smat 😉  On a side note, how often are you “losing” messages in the engine? 95% of my job is pointing a finger at one of the ancillary apps and of course vendor complaints 😉

Jeff,

Beginning with Cloverleaf 5.6 on AIX platforms, there is no issue with referencing SMAT files that are active (with or without the SMAT tool).

Prior to 5.6 on AIX (and perhaps other platforms) one had to cycle the SMAT files to use them.