Fileset-Local DRIVERCTL does not enforce append

[Peter Heggie]

We are using a fileset-local protocol on an outbound thread. We are setting the filename in a tcl proc in the TPS Outbound Data upoc using the DRIVERCTL keylist. This works. We are also setting the append argument, but that does not work.

This is the code:

Code:

set dvr_ctl_str “{FILESET {{OBFILE $file_name} {OBDIR $path} {OBAPPEND 1} {OBSTYLE single}}}”
msgmetaset $newmh DRIVERCTL $dvr_ctl_str

However, the behavior is that the file is replaced, not appended to. Is this expected behavior for fileset-local when setting the file attributes using DRIVERCTL?

We have an HIS that is sending out results, actually an assessment, where data elements are placed in OBX segments. The HIS is breaking up the assessment into multiple messages because of size limitations.

The assessment data can be unique identified by three fields in the OBR (assessment name, timestamp and entering-userid) – we use this as the unique key. We are ‘collecting’ all the messages that match this key. For each message, we create a file name by concatenating all three fields of the key. If the file does not exist, we set the file name in DRIVERCTL and let the message go through to the protocol and it writes the file. If the file exists, we strip out all but the OBX segments we want, put these in a new message, kill the old message and set the filename in DRIVERCTL. It is the same file name as the one that exists, and if the append flag worked, the new message containing only OBX segments should append to the existing file. Another pair of threads will read these files, if they are at least 5 minutes old.

But the append portion is not working and it is just replacing the file.

Any suggestions or fixes out there?

Peter

Peter Heggie
PeterHeggie@crouse.org

Topic

[Jim Kosloskey]

Peter,

I do not have any insight as to whether that Key does or does not work – but if you do not set that but have your Fileset/Local configuration set to Append doesn’t that get your file appended to (in other words just adjust the filename)?

email: jim.kosloskey@jim-kosloskey.com 30+ years Cloverleaf, 60 years IT – old fart.

[Peter Heggie]

Thank you – yes, the protocol configuration for the outbound is style = NL and Append is checked.

Peter Heggie
PeterHeggie@crouse.org

[Jim Kosloskey]

So if you don’t try to mess with the OBAPPEND, does it append?

Have you tried a simple (just sending data) FileSet/Local with Append on? Did that work?

email: jim.kosloskey@jim-kosloskey.com 30+ years Cloverleaf, 60 years IT – old fart.

[Peter Heggie]

Sorry for the delay. Yes we just tried simple file operations, with an output file name hardcoded in the protocol configuration and the append flag checked, and sent through multiple messages, and they all appended into the specified file.

Maybe the OBAPPEND keyword in DRIVERCTL is just not respected for fileset-local.

Peter Heggie
PeterHeggie@crouse.org

[Jim Kosloskey]

So let the NetConfig specification take care of the Append, etc. and just changee the file name.

In the meantiime, you could put a notice to Support and see if this is a known issue.

email: jim.kosloskey@jim-kosloskey.com 30+ years Cloverleaf, 60 years IT – old fart.

[Peter Heggie]

yes that was the first thing we did, just creating a DRIVERCTL keyed list  with only the filename argument, with APPEND checked in the protocol configuration. When that did not work, we specifed OBAPPEND in the DRIVERCTL. I will open a ticket.

Thanks your your help.

Peter Heggie
PeterHeggie@crouse.org

[Russ Ross]

Peter worte

yes that was the first thing we did, just creating a DRIVERCTL keyed list [code]yes that was the first thing we did, just creating a DRIVERCTL keyed list

Russ Ross
RussRoss318@gmail.com

[Russ Ross]

Another thought that acutally tripped me up while doing my test, is make sure the TCL proc you are testing with is the one being invoked possibly by doing a debug echo.

In my case there was a site specific version of the TCL proc that kept superceeding the master site version of the proc I thought I was using.

A more hidden way to not be running the TCL proc you think you are is by having 2 TCL procs in your site that may have different file names but have the same proc name statement and the last one wins when doing mktclindex.

So look in $HCISITEDIR/tclprocs/tclIndex to make sure the proc name points to the right file name.

Russ Ross
RussRoss318@gmail.com

[Peter Heggie]

We were in a hurry and used file write functions in tcl to perform a ‘brute force’ append operation and put that production today.

I will take some time to restage this in our test environment and check the configuration as you suggested. We don’t use master sites (we have HACMP) but we could have missed a process recycle (I do that all the time).

Also, I did not try all combinations of newline vs carriage return, and that might have caused a problem. We use an FTP free-ware editor (PSPad) for our AIX environment and maybe that was not showing content after the first CR.

Thank you for putting in the effort to test this.

Peter Heggie
PeterHeggie@crouse.org

[Peter Heggie]

and now, the rest of the story..

your successful efforts motivated me to try again. I rewrote some of the code and it worked. Thank you for taking the time to check out all those scenarios.

So this is a good way to combine the contents of several messages, that have some kind of matching criteria. In this case it was assessment data coming out of our HIS, which had the same assessment name, datestamp and userid on the OBR segments of all related messages.

But we also found that when we get a flood of these, Cloverleaf is too fast to use just that technique alone. Sending the first message through untouched, but with the DRIVERCTL set, and then sending subsequent messages along with just the required content (OBXs) and also setting DRIVERCTL did not work when all of these messages processed through the first thread before any of them could process through the outbound thread – checking for file existence failed because the outbound thread did not yet have a chance to create the file before the second message checked for its existence back in the first thread. We had to create the file in the inbound thread, for the first message with this code:

Code:

close [open $filename w]

Also, we had to set the outbound fileset-local protocol on the second interface, the inbound thread, to style=single and also set the conversion to CRNL to NL.

Peter

Peter Heggie
PeterHeggie@crouse.org

[Russ Ross]

What method did you use to determine all the appedning of OBX segments was done for a givien append file, which I assume you then pick up and process.

I have often wodered how I might approach holding messages to be combined and I see how this approach could be usefull for doing that.

Russ Ross
RussRoss318@gmail.com

[Peter Heggie]

Again, this was ‘brute force’. For these assessment messages, we know we are getting all of them pretty much within a few seconds. Its basically one assessment that contains too much data for the interfacing program in the HIS to process at once, so it breaks it up into chunks and sends each chunk out as a message. But it will do so one after the other, with no appreciable delay. There may be other patient assessments going on at the same time so we could get messages interleaved, but still, we will see all the messages for one assessment coming out very close together.

So to wind up a long winded answer, we just wait 5 minutes to be sure all the related messages for one assessment have come out of the HIS and been processed. So we use a directory parse TCL to check the file time and make sure it is at least 5 minutes old.

Peter Heggie
PeterHeggie@crouse.org

[Author]

Replies