Homepage › Clovertech Forums › Read Only Archives › Cloverleaf › Cloverleaf › problem with deleting a file after picking it up
- This topic has 10 replies, 5 voices, and was last updated 13 years, 10 months ago by
Scott Folley.
-
CreatorTopic
-
August 18, 2010 at 7:06 pm #51940
Joseph Paquette
ParticipantHello, Have a system that will not tolerate a message with an hl7 test time out of sequence. What I have done is setup a proc to write each message to a file. Then I have another thread pick up the files and sort them by the test time, and then send the on there way scheduled to do this once a minute. The writing to a file works perfectly. I have a read thread setup as fileset local, have my directory and a directory parse tps proc that puts the messages back in order. Have searched for info on deletion proc
-
CreatorTopic
-
AuthorReplies
-
-
August 18, 2010 at 7:24 pm #72405
James Cobane
ParticipantJoseph,
The normal processing of an inbound fileset-local thread is to automatically delete the file after it has processed, so I don’t think there is a need to do any additional coding on the inbound thread (the one that is reading the newly written files). Assuming that the files are named sequentially (which it sounds like they are based on the timestamp naming), then the inbound thread should see the file list in the appropriate order and process them; deleting each file as it is processed.
Jim Cobane
Henry Ford Health
-
August 18, 2010 at 7:25 pm #72406
Jim Kosloskey
ParticipantAre you working in two different directories or trying to do all of this in one directory?
email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.
-
August 18, 2010 at 7:28 pm #72407
Joseph Paquette
ParticipantHello,
Same directory.
Joe
-
August 19, 2010 at 1:10 am #72408
Joseph Paquette
ParticipantHello Jim,
This is sending to c:save on my test system, also the read directory is picking up from c:save as well.
Joe
-
August 19, 2010 at 3:12 pm #72409
Joseph Paquette
ParticipantHere are the .err file and the log from the process folder. I am using the advanced scheduler to read every 2 minutes, so have 0,2,4… up to 58 in the minutes, have everything else as a *. What it seems to me is happening is after it processes the first file it doesn
-
August 19, 2010 at 3:35 pm #72410
Jim Kosloskey
ParticipantJoseph,
Is this address path correct:
c:save/L.CVICU_L.CV04_A.201008092300000
It looks to me like you have a mix of DOS and UNIX directory separators.
email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.
-
October 28, 2010 at 5:10 pm #72411
Shelbi Engle
ParticipantJoe,
This sounds like the same or similar issue I am having. Did you ever find a solution?
Thanks!
Shelbi Engle
-
October 28, 2010 at 8:16 pm #72412
Scott Folley
ParticipantWhat we need here is a quick overview of how the inbound directory parse and the inbound delete parse scripts work:
1. The inbound directory parse script gets as its argument the list of files in the directory. Whether you process the file or not (and in what order) is determined by the list that is returned in the disposition. From the directory parse the engine then gets the list and reads each file according to that list and sends the content to the inbound-TPS for processing.
2. The deletion parse gets the list of files that passed the dir-parse. If they remain in the list when the message is dispositioned with a continue out of the deletion parse then the engine goes out to the local or remote system and deletes the file.
The next thing to understand is that the protocol read is handled by the “read interval”, “scan interval”, and “number of messages” parameters. If the time that it takes to read the files in exceeds the scan interval then it is quite possible that the directory scan will be done on files that are about to be deleted. This means that the read will fail because the list is populated with files that are no longer there. It is important to be careful in the tuning of those parameters so that they take into account the lag time in the processing. Those parameters do wonders to optimize the communications but they can be a double edged sword.
Chances are that this is what you are experiencing.
One other thing to keep in mind is the method that is being used to read messages. If you are reading them “newline terminated” then the number of messages read will be based on the number of newlines in the file that you are reading. If you are reading the messages as “single” or “eof” then the number of messages parameter will truly be the number of files that you are going to read during the read interval. I generally prefer to use “single” and split into messages within the inbound-TPS but it is dependent on what your needs are. If you use “newline terminated” then you will also end up with “fileset-” files in your process directory. These files are used by the engine to keep track of where it is in a file that has been read in “newline terminated”.
Hope that helps.
-
October 28, 2010 at 8:30 pm #72413
Joseph Paquette
ParticipantHello,
I did get this working, what we had to do is create a tcl proc for the directory parse that used the glob cmd to grab the directory contents. Once i had the contents in a list, we were able o sort them and send them out. I had a high volume of records coming through so it appeared that the read interval could only pick up a small amount of files. Grabbing the directory made it easier to do. Here is the code i used for the directory parse. Not sure it
-
October 29, 2010 at 4:52 pm #72414
Scott Folley
ParticipantWhile there is certainly nothing wrong with doing it that way, basically you are “circumventing” what Cloverleaf already does for you. If you set your Directory entry in the Fileset Options Inbound to C:/save_msgs/ and set your style to “single” then your Directory Parse proc is going to get the glob’d list that you are manually doing as its message. At that point you can move right to your lsort in order to get the read to process the messages in the order that you want. A couple of handy things that you might want to try are “file tail”, “file extension”, and “file rootname”. These commands give you:
file tail: returns the filename without the path
file extension: returns the extension of the file
file rootname: returns the filename without the extension
As one additional point of order, you are doing: return “{CONTINUE $mh}” followed by lappend dispList “CONTINUE $mh”
The execution of the proc is going to terminate and return as a direct result of the return command and the lappend is never going to be hit. You can either do the return as you have it or you can do:
lappend dispList “CONTINUE $mh”
return $dispList
I caution you on that mainly to point out that the return command does not set up the return value, it actually does the return and terminates execution of the proc.
These are all just points of order, if what you have done is working for you then that is the important part.
One other potentially helpful aside is that I believe the filename is placed into the driver control metadata once you get to the Inbound-TPS (this did not used to be the case). This is helpful if you need to know the inbound file name while processing the message.
-
-
AuthorReplies
- The forum ‘Cloverleaf’ is closed to new topics and replies.