Homepage › Clovertech Forums › Cloverleaf › Fileset Local and the timer
Tagged: Fileset Local and the timer
- This topic has 14 replies, 4 voices, and was last updated 4 years ago by
Charlie Bursell.
-
CreatorTopic
-
November 11, 2019 at 5:21 pm #113621
Collin Praster
ParticipantHello I am using the Fileset Local timer and I have set the minutes but it doesn’t seem to stop when the time is listed but it starts when the time is listed. Am I doing something incorrectly or is it functioning as designed?
Attachments:
You must be logged in to view attached files. -
CreatorTopic
-
AuthorReplies
-
-
November 11, 2019 at 5:31 pm #113623
Collin Praster
ParticipantSorry little clarification, it starts when the times are listed but I assumed it would stop processing when the time is NOT listed.
-
November 11, 2019 at 6:13 pm #113624
Jim Kosloskey
ParticipantBy stop do you mean the thread stops or it stops scanning for files?
Do you have the scan interval or other timing configurations set?
-
This reply was modified 4 years ago by
Jim Kosloskey.
email: jim.kosloskey@jim-kosloskey.com
-
This reply was modified 4 years ago by
-
November 11, 2019 at 6:19 pm #113626
Collin Praster
ParticipantI would like it to stop scanning for files when the times are not listed, scan internal using the fileset local protocol properties and then scheduling.
-
November 11, 2019 at 6:30 pm #113627
Jim Kosloskey
ParticipantSorry I have not used the Advanced Scheduling so I am not sure how the Scan Interval and Advanced Scheduling work together.
Perhaps others can chime in who have experience with the scheduler.
email: jim.kosloskey@jim-kosloskey.com
-
November 12, 2019 at 5:38 am #113628
Robert Kersemakers
ParticipantYou are correct: the times listed in advanced scheduling is when Cloverleaf is going to read/process the files that can be seen at that time.
Two things to keep in mind:
– seconds is *, so every second Cloverleaf will check whether there is something to process.– all files that are found will be processed! Say you have ‘Read Interval’ = 1 and ‘Max. Messages’ = 1 so that every second 1 message/file (depending on Inbound Style) is processed. If at time xx:35:59 100 messages are found, then these messages will be processed all throughout xx:35:59 until xx:37:39. So Cloverleaf will not scan the directory if the minutes are not in Advanced Scheduling, but will not stop processing messages it has found earlier.
Does this help?
-
This reply was modified 4 years ago by
Robert Kersemakers.
Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands
-
This reply was modified 4 years ago by
-
November 12, 2019 at 11:02 am #113661
Collin Praster
ParticipantCan you look at my attached image? when it gets to the correct minute it starts correctly but lets say there are 100 files and it starts at minute 46, it will continue on until it clears the folder instead of stopping at 46 minutes and 59 seconds or minute 47. So the way I read your response it should stop. So maybe my timer is set up incorrectly by me?
-
November 12, 2019 at 11:45 am #113662
Jim Kosloskey
ParticipantOnce the scan of the directory runs and finds 100 files it will process all of those found files. The schedule is no longer involved at that point.
You could write a DirParse UPoC proc I guess to try to control that but why?
Don’t you want all of the files located processed?
email: jim.kosloskey@jim-kosloskey.com
-
November 12, 2019 at 11:48 am #113663
Collin Praster
ParticipantSo I seem to run into an issue I am copying messages from a share into a cloverleaf folder so they can be processes and it errors because I think it is grabbing a file that is copying and is not finished and causes the fileset local thread to crash. So it grabs the file in process of copying I think?
-
November 12, 2019 at 12:00 pm #113664
Jim Kosloskey
ParticipantWell you certainly don’t want to be reading any files still being written to.
So you could do the COPY then a renames (perhaps to a different folder) with the name Cloverleaf will look for. The renames really runs fast so the potential for collision is reduced significantly.
So let’s say the source file is named filexxx.fil…
Copy filexxx.fil to directory 123, then rename/move filexxx.fil in directory 123 to directory 456.
Configure Cloverleaf to look in directory 456.
If you need to use the same folder then you will need a DirParse UPoC proc (unless you are on a later release of Cloverleaf where there is a built in file name pattern selection filter function I believe) AND rename the filexxx.fil to something like cl_filexxx.fil so that there will be 2 sets of files the original Copied file and the renamed file.
In any case Cloverleaf will only delete the files it picks up – you will need to come up with something to delete the other files. A FileDelete UPoC proc could do that.
Another option is to write a DirParse UPoC proc which checks the time stamp and removes any from the parse list which are current time. They would be picked up at the next parse.
Perhaps you can schedule the copy from the shared FS to Cloverleaf to occur far enough before the time Cloverleaf will parse so that collision potential is diminished.
email: jim.kosloskey@jim-kosloskey.com
-
November 12, 2019 at 12:05 pm #113665
Collin Praster
ParticipantThank you! I will dive further into your suggestion.
Thank you as always for your time!
-
November 13, 2019 at 3:50 am #113680
Robert Kersemakers
ParticipantAttached is a proc to have Cloverleaf check the time of the files in the inbound directory. If the age of the file is less than (default) 5 seconds, the file will not be processed. Time can be configured.
Attachments:
You must be logged in to view attached files.Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands
-
November 18, 2019 at 10:53 am #113791
Collin Praster
ParticipantAwesome thank you Robert!
-
November 19, 2019 at 1:09 am #113808
Charlie Bursell
ParticipantBetter still would be to move the files rather than copy if you can. mv unlike copy is an atomic operation and you will never see until finished.
If archive copies of the messages should be retained on the original server, copy to archive location the mv to Cloverleaf location.
-
November 19, 2019 at 1:11 am #113809
Charlie Bursell
ParticipantShould have mentioned my statement is only true if the source and destination are on the same filesystem. If not simply copy to temporary location the mv to cloverleaf folder.
-
-
AuthorReplies
- You must be logged in to reply to this topic.