Fileset Local and the timer

[Collin Praster]

Hello I am using the Fileset Local timer and I have set the minutes but it doesn’t seem to stop when the time is listed but it starts when the time is listed.  Am I doing something incorrectly or is it functioning as designed?

 

Attachments:

You must be logged in to view attached files.

Topic

[Collin Praster]

Sorry little clarification, it starts when the times are listed but I assumed it would stop processing when the time is NOT listed.

[Jim Kosloskey]

By stop do you mean the thread stops or it stops scanning for files?

Do you have the scan interval  or other timing configurations set?

• This reply was modified 5 years, 1 month ago by Jim Kosloskey.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Collin Praster]

I would like it to stop scanning for files when the times are not listed, scan internal using the fileset local protocol properties  and then scheduling.

[Jim Kosloskey]

Sorry I have not used the Advanced Scheduling so I am not sure how the Scan Interval and Advanced Scheduling work together.

Perhaps others can chime in who have experience with the scheduler.

 

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Robert Kersemakers]

You are correct: the times listed in advanced scheduling is when Cloverleaf is going to read/process the files that can be seen at that time.

Two things to keep in mind:
– seconds is *, so every second Cloverleaf will check whether there is something to process.

– all files that are found will be processed! Say you have ‘Read Interval’ = 1 and ‘Max. Messages’ = 1 so that every second 1 message/file (depending on Inbound Style) is processed. If at time xx:35:59 100 messages are found, then these messages will be processed all throughout xx:35:59 until xx:37:39. So Cloverleaf will not scan the directory if the minutes are not in Advanced Scheduling, but will not stop processing messages it has found earlier.

Does this help?

• This reply was modified 5 years, 1 month ago by Robert Kersemakers.

Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands

[Collin Praster]

Can you look at my attached image?  when it gets to the correct minute it starts correctly but lets say there are 100 files and it starts at minute 46, it will continue on until it clears the folder  instead of stopping at 46 minutes and 59 seconds or minute 47.  So the way I read your response it should stop.  So maybe my timer is set up incorrectly by me?

[Jim Kosloskey]

Once the scan of the directory runs and finds 100 files it will process all of those found files. The schedule is no longer involved at that point.

You could write a DirParse UPoC proc I guess to try to control that but why?

Don’t you want all of the files located processed?

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Collin Praster]

So I seem to run into an issue I am copying messages from a share into a cloverleaf folder so they can be processes and it errors because I think it is grabbing a file that is copying and is not finished and causes the fileset local thread to crash. So it grabs the file in process of copying I think?

[Jim Kosloskey]

Well you certainly don’t want to be reading any files still being written to.

So you could do the COPY then a renames (perhaps to a different folder) with the name Cloverleaf will look for.  The renames really runs fast so the potential for collision is reduced significantly.

So let’s say the source file is named filexxx.fil…

Copy filexxx.fil to directory 123, then rename/move filexxx.fil in directory 123 to directory 456.

Configure Cloverleaf to look in directory 456.

If you need to use the same folder then you will need a DirParse UPoC proc (unless you are on a later release of Cloverleaf where there is a built in file name pattern selection filter function I believe) AND rename the filexxx.fil to something like cl_filexxx.fil so that there will be 2 sets of files the original Copied file and the renamed file.

In any case Cloverleaf will only delete the files it picks up – you will need to come up with something to delete the other files. A FileDelete UPoC proc could do that.

Another option is to write a DirParse UPoC proc which checks the time stamp and removes any from the parse list which are current time. They would be picked up at the next parse.

Perhaps you can schedule the copy from the shared FS to Cloverleaf to occur far enough before the time Cloverleaf will parse so that collision potential is diminished.

 

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Collin Praster]

Thank you!  I will dive further into your suggestion.

Thank you as always for your time!

[Robert Kersemakers]

Attached is a proc to have Cloverleaf check the time of the files in the inbound directory. If the age of the file is less than (default) 5 seconds, the file will not be processed. Time can be configured.

 

Attachments:

You must be logged in to view attached files.

Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands

[Collin Praster]

Awesome thank you Robert!

[Charlie Bursell]

Better still would be to move the files rather than copy if you can.  mv unlike copy is an atomic operation and you will never see until finished.

If archive copies of the messages should be retained on the original server, copy to archive location the mv to Cloverleaf location.

[Charlie Bursell]

Should have mentioned my statement is only true if the source and destination are on the same filesystem.   If not simply copy to temporary location the mv to cloverleaf folder.

[Author]

Replies