Fileset Local and the timer

Homepage Clovertech Forums Cloverleaf Fileset Local and the timer

  • Creator
    Topic
  • #113621
    Collin Praster
    Participant

    Hello I am using the Fileset Local timer and I have set the minutes but it doesn’t seem to stop when the time is listed but it starts when the time is listed.  Am I doing something incorrectly or is it functioning as designed?

     

    Attachments:
    You must be logged in to view attached files.
Viewing 13 reply threads
  • Author
    Replies
    • #113623
      Collin Praster
      Participant

      Sorry little clarification, it starts when the times are listed but I assumed it would stop processing when the time is NOT listed.

    • #113624
      Jim Kosloskey
      Participant

      By stop do you mean the thread stops or it stops scanning for files?

      Do you have the scan interval  or other timing configurations set?

      • This reply was modified 4 years, 11 months ago by Jim Kosloskey.

      email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

    • #113626
      Collin Praster
      Participant

      I would like it to stop scanning for files when the times are not listed, scan internal using the fileset local protocol properties  and then scheduling.

    • #113627
      Jim Kosloskey
      Participant

      Sorry I have not used the Advanced Scheduling so I am not sure how the Scan Interval and Advanced Scheduling work together.

      Perhaps others can chime in who have experience with the scheduler.

       

      email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

    • #113628
      Robert Kersemakers
      Participant

      You are correct: the times listed in advanced scheduling is when Cloverleaf is going to read/process the files that can be seen at that time.

      Two things to keep in mind:
      – seconds is *, so every second Cloverleaf will check whether there is something to process.

      – all files that are found will be processed! Say you have ‘Read Interval’ = 1 and ‘Max. Messages’ = 1 so that every second 1 message/file (depending on Inbound Style) is processed. If at time xx:35:59 100 messages are found, then these messages will be processed all throughout xx:35:59 until xx:37:39. So Cloverleaf will not scan the directory if the minutes are not in Advanced Scheduling, but will not stop processing messages it has found earlier.

      Does this help?

      Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands

    • #113661
      Collin Praster
      Participant

      Can you look at my attached image?  when it gets to the correct minute it starts correctly but lets say there are 100 files and it starts at minute 46, it will continue on until it clears the folder  instead of stopping at 46 minutes and 59 seconds or minute 47.  So the way I read your response it should stop.  So maybe my timer is set up incorrectly by me?

    • #113662
      Jim Kosloskey
      Participant

      Once the scan of the directory runs and finds 100 files it will process all of those found files. The schedule is no longer involved at that point.

      You could write a DirParse UPoC proc I guess to try to control that but why?

      Don’t you want all of the files located processed?

      email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

    • #113663
      Collin Praster
      Participant

      So I seem to run into an issue I am copying messages from a share into a cloverleaf folder so they can be processes and it errors because I think it is grabbing a file that is copying and is not finished and causes the fileset local thread to crash. So it grabs the file in process of copying I think?

    • #113664
      Jim Kosloskey
      Participant

      Well you certainly don’t want to be reading any files still being written to.

      So you could do the COPY then a renames (perhaps to a different folder) with the name Cloverleaf will look for.  The renames really runs fast so the potential for collision is reduced significantly.

      So let’s say the source file is named filexxx.fil…

      Copy filexxx.fil to directory 123, then rename/move filexxx.fil in directory 123 to directory 456.

      Configure Cloverleaf to look in directory 456.

      If you need to use the same folder then you will need a DirParse UPoC proc (unless you are on a later release of Cloverleaf where there is a built in file name pattern selection filter function I believe) AND rename the filexxx.fil to something like cl_filexxx.fil so that there will be 2 sets of files the original Copied file and the renamed file.

      In any case Cloverleaf will only delete the files it picks up – you will need to come up with something to delete the other files. A FileDelete UPoC proc could do that.

      Another option is to write a DirParse UPoC proc which checks the time stamp and removes any from the parse list which are current time. They would be picked up at the next parse.

      Perhaps you can schedule the copy from the shared FS to Cloverleaf to occur far enough before the time Cloverleaf will parse so that collision potential is diminished.

       

      email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

    • #113665
      Collin Praster
      Participant

      Thank you!  I will dive further into your suggestion.

      Thank you as always for your time!

    • #113680
      Robert Kersemakers
      Participant

      Attached is a proc to have Cloverleaf check the time of the files in the inbound directory. If the age of the file is less than (default) 5 seconds, the file will not be processed. Time can be configured.

       

      Attachments:
      You must be logged in to view attached files.

      Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands

    • #113791
      Collin Praster
      Participant

      Awesome thank you Robert!

    • #113808
      Charlie Bursell
      Participant

      Better still would be to move the files rather than copy if you can.  mv unlike copy is an atomic operation and you will never see until finished.

      If archive copies of the messages should be retained on the original server, copy to archive location the mv to Cloverleaf location.

    • #113809
      Charlie Bursell
      Participant

      Should have mentioned my statement is only true if the source and destination are on the same filesystem.   If not simply copy to temporary location the mv to cloverleaf folder.

Viewing 13 reply threads
  • You must be logged in to reply to this topic.

Forum Statistics

Registered Users
5,129
Forums
28
Topics
9,301
Replies
34,447
Topic Tags
288
Empty Topic Tags
10