Output to multiple files

Homepage Clovertech Forums Read Only Archives Cloverleaf Cloverleaf Output to multiple files

  • Creator
    Topic
  • #54429
    JP Trosclair
    Participant

    Hi folks, I’m looking for a way to output messages to individual files, 1 message per file, to the specified directory. I’ve looked at the fileset protocol but it doesn’t seem to be what I need. Was hoping someone could offer some advice.

    Thanks

Viewing 10 reply threads
  • Author
    Replies
    • #81430
      Jim Kosloskey
      Participant

      You will need a Tcl proc to do that.

      I have one – if you want it email me.

      email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

    • #81431
      JP Trosclair
      Participant

      Thanks Jim. I’ve already got one for filtering the messages, I’ll work in there. Was hoping there was another option but that works.

      Thanks again

    • #81432
      Jim Kosloskey
      Participant

      I thought you wanted to place one message per file using fileset protocol.

      That is the proc I was offering not a filter proc.

      email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

    • #81433
      JP Trosclair
      Participant

      That’s indeed what I’m trying to do. So it’s obvious now that I’m new to this; where in the fileset configuration would you place such a proc? The outbound section for fileset-local doesn’t seem to have a place.

      All the same, I appreciate the offer for the pre-written proc but I’m more interested in learning the ropes.

    • #81434
      Russ Ross
      Participant

      I use a TCL proc called tps_fileset_outbound to write one message per file.

      Here is a URL with Source code and screen shots to illustrate:

      Russ Ross
      RussRoss318@gmail.com

    • #81435
      Jim Kosloskey
      Participant

      It would go in the Outbound TPS.

      email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

    • #81436
      JP Trosclair
      Participant

      Thanks guys, this gives me a lot to work with. I appreciate the information and time.

    • #81437
      JP Trosclair
      Participant

      This is good, but there’s a glaring race condition here if you’ve got a 3rd party system picking these messages up. Any of you guys run into it? Any solution? I’m looking at the temp file option in the fileset properties dialog now. The docs specifically say the temp file is renamed and not copied so I think this would solve the problem so long as it’s not writing the temp file in the same output directory.

    • #81438
      Terry Kellum
      Participant

      The way that I have handled this in the past is to have the third party pick up files like *.rdy .  Cloverleaf puts in files to the dir without the extension.  I then have a script that looks at the dir every minute and renames files that are over three minutes old.

    • #81439
      Charlie Bursell
      Participant

      A bettor way IMHO

      Write the files out to a temp directory.  Then have another thread in the same process with UPoC protocol.  Schedule that thread to run as often as you want to move files.

      When the UPoC thread runs, move (don’t copy) the files to the final directory.  Move is an atomic operation so no way the fles can be picked up early. Also, since the UPoC thread runs in the same process there is no way the engine can be writing files at the same time you perform the move.

    • #81440
      Russ Ross
      Participant

      I posted how we’ve been batching up real-time numfiles for FTP on a scheduled basis on the second page of this URL:

          <a href="http://clovertech.infor.com/viewtopic.php?t=1907&postdays=0&postorder=asc&highlight=batchnumfile&start=15&#8243; class=”bbcode_url”>http://clovertech.infor.com/viewtopic.php?t=1907&postdays=0&postorder=asc&highlight=batchnumfile&start=15

      The basic concept at a high level is that we have a real-time interface create a num file of each ADT message real-time so that each num file has one ADT message in it.

      Since the num files are sequntially named, when the directory listing is sorted they are also in chronological order.

      A proc called batch_num_file.tcl that is argument driven is run on the desired time interval via a cron entry to batch up the currently available num files into a concatenated batch file, which is then available for the destination system to FTP/get or have Cloverleaf FTP/put.

      In our case, I prefer to have the source system FTP/get the file which happens once an hour for this interface.

      Granted much of the solution uses standalone scripts to complement cloverleaf because at that time in the year 2000 I was not fully capable in cloverleaf.

      It has wokred well so no motivation to change it plus it hasn’t ever been impacted by a cloverleaf upgrade.

      Russ Ross
      RussRoss318@gmail.com

Viewing 10 reply threads
  • The forum ‘Cloverleaf’ is closed to new topics and replies.

Forum Statistics

Registered Users
5,117
Forums
28
Topics
9,293
Replies
34,435
Topic Tags
286
Empty Topic Tags
10