Output to multiple files

[JP Trosclair]

Hi folks, I’m looking for a way to output messages to individual files, 1 message per file, to the specified directory. I’ve looked at the fileset protocol but it doesn’t seem to be what I need. Was hoping someone could offer some advice.

Thanks

Topic

[Jim Kosloskey]

You will need a Tcl proc to do that.

I have one – if you want it email me.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[JP Trosclair]

Thanks Jim. I’ve already got one for filtering the messages, I’ll work in there. Was hoping there was another option but that works.

Thanks again

[Jim Kosloskey]

I thought you wanted to place one message per file using fileset protocol.

That is the proc I was offering not a filter proc.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[JP Trosclair]

That’s indeed what I’m trying to do. So it’s obvious now that I’m new to this; where in the fileset configuration would you place such a proc? The outbound section for fileset-local doesn’t seem to have a place.

All the same, I appreciate the offer for the pre-written proc but I’m more interested in learning the ropes.

[Russ Ross]

I use a TCL proc called tps_fileset_outbound to write one message per file.

Here is a URL with Source code and screen shots to illustrate:

Russ Ross
RussRoss318@gmail.com

[Jim Kosloskey]

It would go in the Outbound TPS.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[JP Trosclair]

Thanks guys, this gives me a lot to work with. I appreciate the information and time.

[JP Trosclair]

This is good, but there’s a glaring race condition here if you’ve got a 3rd party system picking these messages up. Any of you guys run into it? Any solution? I’m looking at the temp file option in the fileset properties dialog now. The docs specifically say the temp file is renamed and not copied so I think this would solve the problem so long as it’s not writing the temp file in the same output directory.

[Terry Kellum]

The way that I have handled this in the past is to have the third party pick up files like *.rdy .  Cloverleaf puts in files to the dir without the extension.  I then have a script that looks at the dir every minute and renames files that are over three minutes old.

[Charlie Bursell]

A bettor way IMHO

Write the files out to a temp directory.  Then have another thread in the same process with UPoC protocol.  Schedule that thread to run as often as you want to move files.

When the UPoC thread runs, move (don’t copy) the files to the final directory.  Move is an atomic operation so no way the fles can be picked up early. Also, since the UPoC thread runs in the same process there is no way the engine can be writing files at the same time you perform the move.

[Russ Ross]

I posted how we’ve been batching up real-time numfiles for FTP on a scheduled basis on the second page of this URL:

    <a href="https://usspvlclovertch2.infor.com/viewtopic.php?t=1907&postdays=0&postorder=asc&highlight=batchnumfile&start=15&#8243; class=”bbcode_url”>https://usspvlclovertch2.infor.com/viewtopic.php?t=1907&postdays=0&postorder=asc&highlight=batchnumfile&start=15

The basic concept at a high level is that we have a real-time interface create a num file of each ADT message real-time so that each num file has one ADT message in it.

Since the num files are sequntially named, when the directory listing is sorted they are also in chronological order.

A proc called batch_num_file.tcl that is argument driven is run on the desired time interval via a cron entry to batch up the currently available num files into a concatenated batch file, which is then available for the destination system to FTP/get or have Cloverleaf FTP/put.

In our case, I prefer to have the source system FTP/get the file which happens once an hour for this interface.

Granted much of the solution uses standalone scripts to complement cloverleaf because at that time in the year 2000 I was not fully capable in cloverleaf.

It has wokred well so no motivation to change it plus it hasn’t ever been impacted by a cloverleaf upgrade.

Russ Ross
RussRoss318@gmail.com

[Author]

Replies