HELP: Output file keeps accumulating ???

[Rentian Huang]

Dear all,

I am testing two routes:

1. fileset-FTP -> File (with “Append to file” unchecked)

2. File -> File (with “Append to file” unchecked)

Everything works fine, the only problem kept happen to both these two routes is, the output file kept accumulate with whatever I have sent before!!!

Meaning that if I first start the process and get an output file of 20k, then I stop the process, and even delete the output file, and run the process the 2nd time, the filel will be 40k, 3rd time 60k…..???

Help…

Sam   😕

Topic

[David Caragay]

Hi Sam,

This might be related to the temp files that fileset-FTP uses while processing files.  It confused me for a bit until I figured out how fileset uses a temporary file for processing.

When using the fileset-FTP protocol to read incoming batch files, Cloverleaf will read the entire file into a temporary location before deleting the original file.  As a result of this “hidden” process, you will be unable to use standard Database Admin functions to delete unprocessed fileset-FTP records (should it be required).  This temporary file is not recognized by the Database Administrator (since it is not considered a valid recovery queue).  To delete the temporary fileset-FTP file and corresponding index file, do the following:

1.) Stop the fileset-FTP thread.

2.) Locate the temporary FTP file.  This can be found under $HCISITEDIR/exec/processes/”process_name”/ (where “process_name” is the name of the process where the inbound fileset-FTP thread is located).  The actual file name will be “thread_name”-FTP-temp (where “thread_name” is the fileset-FTP inbound thread name).  Remove this file.

3.) Locate the temporary FTP index file.  This can be found under $HCISITEDIR/exec/processes/”process_name”/ (where “process_name” is the name of the process where the inbound fileset-FTP thread is located).  The actual file name will be fileset-“thread_name” (where “thread_name” is the fileset-FTP inbound thread name).  Remove this file.

4.) If desired, delete appropriate records that have already been read by the inbound thread.  Use Database Administrator as required.

This will delete all pending fileset-FTP records.  The index file will be recreated when the thread is restarted.

The processes indicated above will give you a clean fileset startup.

[Rentian Huang]

David,

You are absolutely right! I have noticed them before and you gave me a clear definition 🙂

My problem was actually solved by UN-checking the “Use Recovery database” option in the inbound thread, since that will cause the engine to grab all msgs sent before and resend them, which makes the output file accumulated.

Hope that help other people with simular problems..

Good day,

Sam  😀

[David Caragay]

Sam,

Glad I could help.  Just be aware the tmp files are still going to be created with or without selecting recovery DB option.

Take care.

[John Stafford]

Sorry to have to bump this old thread, but I just noticed that the SMAT files for some of my FTP threads were empty, and the exec/processes/ftp directory had some large temp files in them. I stopped the threads, backed up the temp files and deleted them, and restarted the threads. I’ll be keeping an eye on the threads to see if the .idx and .msg files start populating. I also got a warning about opening SMAT files from an older version when I tried to open some of them.

What would cause the [thread]-FTP-temp files to get so large, and why would the messages not write to the SMAT files as expected? Is this an indication of something going wrong with the threads?

[Author]

Replies