repeated FTP inbound processing – out of space

[Peter Heggie]

There is a Cloverleaf inbound thread configured for Fileset-FTP. The Inbound SMAT DB has 61 entries for the same input file (over an hour’s time). There was a space issue with that file system, and the external system that was placing files there (also using FTP) was hung up and eventually disabled until we cleared out some old files.

The file in question was sent through the Cloverleaf interface 61 times, resulting in 61 emails to the recipient, which was not well received.

The inbound fileset-FTP configuration does not have a tps specified for the directory parse or delete behind, so I am curious why the file was processed 61 times, unless it actually was delivered by the external system 61 times because of its own problems with the file system space.

FYI – the external system delivers about 25 different files each morning to this directory and there could have been other file transfers in progress. I’m not sure if the external system is capable of running multiple simultaneous file transfers. Before I blame the external system, I’m wondering if any Cloverleaf behavior could reprocess the same file multiple times.

Peter

Peter Heggie
PeterHeggie@crouse.org

Topic

[Charlie Bursell]

If the file did not get deleted after it was processed it could be received many times.  The default action is to pick up any file in the remote directory.

Perhaps during your troubled times the remote system was not deleting the file as instructed by the Cloverleaf Fileset driver.

Did you look at the process logs?

[Peter Heggie]

Yes I was looking at the process logs and saw the debugging output but did not see any error output for that particular file. I did see broken pipe errors for other files being written (attempted) to another folder in the same file system:

[fset:wrte:ERR /0:   to_backups:09/23/2016 03:35:30] Error while trying to write EPSiRefActvCd_CH_20160923020046.txt.

[fset:wrte:ERR /0:   to_backups:–/–/—- –:–:–]                           Detailed error:Send failure: There is no process to read data written to a pipe.

[fset:wrte:ERR /0:   to_backups:–/–/—- –:–:–]                           Curl errCode:55 Curl error: Failed sending data to the peer

The above error repeats every 10-15 seconds, for about 30 minutes.

Now that I look at this more closely, I see that this other file name above was also continually re-processed for 29 times on inbound side, and the above messages came out of the outbound side threads.

The first file I talked about was empty for the first 60 entries in the inbound SMAT, but the last entry was a complete file. Maybe I’m jumping to conclusions, but maybe the file system was full, and the first 60 times, no data could be written, although an empty file was created. Each one triggered a Cloverleaf process. Part of the process is to write a backup to another folder in the same file system. Maybe that’s a bad design.

So no errors per se on the inbound side, but looks like at least two different files were transmitted several times, with many of those empty or partially written. And then we saw errors in the log from the outbound side, when Cloverleaf attempted to write the backup copy of the file.

Peter Heggie
PeterHeggie@crouse.org

[Charlie Bursell]

When you have FTP problems you should always test it manually doing the FTP from a command line to see if it is OK there.

I would turn up Fileset EO or enable all.

If you continue to have problems you should contact Support

[Peter Heggie]

Thank you – I think it was working correctly, and there were two problems – one caused directly by being out of space on the Write. The other was the multiple copies of the same inbound file, also caused by out of space, but it was the external system that kept re-trying to send the same file.

I think my design needs to be strengthened to manage this better, and I need to automate an archiving solution.

Peter Heggie
PeterHeggie@crouse.org

[Author]

Replies