Invalid arguments during FTP scan filled up the whole log

[B Pariyesana]

Good morning,

We just experienced problem where Cloverlear was having problem during FTP directory listing and then the process hang and Cloverleaf filled up the log file with millions of lines of this :

[pti :sele:WARN/0:      guh_cmd:03/23/2010 05:20:07] Select returns -1 22: Invalid argument

It happened when cloverleaf was starting to scan the FTP directory based on the advanced scheduling.

We are using 5.6 on AIX. Anybody has experience this issue before and can share what is the cause and how to fix it?

Thanks

Budhi Pariyesana

MedStar Health

Topic

[Robert Kersemakers]

Morning!

I’m guessing one of the advanced scheduling settings is invalid. I have no experience with advanced scheduling, but maybe if you post your settings, others can help you with that.

Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands

[Vinita Dutta]

This happened to us also today – were you able to find out what happened.

Same error – [pti :sele:WARN/0:live_rpt2_cmd:05/26/2010 03:44:01] Select returns -1 22: Invalid argument.

No settings have been changed and we are using 5.5/AIX

Thanks

[Jennifer Hardesty]

Ours occurred just after our nightly cyclesave in our test 5.7 area.

Quote:

[prod:prod:INFO/0:parx_bill_cmd:07/05/2010 23:59:21] FEATURE cl-aom-ihb hcilicmgrd 5.7 permanent uncounted 57A414FFCFC9

[prod:prod:INFO/0:parx_bill_cmd:07/05/2010 23:59:21] HOSTID=9d37b633cd117a77 ck=172 category=testdev

[prod:prod:INFO/0:parx_bill_cmd:07/05/2010 23:59:21] Started at Mon Jul  5 23:59:21 2010

[prod:prod:INFO/0:parx_bill_cmd:07/05/2010 23:59:21] Engine process is 1044616 on host bmc_p550_01

[pti :sele:WARN/0:parx_bill_cmd:07/05/2010 23:59:21] Select returns -1 22: Invalid argument

Has anyone figured out what this is?

[Jennifer Hardesty]

Well, I thought I’d check back in with this topic to see if anyone else is still experiencing this and/or if any resolutions have been found.

We have had this occur several times in Production now and the error message caused the disc space on Cloverleaf to almost fill up before it was caught.  It did fill up on Test once.

In once scenario, the issue occurred after a file was picked up and and routed but the ftp protocol continued to scan the ftp folder for files — it was scheduled to pick up at 21:45:* and it appears that the error occurred at 21:46:01.

In another scenario, the advanced scheduler is scanning at 00, 20, and 40 minutes but no hours or seconds are set.  At approxamately 1am every morning, the connection is “broken” and the log fills up with error messages, filling up the Cloverleaf disc space until bad things happen.

Both ftp servers are within the local network so no VPN issue.

Any ideas?  Advice and Suggestions are much welcome.  Thanks!

[Keith McLeod]

First thing I would do if you are referring to filling up the process log is configure it to automatically cycle at a defined file size.

In the NetConfig, select process, configure and then for each process define a max auto Log Cycling threshold.  This will stabilize the volume of your process logs.  I set my logs for 50000KB so for each process, my max is 100000KB since when it cycles it creates the .old version.

Not sure of the ftp issue without seeing any detail.

[Chris Williams]

Jennifer,

Taking you literally, when you set the time as 21:45:*, it will run once every second. Do you intend for it to run 60 times during that minute? If you want it to run only once during that minute you need to sepcify at which second during that minute.

Chris

[Jennifer Hardesty]

I have discovered in the past that if I put a 0 in the seconds spot and for some reason there is a glitch in the connection at exactly that second, then the ftp function will hang/not try to reconnect for 24 hours, which is a bad thing if you are using the functionality to handle billing batches once a day.  However, if I leave the 0 as a * and there is a glitch such as the file on the ftp server is late or the host connection was momentarily dropped, then the ftp functionality will immediately try again; it’s a 60-second precaution window that 99% of the time does not cause any issues.

Also, it does not appear to attempt to run every second.  It only does so every 5 or so seconds, which I suspect means it waits the read interval.  However, when the error occurs in the first scenario, it is 1 second after that scheduled minute has concluded.

[John Stafford]

Looks like I’ve had the same issue as the original poster.

We have an FTP job that runs each night, and has done so fine for the last year (as long as I’ve been in this position). Last night, however, the audit and error logs exploded to over 550MB a piece, filled with the following error:

[fset:read:ERR /0:    f_impac_c:05/06/2013 20:15:05] Error while trying to get directory listing.

                        Detailed error:a timeout was reached

                          Curl errCode:28 Curl error:a timeout was reached

[pti :sele:WARN/0:  billing_cmd:05/06/2013 20:15:07] Select returns -1 22: Invalid argument

[pti :sele:WARN/0:  billing_cmd:05/06/2013 20:15:07] Select returns -1 22: Invalid argument

[pti :sele:WARN/0:  billing_cmd:05/06/2013 20:15:07] Select returns -1 22: Invalid argument

[pti :sele:WARN/0:  billing_cmd:05/06/2013 20:15:07] Select returns -1 22: Invalid argument

[pti :sele:WARN/0:  billing_cmd:05/06/2013 20:15:07] Select returns -1 22: Invalid argument

[pti :sele:WARN/0:  billing_cmd:05/06/2013 20:15:07] Select returns -1 22: Invalid argument

This is a scheduled FTP job, and we have had no issues with this prior to last night. I realize that this is aberrant behavior, but is there any way to deal with it other than the filling the filesystem and having production crash? Can I set up an alert related to the log file size? Is there any way to narrow down the cause of this issue?

As always, thanks in advance for your wisdom.

[Michael Hertel]

I had an issue with an interface once. Looking on the internet for error 22, it looks like it could be

[Author]

Replies