FTP error

[Jawad Chaudhry]

I am using the ftp protocol to transfer evtx tfile. These files are about 400 MB each. I tried to transfer one, but I received this error:

[sms :sms :ERR /0:  logs_FW_FTP:09/27/2013 15:56:24] Tcl error:

       msgId   = message0

       proc    = ‘set_filename_driverctl’

       args    = ”

       result  = ‘unable to allocate fetch buffer’

       errorInfo: ‘

unable to allocate fetch buffer

   while executing

“msgget $mh “

   (“run” arm line 28)

   invoked from within

“switch -exact — $mode {

       start {

           # Perform special init functions

           # N.B.: there may or may not be a MSGID key in args

      …”

   (procedure “set_filename_driverctl” line 8)

   invoked from within

“set_filename_driverctl {MSGID message0} {CONTEXT sms_ib_data} {ARGS {}} {MODE r

un} {VERSION 3.0}”‘

Does anyone know why this is happening? And how it can be fixed? Thanks.

Topic

[David Barr]

There’s a memory limit in the TCL intpreter that is in the hciengine process. The limit is set at compile time, so there’s not much you can do to overcome the limit.

You can pass byte offset arguments to msgget to read the message in chunks  so that you don’t have to load the entire message into memory at once.

[David Barr]

I don’t know what your set_filename_driverctl  proc is doing, but maybe you could modify it to not call msgget. I don’t know why a proc that puts a filename into message metadata would need to know the contents of the message.

[Jawad Chaudhry]

Thanks for the reply. It looks like msgget is being used to see if the file is a zero byte file. I’ll remove that and try it.

[David Barr]

You can use “msglength $mh” to get the message size.

[Peter Heggie]

Does the msglength command access the entire file or allocate memory for the entire file?

I have a process that routes files based on filename, and the process is occasionally hanging when it handles large files. I do have Recovery On and we are working on a redesign that will eliminate the need for Recovery, but until then we need it On.

I went through all my tcl procs and removed any code that read the message, but I do have msglength in the code in order to get the size of the message.

Just wondering if the process is still incurring allocation of memory for the entire file just to execute this command.

Peter Heggie

[Jim Kosloskey]

What style are you using? Are you using eof?

With eof I think the entire file becomes one message. You might not want to do that.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Peter Heggie]

It is EOF.

Is there a better alternative? We want to use the facilities of Cloverleaf to act as a file router. We don’t want to do anything with the content of the files. We do need to look at the meta data, like the filename and the message creation time.

Would ‘single’ be a better way to manage the files?

Peter Heggie

[Jim Kosloskey]

What is the purpose of the msglength?

Are you trying to determine if you have a zero length file?

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Peter Heggie]

It is for auditing. Meta data about the file, the source, the target and the pickup & delivery activity are stored in an auditing table. We do not have logic for zero-length files, but that is a good use for it.

Peter Heggie

[Jim Kosloskey]

Well you could use a style which breaks the file into logical messages. The metadata will be the same for each message as far as filename, etc.

As for length of the file, perhaps you could do a msglength for each logical message adding to a global variable. Then at the File Delete UPoC stack a proc to get the length global variable (which should now be the logical length). The logical length may not be the same as the O/S recorded file length as some control characters may be included in the latter.

A benefit in having the file broken up into logical messages is the ability to use the throttling settings on the Fileset protocol which, when usaing EOF, are of little or no use.

I guess another option could be to increase memory.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Author]

Replies