FTP error

  • Creator
    Topic
  • #53845
    Jawad Chaudhry
    Participant

      I am using the ftp protocol to transfer evtx tfile. These files are about 400 MB each. I tried to transfer one, but I received this error:

      [sms :sms :ERR /0:  logs_FW_FTP:09/27/2013 15:56:24] Tcl error:

             msgId   = message0

             proc    = ‘set_filename_driverctl’

             args    = ”

             result  = ‘unable to allocate fetch buffer’

             errorInfo: ‘

      unable to allocate fetch buffer

         while executing

      “msgget $mh “

         (“run” arm line 28)

         invoked from within

      “switch -exact — $mode {

             start {

                 # Perform special init functions

                 # N.B.: there may or may not be a MSGID key in args

            …”

         (procedure “set_filename_driverctl” line 8)

         invoked from within

      “set_filename_driverctl {MSGID message0} {CONTEXT sms_ib_data} {ARGS {}} {MODE r

      un} {VERSION 3.0}”‘

      Does anyone know why this is happening? And how it can be fixed? Thanks.

    Viewing 9 reply threads
    • Author
      Replies
      • #79161
        David Barr
        Participant

          There’s a memory limit in the TCL intpreter that is in the hciengine process. The limit is set at compile time, so there’s not much you can do to overcome the limit.

          You can pass byte offset arguments to msgget to read the message in chunks  so that you don’t have to load the entire message into memory at once.

        • #79162
          David Barr
          Participant

            I don’t know what your set_filename_driverctl  proc is doing, but maybe you could modify it to not call msgget. I don’t know why a proc that puts a filename into message metadata would need to know the contents of the message.

          • #79163
            Jawad Chaudhry
            Participant

              Thanks for the reply. It looks like msgget is being used to see if the file is a zero byte file. I’ll remove that and try it.

            • #79164
              David Barr
              Participant

                You can use “msglength $mh” to get the message size.

              • #79165
                Peter Heggie
                Participant

                  Does the msglength command access the entire file or allocate memory for the entire file?

                  I have a process that routes files based on filename, and the process is occasionally hanging when it handles large files. I do have Recovery On and we are working on a redesign that will eliminate the need for Recovery, but until then we need it On.

                  I went through all my tcl procs and removed any code that read the message, but I do have msglength in the code in order to get the size of the message.

                  Just wondering if the process is still incurring allocation of memory for the entire file just to execute this command.

                  Peter Heggie
                  PeterHeggie@crouse.org

                • #79166
                  Jim Kosloskey
                  Participant

                    What style are you using? Are you using eof?

                    With eof I think the entire file becomes one message. You might not want to do that.

                    email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

                  • #79167
                    Peter Heggie
                    Participant

                      It is EOF.

                      Is there a better alternative? We want to use the facilities of Cloverleaf to act as a file router. We don’t want to do anything with the content of the files. We do need to look at the meta data, like the filename and the message creation time.

                      Would ‘single’ be a better way to manage the files?

                      Peter Heggie
                      PeterHeggie@crouse.org

                    • #79168
                      Jim Kosloskey
                      Participant

                        What is the purpose of the msglength?

                        Are you trying to determine if you have a zero length file?

                        email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

                      • #79169
                        Peter Heggie
                        Participant

                          It is for auditing. Meta data about the file, the source, the target and the pickup & delivery activity are stored in an auditing table. We do not have logic for zero-length files, but that is a good use for it.

                          Peter Heggie
                          PeterHeggie@crouse.org

                        • #79170
                          Jim Kosloskey
                          Participant

                            Well you could use a style which breaks the file into logical messages. The metadata will be the same for each message as far as filename, etc.

                            As for length of the file, perhaps you could do a msglength for each logical message adding to a global variable. Then at the File Delete UPoC stack a proc to get the length global variable (which should now be the logical length). The logical length may not be the same as the O/S recorded file length as some control characters may be included in the latter.

                            A benefit in having the file broken up into logical messages is the ability to use the throttling settings on the Fileset protocol which, when usaing EOF, are of little or no use.

                            I guess another option could be to increase memory.

                            email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

                        Viewing 9 reply threads
                        • The forum ‘Cloverleaf’ is closed to new topics and replies.