FTP Protocol filename embedded spaces

[Peter Heggie]

Sorry if this has been asked before, I can’t seem to get the search to work today.

Has anyone been able to use the FTP protocol to get and delete-behind a file having a filename with embedded spaces?

I’ve always gone out of my way to avoid dealing with these but I may not have a choice on this one.

We are running on AIX and the files are on Windows.

Peter Heggie

Topic

[Jim Kosloskey]

If the problem is retrieving the file name, you may need to use a dirparse routine. I think when the list of found files ia presented to the dirparse routine, the spaces imbedded in the found file name is preserved but I also think you need to then enclose that name in single quotes (if memory is correct) then return the list with the quote enclosed name(s) to Cloverleaf.

If the problem is writing the file name then you may need (either in addition or alone) to change the file name in the metadata adding the single quotes on the OB Tps before Cloverleaf writes the messages.

This is all from memory but I think AIX would like single quotes around file names with imbedded spaces. If AIX wants something different then do that.

email: jim.kosloskey@jim-kosloskey.com

[Peter Heggie]

ok I will try that. Just judging by what I see and what I don’t see in the log, the issue seems to occur when the FTP protocol invokes the delete behind. That is when the FTP DELE command is “not understood”.

I will try a DirParse routine that wraps single quotes around the name.

Peter Heggie

[Jim Kosloskey]

If the issue is happening at the Delete, then use a delete proc and change the name there.

email: jim.kosloskey@jim-kosloskey.com

[Peter Heggie]

I used the same proc for DirParse and DirParseDelete. It checks to make sure that it does not already have a single quote, and if not, then it wraps the name with a single quote. I don’t see any errors, yet the file is not deleted and it keeps re-sending. So I think I’m halfway there, and I need to fix the Delete portion. Thank you – I will keep you posted.

Peter Heggie

[Jim Kosloskey]

If you are trying to delete from a Windows platform, the rules might be different. For example maybe you need to make the file name on the deletion portion seem to be a true list element. I seem to recall Cloverleaf is wanting a list and if you just send the file name with spaces imbedded it will seem to be a multiple element list. Perhaps just use a LIST Tcl command around the file name before returning to Cloverleaf.

email: jim.kosloskey@jim-kosloskey.com

[Peter Heggie]

Very interesting results!

Not using any directory parse for the ‘Get’ operation works fine with files having spaces in the file names. However I did need that directory parse routine for the Delete (Behind) function. The strange part is that the Delete routine does not do anything special – it does not add quotes.

It strips out the open and close curly brackets and then lappends each file to a list which, after the processing loop, is msgset to the message.

Now that I have gotten past this issue, I am having an issue with the outbound FTP thread. I cannot set the file name! I think I had this same issue with Cloverleaf 6.1 – when using Fileset-FTP in an outbound thread, the DRIVERCTL value on the message is ignored. I don’t remember the exact combination of settings where this error occurs. I’m trying some variations.

Here is some debug log output:

Code:

[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–] msg: 0xeffe403c
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgType           : DATA
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgClass          : PROTOCOL
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgState          : OB in prewrite (17)
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgPriority       : 5120
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgRecoveryDbState: Log:off (1)
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgFlags          : 0x8008002
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgMid            : [0.0.17431683]
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgSrcMid         : [0.0.17431680]
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgSrcMidGroup    : midNULL
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgHostId         : 871487365
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgOrigSrcThread  : fr_mpf_si
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgOrigDestThread : to_mpf_doc
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgSrcThread      : fr_mpf_si
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgDestThread     : to_mpf_doc
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgXlateThread    :
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgSkipXlate      : 0
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgSepChars       :
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgNumRetries     : 0
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgGroupId        : 0
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgDriverControl  : {FILESET {{OBFILE 200560296_DLVNT_Crouse Hospital_Provider Delivery Note_MISSING to MISSING.pdf}}}
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgRecordFormat   :
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgRoutes         :
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgUserData       :
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgStaticIsDirty  : 1
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgVariableIsDirty: 1
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgRetryTimes     :
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgTimeStartIb    : 1540996328.202
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgTimeStartOb    : 1540996328.272
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgTimeCurQueStart: 0.000
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgTimeTotalQue   : 0.155
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgTimeRecovery   : 1540996328.446
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgEoConfig       : 0x0
[pd  :thrd:DBUG/3:   to_mpf_doc:–/–/—- –:–:–]     msgData (BO)      : 0xeffe4144

Code:

[pd  :pdtd:INFO/0:   to_mpf_doc:10/31/2018 10:32:08] [0.0.17431683] Writing message to Protocol Driver fileset-ftp
[fset:wrte:INFO/1:   to_mpf_doc:10/31/2018 10:32:08] Parsing control string ‘{OBFILE 200560296_DLVNT_Crouse Hospital_Provider Delivery Note_MISSING to MISSING.pdf}’
[fset:wrte:ERR /0:   to_mpf_doc:10/31/2018 10:32:08] ‘OBFILE 200560296_DLVNT_Crouse Hospital_Provider Delivery Note_MISSING to MISSING.pdf’ does not match ‘key value’

I don’t know what this error message means – does not match ‘key value’. Is my DRIVERCTL format wrong?

Peter Heggie

[Jim Kosloskey]

It looks like your OBFILE has multiple file names (at least it will look that way to DRIVERCTL). OBFILE is a keyed list with a single element as the value in thee KEY/value pair. What you are providing I think is supposed to be a single file name but looks like multiple elements wit the embedded spaces.

Try making your file name a single list element (I would use one of the list commands to make sure it set correctly.

Anyway that is my take on what you are seeing.

email: jim.kosloskey@jim-kosloskey.com

[Peter Heggie]

OK I got it to work. I don’t think it is a bug in Cloverleaf. Because of the embedded spaces (and embedded dashes!) it needed another pair of curly brackets.

Here is the code:

Code:

set driverctl “{FILESET {{OBFILE {$filename}}}}”

This seems to work well.

Thank you for your help!

Peter

Peter Heggie

[Peter Heggie]

I missed your last reply! I was close..?

So, for an output thread – I just want to confirm – the thread is processing one message / one file at a time, so I don’t have to worry about having multiple file names to worry about – there should just be one file name as the value of OBFILE? And I just need to enclose that variable in an extra pair of brackets and I should be good to go..

Peter Heggie

[Jim Kosloskey]

The FILESET keyed list has a child keyed list with the key OBFILE. The OBFILE Keyed list expects a single value (since it is applied with each message and logically a message can only be placed in one file at a time) and I believe that value is expected to be a list of a single element.

When building the OBFILE keyed list, keyed list Tcl commands should be used. If the value to be associated with the OBFILE key has imbedded spaces then I would use a LIST command (which will apply the proper encapsulatioon for a list). You can always just enclose your value with curly braces but I prefer to use the proper Tcl commands.

I would also use the keyed list commands (probably keylset) to properly set the Keyed list set along with list commands rather than code the curly braces myself in a set command.

email: jim.kosloskey@jim-kosloskey.com

[Peter Heggie]

The following code:

Code:

     set filename [file tail $filename]
     if {$debug > 1} {echo “$module base filename: $filename”}

     set filenamelist [list “$filename”]
     keylset obfile OBFILE $filenamelist
     keylset driverctl FILESET $obfile

writes this file to the output directory:

{600006076_AS0313_Crouse Hospital_Patient Clinical Factors_10-29-2018 to 10-29-2018.pdf}

The following code:

Code:

     set filename [file tail $filename]
     if {$debug > 1} {echo “$module base filename: $filename”}

     keylset obfile OBFILE $filename
     keylset driverctl FILESET $obfile

writes this file to the output directory:

600006076_AS0313_Crouse Hospital_Patient Clinical Factors_10-29-2018 to 10-29-2018.pdf

Could it be that the keylset creates a one-item, embedded-space wrangling, list automatically?

Peter Heggie

[Jim Kosloskey]

I think the OBFILE keyed list is the child of a nested keyed list (FILESET.OBFILE  would be the proper syntax I think). Setting the keys separately then mashing them together could produce incorrect results

email: jim.kosloskey@jim-kosloskey.com

[Peter Heggie]

I tried this code:

Code:

     set filename [file tail $filename]
     if {$debug > 1} {echo “$module base filename: $filename”}

     set filenamelist [list “$filename”]
     keylset driverctl FILESET.OBFILE $filenamelist

and the file written to the output directory is this:

{600006076_AS0313_Crouse Hospital_Patient Clinical Factors_10-29-2018 to 10-29-2018.pdf}

Then I tried this:

Code:

     set filename [file tail $filename]
     if {$debug > 1} {echo “$module base filename: $filename”}

     keylset driverctl FILESET.OBFILE $filename

and the file written to the output directory is this:

600006076_AS0313_Crouse Hospital_Patient Clinical Factors_10-29-2018 to 10-29-2018.pdf

The latter way seems to be good.

Peter Heggie

[Jim Kosloskey]

Something like this:

keylset msg_driverctl_kl FILESET.OBFILE $file_name

msgmetaset $mh DRIVERCTL $msg_driverctl_kl

Where $file_name is previously set in this case

email: jim.kosloskey@jim-kosloskey.com

[Peter Heggie]

yes – sorry – should have posted all of the code:

Code:

     set filename [file tail $filename]
     if {$debug > 1} {echo “$module base filename: $filename”}

     keylset driverctl FILESET.OBFILE $filename
     if {$debug > 1} {echo “$module driverctl: $driverctl”}

     msgmetaset $mh DRIVERCTL “$driverctl”

thank you

Peter Heggie

[Author]

Replies