SFTP upload options

[HoedjeRo]

Just a quick question really. On CL19.1 using RHEL I want to make a connection to an SFTP server and upload files.
Does it require a special license to do this? (Maybe cl-aom-ftps?)

Are the configuration examples?

I have tried this but get:

* Upload failed: No such file or directory (2/-31)

I know the directory is there and I made a dummy file to dump data into but nothing changes.

Topic

[Arie Klop]

What a coincidence, i have just been breaking my head about the sftp options.

The fields are very non descriptive and the documentation says nothing about what to fill in where.

I have a working config now, see screenprint. I think the tric is to use the full pathname and dont fill in any field you don’t need (that was my mistake).

If you don’t have the ssl licence, you will be notified in the process log.

Attachments:

You must be logged in to view attached files.

[HoedjeRo]

I’m still getting the error:

===========

Error while trying to write /Outbox/2023-02-15Benifit.csv.
Detailed error:Upload failed: No such file or directory (2/-31)
Curl errCode:78 Curl error: Remote file not found

 

I’m also using “/Outbox/%SYS.DATE%Benifit.csv” as the file template and “/Outbox/Benifit.csv” as the file config.
Nothing in the directories item.

Confused as I can easily do this using the same credentials with WinSCP

[Arie Klop]

My guess would be that WinSCP automatically redirects you to a home directory and you don’t see the full path.  As the sftp fileset does not have that possibility, you would need to find out the full path. Or you could try what happens if you remove the first slash from your path and see what happens.

[HoedjeRo]

Also when I use manual commend from a putty session I can log in on SFTP and put any file in the destination location.
I may need to do this with tcl as I can’t make the GUI to do the correct function and put the file there

[HoedjeRo]

When I have a putty session open to my RHEL CL server and there start stp user@<dest-ip> I get prompted for the password.
Enter password connect is established.

do command:
put HCAEmployeeSpecificBenefit14022023.csv Outbox/
And the file is transported without issue.

Still the GUI doesn’t give me the same result. I get:

[pd :pdtd:INFO/0:FTP_PERQUISITE:02/16/2023 09:57:45] [0.0.38518] Writing message to Protocol Driver fileset-ftp
[fset:init:INFO/1:FTP_PERQUISITE:02/16/2023 09:57:45] set ACTIVE mode
[fset:init:DBUG/0:FTP_PERQUISITE:02/16/2023 09:57:45] fcDoNetworkStuff: mode=5
* Trying 62.32.120.35…
* TCP_NODELAY set
* Connected to 62.32.120.35 (62.32.120.35) port 22 (#0)
< SSH-2.0-VShell_4_3_2_1566 VShell
* server response timeout
* Closing connection 0
[fset:wrte:ERR /0:FTP_PERQUISITE:02/16/2023 09:57:55] Error while trying to write Outbox/2023-02-16Benifit.csv.
[fset:wrte:ERR /0:FTP_PERQUISITE:–/–/—- –:–:–] Detailed error:server response timeout
[fset:wrte:ERR /0:FTP_PERQUISITE:–/–/—- –:–:–] Curl errCode:28 Curl error: Timeout was reached
[pd :pdtd:INFO/1:FTP_PERQUISITE:02/16/2023 09:57:55] Set driver status to PD_STATUS_OPENING
[pd :pdtd:INFO/1:FTP_PERQUISITE:02/16/2023 09:57:57] Set driver status to PD_STATUS_ERROR
[pd :pdtd:INFO/1:FTP_PERQUISITE:02/16/2023 09:57:57] [0.0.38518] Writing message failed

Attachments:

You must be logged in to view attached files.

[Paul Stein]

Any luck w/o using TCL? I have had to do a few of these in 19.1 and I have to define the ‘Directories’ location in outbound portion in addition to the file name template. I believe you can do absolute or relative path depending on where your home directory anchors your user account to.

[Arie Klop]

I see you did not specify a local binding address, that is a mandatory field. It won’t work without it.

It should be the dns name of your local server/cluster.  We have defined a global variable for this $$BINDING_ADDRESS so that we only have to change it once per site.

Also: not sure if you want active mode, i don’t use it.

• This reply was modified 1 year, 10 months ago by Arie Klop.

[Robert Kersemakers]

As Paul said, you need to specify ‘Directories’ in ‘Fileset Options’. So try:

Directories: /Outbox/
File Name Template: %SYS.DATE%Benefit.csv
File: Benefit.csv

Imho option ‘Local binding address’ is optional; I have never used it unless a specific system requires messages alqays coming from a certain address.

Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands

[HoedjeRo]

I have made the changes as suggested (not with a local binding address) and no joy.

Active mode disabled. Directory and template adjusted accordingly. Removed the SFTP option and left the drop down empty and made sure we’re still connecting to port 22.

[msg :Msg :DBUG/0:FTP_PERQUISITE:02/22/2023 15:38:52] msgFree 0x0x7f8eafda8f50 refCnt 1
[pd :pdtd:INFO/0:FTP_PERQUISITE:02/22/2023 15:38:52] [0.0.38559] Writing message to Protocol Driver fileset-ftp
[fset:init:DBUG/0:FTP_PERQUISITE:02/22/2023 15:38:52] fcDoNetworkStuff: mode=5
[pd :pdtd:INFO/1:FTP_PERQUISITE:02/22/2023 15:38:52] Set driver status to PD_STATUS_UP
* Trying 62.32.120.35…
* TCP_NODELAY set
* Connected to 62.32.120.35 (62.32.120.35) port 22 (#0)
< SSH-2.0-VShell_4_3_2_1566 VShell
* server response timeout
* Closing connection 0
[fset:wrte:ERR /0:FTP_PERQUISITE:02/22/2023 15:39:22] Error while trying to write 2023-02-22Benefit.csv.
[fset:wrte:ERR /0:FTP_PERQUISITE:–/–/—- –:–:–] Detailed error:server response timeout
[fset:wrte:ERR /0:FTP_PERQUISITE:–/–/—- –:–:–] Curl errCode:28 Curl error: Timeout was reached
[pd :pdtd:INFO/1:FTP_PERQUISITE:02/22/2023 15:39:22] Set driver status to PD_STATUS_OPENING
[pd :pdtd:INFO/1:FTP_PERQUISITE:02/22/2023 15:39:24] Set driver status to PD_STATUS_ERROR
[pd :pdtd:INFO/1:FTP_PERQUISITE:02/22/2023 15:39:24] [0.0.38559] Writing message failed

 

 

I’m trying to find out if I can authenticate with private and public key instead of username and password.

Jammer maar helaas.

[Arie Klop]

Do you have a source thread where you get the message from or want to upload from a directory?

If you have a source thread where you get the messages from you can leave the directory empty. I did and it works fine in my case.

But seriously, try filling the local binding address with the ip you are coming from, not 127.0.0.1.  Not sure why but it only works when i fill that.

[HoedjeRo]

I tried but that’s even worse. I have a thread feeding the SFTP connection but when I add a local binding address it fails (quicker).

[msg :Msg :DBUG/0:FTP_PERQUISITE:02/23/2023 09:01:01] msgFree 0x0x7fc9dd740d50 refCnt 1
[pd :pdtd:INFO/0:FTP_PERQUISITE:02/23/2023 09:01:01] [0.0.38559] Writing message to Protocol Driver fileset-ftp
[fset:init:DBUG/0:FTP_PERQUISITE:02/23/2023 09:01:01] fcDoNetworkStuff: mode=5
[pd :pdtd:INFO/1:FTP_PERQUISITE:02/23/2023 09:01:01] Set driver status to PD_STATUS_UP
* Trying 62.32.120.35…
* TCP_NODELAY set
* Name ‘uksalqapcl01’ family 2 resolved to ‘127.0.0.1’ family 2
* Local port: 0
* Immediate connect fail for 62.32.120.35: Invalid argument
* Closing connection 0
[fset:wrte:ERR /0:FTP_PERQUISITE:02/23/2023 09:01:01] Error while trying to write 2023-02-23Benefit.csv.
[fset:wrte:ERR /0:FTP_PERQUISITE:–/–/—- –:–:–] Detailed error:
[fset:wrte:ERR /0:FTP_PERQUISITE:–/–/—- –:–:–] Curl errCode:7 Curl error: Couldn’t connect to server

 

Or:

[msg :Msg :DBUG/0:FTP_PERQUISITE:02/23/2023 09:09:08] msgFree 0x0x7f907d98ed50 refCnt 1
[pd :pdtd:INFO/0:FTP_PERQUISITE:02/23/2023 09:09:08] [0.0.38559] Writing message to Protocol Driver fileset-ftp
[fset:init:DBUG/0:FTP_PERQUISITE:02/23/2023 09:09:08] fcDoNetworkStuff: mode=5
[pd :pdtd:INFO/1:FTP_PERQUISITE:02/23/2023 09:09:08] Set driver status to PD_STATUS_UP
* Trying 62.32.120.35…
* TCP_NODELAY set
* Name ‘10.96.244.30’ family 2 resolved to ‘10.96.244.30’ family 2
* Local port: 0
* Connected to 62.32.120.35 (62.32.120.35) port 22 (#0)
< SSH-2.0-VShell_4_3_2_1566 VShell
* server response timeout
* Closing connection 0
[fset:wrte:ERR /0:FTP_PERQUISITE:02/23/2023 09:09:38] Error while trying to write 2023-02-23Benefit.csv.
[fset:wrte:ERR /0:FTP_PERQUISITE:–/–/—- –:–:–] Detailed error:server response timeout
[fset:wrte:ERR /0:FTP_PERQUISITE:–/–/—- –:–:–] Curl errCode:28 Curl error: Timeout was reached
[pd :pdtd:INFO/1:FTP_PERQUISITE:02/23/2023 09:09:38] Set driver status to PD_STATUS_OPENING
[pd :pdtd:INFO/1:FTP_PERQUISITE:02/23/2023 09:09:40] Set driver status to PD_STATUS_ERROR
[pd :pdtd:INFO/1:FTP_PERQUISITE:02/23/2023 09:09:40] [0.0.38559] Writing message failed

• This reply was modified 1 year, 10 months ago by HoedjeRo.

[John Mercogliano]

Since infor switched to using curl as it’s engine, you cannot use the dot operator to indicate the home directory.
When you log into winscp and drill down to the drop off directory, whatever directory you see in the current path bar is what you need to use in the directory entry in cloverleaf.  Otherwise you will get the “Detailed error:Upload failed: No such file or directory ” error.

The value you need to use will depend on the configuration of the server you are logging into.  Here is a more detailed explanation https://unix.stackexchange.com/questions/293756/set-startup-folder-for-sftp-to-be-other-than-home-username-is-throwing-me-permi

But I always use winscp to check that actual working directory value.

Also are you using CURLOPT_VERBOSE set to 1 in the cURL options?  It doesn’t look like in your later errors you are actually authenticating.  The timeout usually is either authentication, whitelisting, or firewall issues that I have encountered.

For the binding address definitely not 127.  When connecting to a public sftp server I have never had to set a binding address as they are seeing the public addresses of the firewall.

Hope this helps.

John Mercogliano
Sentara Healthcare
Hampton Roads, VA

[HoedjeRo]

@John Mercogliano,

That is very helpful. I have tried something like this in the past but it seems that I must have missed something.
I have no successfully managed to test 2 different SFTP servers and managed to write to them.
As you can see in screenshots attached the directories in CL SFTP set up needs to match what is in WinSCP.

Many thanks,

Roland
HCA Healthcare UK

Attachments:

You must be logged in to view attached files.

[John Mercogliano]

Remove everything to the left of the colon including the colon.  You should only have /Outbox

I’ve attached a screen shot

It looks like it is adding the user name to the path which I’ve never seen before but winscp has a lot of configuration options.

The rest looks fine, at least similar configurations work for me.  If you have not set curl verbose to 1 you should do that to help troubleshoot.

 

 

 

Attachments:

You must be logged in to view attached files.

John Mercogliano
Sentara Healthcare
Hampton Roads, VA

[HoedjeRo]

@John-Mercogliano,
I tried removing the everything left of the colon including the colon but for this particular site it will then result in:
Error while trying to write Benefit.csv.
Detailed error:Upload failed: No such file or directory (2/-31)
Curl errCode:78 Curl error: Remote file not found

I have set the curl option.

If I add the username including the colon it works for this site.
For another site I have the same as in your screenshots and that works fine.
Just to let you know as some sites are a little weird it seems.

 

Thanks for your help.

Roland
HCA Healthcare UK

Attachments:

You must be logged in to view attached files.

[Author]

Replies