secure ftp question

[Kevin Crist]

i have been tasked with building a secure ftp interface and have never done it before. Isn’t there a secure certificate you have to have to do this or am i mistaken? We are on CL 6.0.1 using RHLX.

Any help would be appreciated.

thanks.

Topic

[Jim Kosloskey]

Yes – as I reecall it is a no-cost license. Check with your Infor person.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Peter Heggie]

Could that policy have changed over the years? Maybe I misread other posts previously that seemed to imply that, although FTP comes with the base product, SFTP requires a purchased add-on component. I’m not sure what the add-on component is. Or is this something else?

Thanks,

Peter

Peter Heggie

[Jim Kosloskey]

There is an add-on which I think covers both SSL and SFTP.

I believe it is a no-cost add-on but must be ordered and licensed – again your account rep is the best source of that information.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Kevin Crist]

Unfortunately we have to go through McKesson so not sure if it will be same as Infor. I have requested the information. Will let you know what i find

[Robert Kersemakers]

Hmmm. I thought there were costs involved with this.

Kevin: if you need some scripts to use sftp without these add-on but with expect and sftp-capabilities of the OS, just let me know. Depends on your OS though; we are on RHEL.

Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands

[Peter Heggie]

I thought there were costs also. I have a call scheduled for Friday with our rep to find out.

We do have batch scripts to perform SFTP which work fine, but prefer to incorporate as much as possible inside a Cloverleaf interface.

Peter Heggie

[Robert Kersemakers]

These are UPOC tcl scripts to get/put messages/files through sftp. Have a look and see if you can use them or change them to your own needs. They work for us, but no guarantee they work for others. Based on a few scripts I got off this site from Max Drown.

######################################################################
# Name:      orbis_sftp_in
# Purpose:   Custom protocol for receiving messages via sftp
#            Put this proc in UPOC to read the files from sftp.
#            Also put this in TPS Inbound Data to pass the original
#            filename of the file. This is done through a global
#            ‘orbis_sftp_filenames’ which is a list of filenames.
# UPoC type:   tps
# Args:    tps keyedlist containing the following keys:
#          MODE    run mode (”start”, “run” or “time”)
#          MSGID   message handle
#          ARGS    user-supplied arguments:
#                  SFTPSERVER – sftp server hostname or IP
#                  SFTPUSER   – sftp user name
#                  SFTPPWD    – sftp user password
#                  SFTPDIR    – sftp directory where to get files; all files are taken!!
#                  SFTPFILTER – simpel filter (like ‘*.txt’ or ‘*log*.txt’) to get certain files.
#                               Default: ‘*’ (= all files in directory).
#                  BACKUPDIR  – directory (local, so should be available by Cloverleaf) where a copy of the files are placed
#                               Default: empty, so no backup is made
#                  LOCALDIR   – local directory where files are temporarily saved
#                  TMPDIR     – local tmp directory where expect script is written; should be different from LOCALDIR!
#                               Default: ${HCIROOTDIR}/temp
#                  EXPECTSN   – Expect script name (temp)
#                  DEBUG      – Debug information level (default = 0)
#
# Returns: tps disposition list:
#          
#

proc orbis_sftp_in { args } {
   keylget args MODE mode                 ;# Fetch mode
   keylget args ARGS uargs                ;# Get user arguments
   keylget args CONTEXT ctx               ;# Context

   global HciConnName HciRootDir HciSiteDir orbis_sftp_filenames

   set debug 0 ; keylget uargs DEBUG debug

   if { ! [info exists HciConnName] } {
       set HciConnName “UNKNOWN_TD”
   }
   set procName [lindex [info level 0] 0]
   set module “$HciConnName: $procName”

   set dispList {}            ;# Nothing to return

   switch -exact — $mode {
       start {
           # Perform special init functions
           # N.B.: there may or may not be a MSGID key in args

           # Initialiseer de global ‘orbis_sftp_filenames’
           set orbis_sftp_filenames {}

           if {$debug > 0} {
             echo “Start – orbis_sftp_filenames: ”
           }

       }

       run {
           # ‘run’ mode always has a MSGID; fetch and process it

           # Neem het eerste element van global ‘orbis_sftp_filenames’ als naam.
           # Indien deze global leeg is, dan een tijdelijke naam verzinnen.

           keylget args MSGID mh
           set filename [lvarpop orbis_sftp_filenames 0]

           # Indien geen originele filenaam, dan filenaam maken
           if {[string length $filename] 0} {
             echo “Run – filename: ”
           }

       }

       time {
           # Timer-based processing
           # N.B.: there may or may not be a MSGID key in args

           # sftp user name
           keylget uargs SFTPUSER user
           if { ![info exists user] } {
             error “$module Please set up SFTPUSER parameter”
           }

           # sftp password
           keylget uargs SFTPPWD password
           if { ![info exists password] } {
             error “$module Please set up SFTPPWD parameter”
           }

           # sftp host name
           keylget uargs SFTPSERVER host
           if { ![info exists host] } {
             error “$module Please set up SFTPSERVER”
           }

           # sftp dir name
           keylget uargs SFTPDIR dir
           if { ![info exists dir] } {
             error “$module Please set up SFTPDIR”
           }

           # sftp filter
           set filter “*”
           keylget uargs SFTPFILTER filter

           # backup dir
           set backupdir “”
           keylget uargs BACKUPDIR backupdir
           if { ([string length $backupdir] > 0) && ![file isdirectory $backupdir] } {
             error “$module BACKUPDIR ${backupdir} does not exist”
           }

           # local dir name
           keylget uargs LOCALDIR localdir
           if { ![info exists localdir] || ![file isdirectory $localdir] } {
             error “$module Please set up LOCALDIR”
           }

           # tmp dir name
           set tmpdir [file join ${HciRootDir} temp]
           keylget uargs TMPDIR tmpdir
           if { ![file isdirectory $tmpdir] } {
             error “$module Please set up TMPDIR”
           }

           # Expect script name
           keylget uargs EXPECTSN expectSN
           if { ![info exists expectSN] } {
             # Bepaal uniek nummer om bestandsnaam uniek te maken
             set ctrfile “orbis_sftp_in_icr”
             if ![file exists “$ctrfile.ctr”] {
               CtrInitCounter $ctrfile file 1 10000 1
             }
             # Get next counter value
             set number [CtrNextValue $ctrfile file]
             set expectSN “sftp_in_tmpscript_${number}”
           }
           set esn [file join ${tmpdir} ${expectSN}]

           # Haal nu de bestanden via sftp uit de genoemde directory op
           # en zet deze in de localdir.

           # create expect script
           set es [open $esn w]

           # Zet de timeout op 60 seconden
           puts $es “set timeout 60”
           # Start sftp
           puts $es “spawn sftp $user@$host”
           # Wacht op wachtwoord en voer dit in
           puts $es “expect {”
           puts $es ” “password:” {send “$password\r”}”
           puts $es ” “Connection closed” return”
           puts $es “}”
           puts $es “expect {”
           puts $es ” timeout {exit 1}”
           puts $es ” “sftp>””
           puts $es “}”
           # Ga lokaal naar local directory
           puts $es “send “lcd $localdir\r””
           puts $es “expect {”
           puts $es ” timeout {exit 1}”
           puts $es ” “sftp>””
           puts $es “}”
           # Ga remote naar de sftp directory
           puts $es “send “cd $dir\r””
           puts $es “expect {”
           puts $es ” “Couldn’t canonicalise: No such file or directory” {exit 1}”
           puts $es ” timeout {exit 1}”
           puts $es ” “sftp>””
           puts $es “}”
           # Haal de bestanden via de filter op en controleer of ophalen goed is gegaan
           puts $es “send “mget $filter\r””
           puts $es “expect {”
           puts $es ” timeout {exit 1}”
           puts $es ” “sftp>””
           puts $es “}”
           puts $es “send “bye\r””
           puts $es “expect eof”

           flush $es
           close $es
           if {$debug > 1} {
             echo “expect script:[exec cat $esn]”
           }

           # execute expect
           set retVal [exec $HciRootDir/tcl/bin/expect $esn]
           puts “$module retVal: ”
           if { [info exists errorCode] } {
               puts “$module errorCode: ”
           }
           file delete $esn

           # Verwerken van eventueel zojuist opgehaalde bestanden
           # Bestanden moeten daarna weer verwijderd worden via SFTP; via removelist
           set removelist {}
           # Haal de bestanden op uit de localdir
           # Directories krijgen door de -1 (min [code]######################################################################
# Name:      orbis_sftp_in
# Purpose:   Custom protocol for receiving messages via sftp
#            Put this proc in UPOC to read the files from sftp.
#            Also put this in TPS Inbound Data to pass the original
#            filename of the file. This is done through a global
#            ‘orbis_sftp_filenames’ which is a list of filenames.
# UPoC type:   tps
# Args:    tps keyedlist containing the following keys:
#          MODE    run mode (”start”, “run” or “time”)
#          MSGID   message handle
#          ARGS    user-supplied arguments:
#                  SFTPSERVER – sftp server hostname or IP
#                  SFTPUSER   – sftp user name
#                  SFTPPWD    – sftp user password
#                  SFTPDIR    – sftp directory where to get files; all files are taken!!
#                  SFTPFILTER – simpel filter (like ‘*.txt’ or ‘*log*.txt’) to get certain files.
#                               Default: ‘*’ (= all files in directory).
#                  BACKUPDIR  – directory (local, so should be available by Cloverleaf) where a copy of the files are placed
#                               Default: empty, so no backup is made
#                  LOCALDIR   – local directory where files are temporarily saved
#                  TMPDIR     – local tmp directory where expect script is written; should be different from LOCALDIR!
#                               Default: ${HCIROOTDIR}/temp
#                  EXPECTSN   – Expect script name (temp)
#                  DEBUG      – Debug information level (default = 0)
#
# Returns: tps disposition list:
#          
#

proc orbis_sftp_in { args } {
   keylget args MODE mode                 ;# Fetch mode
   keylget args ARGS uargs                ;# Get user arguments
   keylget args CONTEXT ctx               ;# Context

   global HciConnName HciRootDir HciSiteDir orbis_sftp_filenames

   set debug 0 ; keylget uargs DEBUG debug

   if { ! [info exists HciConnName] } {
       set HciConnName “UNKNOWN_TD”
   }
   set procName [lindex [info level 0] 0]
   set module “$HciConnName: $procName”

   set dispList {}            ;# Nothing to return

   switch -exact — $mode {
       start {
           # Perform special init functions
           # N.B.: there may or may not be a MSGID key in args

           # Initialiseer de global ‘orbis_sftp_filenames’
           set orbis_sftp_filenames {}

           if {$debug > 0} {
             echo “Start – orbis_sftp_filenames: “
           }

       }

       run {
           # ‘run’ mode always has a MSGID; fetch and process it

           # Neem het eerste element van global ‘orbis_sftp_filenames’ als naam.
           # Indien deze global leeg is, dan een tijdelijke naam verzinnen.

           keylget args MSGID mh
           set filename [lvarpop orbis_sftp_filenames 0]

           # Indien geen originele filenaam, dan filenaam maken
           if {[string length $filename] <= 0} {
             set ctrfile “sftp_filename_in_icr”
             if ![file exists “${ctrfile}.ctr”] {
               CtrInitCounter $ctrfile file 1 100000000 1
             }
             # Get next counter value
             set counter [list [CtrNextValue $ctrfile file]]

             set filename “TEMP_FILENAME_${counter}”
           }

           set driverctl “{FILESET {{OBFILE $filename}}}”
           msgmetaset $mh DRIVERCTL $driverctl
           lappend dispList “CONTINUE $mh”

           if {$debug > 0} {
             echo “Run – filename: “
           }

       }

       time {
           # Timer-based processing
           # N.B.: there may or may not be a MSGID key in args

           # sftp user name
           keylget uargs SFTPUSER user
           if { ![info exists user] } {
             error “$module Please set up SFTPUSER parameter”
           }

           # sftp password
           keylget uargs SFTPPWD password
           if { ![info exists password] } {
             error “$module Please set up SFTPPWD parameter”
           }

           # sftp host name
           keylget uargs SFTPSERVER host
           if { ![info exists host] } {
             error “$module Please set up SFTPSERVER”
           }

           # sftp dir name
           keylget uargs SFTPDIR dir
           if { ![info exists dir] } {
             error “$module Please set up SFTPDIR”
           }

           # sftp filter
           set filter “*”
           keylget uargs SFTPFILTER filter

           # backup dir
           set backupdir “”
           keylget uargs BACKUPDIR backupdir
           if { ([string length $backupdir] > 0) && ![file isdirectory $backupdir] } {
             error “$module BACKUPDIR ${backupdir} does not exist”
           }

           # local dir name
           keylget uargs LOCALDIR localdir
           if { ![info exists localdir] || ![file isdirectory $localdir] } {
             error “$module Please set up LOCALDIR”
           }

           # tmp dir name
           set tmpdir [file join ${HciRootDir} temp]
           keylget uargs TMPDIR tmpdir
           if { ![file isdirectory $tmpdir] } {
             error “$module Please set up TMPDIR”
           }

           # Expect script name
           keylget uargs EXPECTSN expectSN
           if { ![info exists expectSN] } {
             # Bepaal uniek nummer om bestandsnaam uniek te maken
             set ctrfile “orbis_sftp_in_icr”
             if ![file exists “$ctrfile.ctr”] {
               CtrInitCounter $ctrfile file 1 10000 1
             }
             # Get next counter value
             set number [CtrNextValue $ctrfile file]
             set expectSN “sftp_in_tmpscript_${number}”
           }
           set esn [file join ${tmpdir} ${expectSN}]

           # Haal nu de bestanden via sftp uit de genoemde directory op
           # en zet deze in de localdir.

           # create expect script
           set es [open $esn w]

           # Zet de timeout op 60 seconden
           puts $es “set timeout 60”
           # Start sftp
           puts $es “spawn sftp $user@$host”
           # Wacht op wachtwoord en voer dit in
           puts $es “expect {”
           puts $es ” “password:” {send “$password\r”}”
           puts $es ” “Connection closed” return”
           puts $es “}”
           puts $es “expect {”
           puts $es ” timeout {exit 1}”
           puts $es ” “sftp>””
           puts $es “}”
           # Ga lokaal naar local directory
           puts $es “send “lcd $localdir\r””
           puts $es “expect {”
           puts $es ” timeout {exit 1}”
           puts $es ” “sftp>””
           puts $es “}”
           # Ga remote naar de sftp directory
           puts $es “send “cd $dir\r””
           puts $es “expect {”
           puts $es ” “Couldn’t canonicalise: No such file or directory” {exit 1}”
           puts $es ” timeout {exit 1}”
           puts $es ” “sftp>””
           puts $es “}”
           # Haal de bestanden via de filter op en controleer of ophalen goed is gegaan
           puts $es “send “mget $filter\r””
           puts $es “expect {”
           puts $es ” timeout {exit 1}”
           puts $es ” “sftp>””
           puts $es “}”
           puts $es “send “bye\r””
           puts $es “expect eof”

           flush $es
           close $es
           if {$debug > 1} {
             echo “expect script:[exec cat $esn]”
           }

           # execute expect
           set retVal [exec $HciRootDir/tcl/bin/expect $esn]
           puts “$module retVal: “
           if { [info exists errorCode] } {
               puts “$module errorCode: “
           }
           file delete $esn

           # Verwerken van eventueel zojuist opgehaalde bestanden
           # Bestanden moeten daarna weer verwijderd worden via SFTP; via removelist
           set removelist {}
           # Haal de bestanden op uit de localdir
           # Directories krijgen door de -1 (min

Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands

[Peter Heggie]

that is a cool script! thank you. I’ve never done expect like that, and never used a time mode.

Peter Heggie

[Robert Kersemakers]

First have a look if you can use this on your OS. Expect needs a bit of getting used-to and there are still parts where I think I could have done a better job when catching certain errors.

You need to do this in timed mode, as you want to have Cloverleaf check every x seconds for new files/messages.

Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands

[Grady Stephens]

Have any of you guys any experience using expect sftp with keys?  I cannot get it to work as it telling me it cannot find my private key which is located in /home/hci/.ssh.  It works from the unix command line but not in expect. thanks in advance if you can help.

[Peter Heggie]

I talked to my sales rep and he confirmed it is not a free add-on. With the low number of SFTP interfaces, I will use the ‘expect’ approach.

As far as keys, the only experience I have with SFTP keys is on AIX. The very first time we connect to a new SFTP host, we do it manually at the command line (doing it manually first is always a good practice – thank you everyone who has recommended that method over the years). On AIX, the SFTP package prompts you for saving the key. We respond yes or whatever the actual response is (I forget the exact syntax). The key gets stored somewhere and we never have to worry about it again. We don’t reference it in any way – the OS takes care of it. After that, we can run SFTP in a script.

At the command line, we type:

sftp

I think I remember that this is where the key store prompt is issued. After you respond yes or take the default, then the session continues with the password handshake –

the response is a prompt for a password. After entering the password, the session behaves pretty much like an FTP session.

But I assume that there are many kinds of SFTP implementations so your mileage may vary.

The URL looks something like this:

@

But in the batch script, the format is more like this:

spawn /usr/bin/sftp -oPort= $userid

where userid = the above URL

then we get into the expect commands and look for the password prompt and do a send command with the password and carriage return. etc.

Hope that helps

Peter Heggie

[Keith McLeod]

IF it hasn’t already been mentioned, look for the command autexpect and confirm the top line is correct.

[Matthew Seacat]

We use PERL scripts on our AIX server to send and retrieve files from SFTP servers.  If you are interested in this I can send them to you.  It works really well for us and we just setup cron jobs to run them as often as we need to.

[Author]

Replies