Using CURL to POST data to webservice

[Rajesh Purushothaman]

I need some help from the Clovertech Guru’s –

This is what I am trying to accomplish.

1. Make a call from within Cloverleaf to a web service first to get the ‘Security Token’.

2. Once I get the token, use the token to send data from Cloverleaf to that web service in JSON message format.

I am now trying to get the first step to work. This is the command that I am trying from command line after assigning values to the variables.

curl -v –cacert cacert.pem

POST -H –URL $URL

-H ‘Content-Type: application/x-www-form-urlencoded’

-d ‘grant_ENGINE=password’

-d ‘client_id=$clientID’

-d ‘client_secret=$clientSECRET’

-d ‘username=$USERNAME’

-d ‘password=$PASSWORD’

The vendor web service receives this request but then it fails to process. The error message that is being logged is ‘unable to process request from “unsecure channel”‘.

I am not supplying any SSL certificate from client side (Cloverleaf).

I am not sure how to generate one.

Has anyone done similar things and if so can you guide me in how to do this? I am planning to read about curl::transfer.

Appreciate any help in advance. Thanks.

Topic

[Jim Kosloskey]

There is a Cloverleaf Add-on Secure Messenger which is available for using SSL based protocols (TCP/IP, Fileset/FTP, HTTP).

There is also CAA_WS Add-on specifically for Web Service Interaction.

Of course, if you are not already licensed for the above, there is a fee.

There may be Tcl specific ways to accomplish what you want as well but I am not well versed in them.

email: jim.kosloskey@jim-kosloskey.com 30+ years Cloverleaf, 60 years IT – old fart.

[Rajesh Purushothaman]

Jim

We dont have license for Add-on Secure Messenger.

That is the reason why I am trying to do this using TclCurl/curl

[Levy Lazarre]

If you use the –cacert option with your curl command, then you have to supply a client certificate filename with it.

Many Authentication APIs allow users to pass in credentials (username, password) in order to receive an authentication token. You are already doing this in your curl command, so you shouldn’t need a client certificate. You should therefore try the curl call without the –cacert option and see what is returned to you.

Find out from the API documentation if a client certificate is really needed. If so, you have to send a request for a signed client certificate to the vendor. You cannot generate your own certificate for this purpose because it will never authenticate at the vendor site.

[Rajesh Purushothaman]

Levy

Thanks for your reply. I downloaded the cacert.perm file from the CURL website and when I remove that option and execute the curl command, I get the following warning message

* Server certificate:

*        subject: C=US; ST=California; L=San Francisco; O=Salesforce.com, Inc; OU=Applications; CN=test.salesforce.com

*        start date: 2015-06-05 00:00:00 GMT

*        expire date: 2018-06-04 23:59:59 GMT

*        issuer: C=US; O=Symantec Corporation; OU=Symantec Trust Network; CN=Symantec Class 3 Secure Server CA – G4

*        SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.

There is no API documentation from the vendor. All the requirement was made in a single email  🙄 .

I have been asking him the question saying if we need to include a ‘signed’ client certificate when requesting for the token, the answer is a big NO. I have sent another email asking him to confirm the same question again. I will keep you posted once I get a response back.

This is what I have tried so far with curl::transfer

package require TclCurl

if {[catch {[curl::transfer -url https://test.salesforce.com/services/oauth2/token?

-verbose 1

-post 1

-sslverifypeer 0

-bodyvar body

-postfields “grant_ENGINE=password&client_id=xxx&client_secret=yyy&username=aaa&password=bbb”

]} ret_cd]} {

echo “return code from WebService Call is: $ret_cd”

echo “body variable output is $body”

}

This is the output I got

* Hostname was NOT found in DNS cache

*   Trying 96.43.146.125…

* Connected to test.salesforce.com (96.43.146.125) port 443 (#0)

* SSL connection using TLSv1.0 / AES256-SHA

* Server certificate:

*        subject: C=US; ST=California; L=San Francisco; O=Salesforce.com, Inc; OU=Applications; CN=test.salesforce.com

*        start date: 2015-06-05 00:00:00 GMT

*        expire date: 2018-06-04 23:59:59 GMT

*        subjectAltName: test.salesforce.com matched

*        issuer: C=US; O=Symantec Corporation; OU=Symantec Trust Network; CN=Symantec Class 3 Secure Server CA – G4

*        SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.

> POST /services/oauth2/token? HTTP/1.1

Host: test.salesforce.com

Accept: */*

Content-Length: 251

Content-Type: application/x-www-form-urlencoded

* upload completely sent off: 251 out of 251 bytes

< HTTP/1.1 400 Bad Request

< Date: Fri, 07 Apr 2017 16:44:24 GMT

< Strict-Transport-Security: max-age=31536000; includeSubDomains

< X-Content-Type-Options: nosniff

< X-XSS-Protection: 1; mode=block

< Content-Security-Policy: reflected-xss block;report-uri /_/ContentDomainCSPNoAuth?ENGINE=xss

< Content-Security-Policy: referrer origin-when-cross-origin

< Set-Cookie: BrowserId=GNEuKMgvRbKSybot63DZeA;Path=/;Domain=.salesforce.com;Expires=Tue, 06-Jun-2017 16:44:24 GMT

< Expires: Thu, 01 Jan 1970 00:00:00 GMT

< Pragma: no-cache

< Cache-Control: no-cache, no-store

< X-ReadOnlyMode: false

< Content-Type: application/json;charset=UTF-8

< Transfer-Encoding: chunked

<

* Connection #0 to host test.salesforce.com left intact

return code from WebService Call is: invalid command name “0”

body variable output is {“error”:”invalid_grant”,”error_description”:”authentication failure”}

tcl>

[Levy Lazarre]

The vendor is correct, you do not need a certificate.

Code:

{”error”:”invalid_grant”,”error_description”:”authentication failure”}

-postfields “grant_ENGINE=password&client_id=xxx&client_secret=yyy&username=aaa&password=bbb”

So now you have a parsing problem with the -postfields data. The web server is not correctly interpreting the ‘&’ and therefore is not recognizing the grant_type and user credentials you are passing, hence the errors.

The documentation for TclCurl / -postfields specifies: “You must make sure that the data is formatted the way you want the server to receive it. TclCurl will not convert or encode it for you.  Most web servers will assume this data to be URL-encoded.”

So you should try to replace each ‘&’ in the string by its URL-encoded value: %26

[Rajesh Purushothaman]

Hi,

I replaced the & to %26 and ! (found in password string) to %21, I got the error message as {‘”error”:”unsupported_grant_type”,”error_description”:”grant type not supported”}

I was getting

{“error”:”invalid_grant”,”error_description”:”authentication failure”}

Vendor executed the following Curl command in his machine and he was able to connect.

curl -v POST -H –URL ‘https://test.salesforce.com/services/oauth2/token&#8217; -H ‘Content-Type: application/x-www-form-urlencoded’ -d ‘grant_ENGINE=password&client_id=&client_secret=&username=&password=’

He is seeing my request (executing the Curl command from Cloverleaf server) making it to the server but its getting rejected with the error message ‘”Failed: Login over insecure channel”

[Rajesh Purushothaman]

I just wanted to post an update.

This is the version of the CURL we have in our server

Code:

curl 7.37.0 (powerpc-ibm-aix6.1.0.0) libcurl/7.37.0 OpenSSL/1.0.0g zlib/1.2.3 libssh2/1.4.3
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: IPv6 Largefile NTLM NTLM_WB SSL libz

This version of OpenSSL makes the connection using TLS 1.0

The vendor doesnt support TLS 1.0

I was developing a .Net version of this code and I force the connection to be made using TLS 1.1 and I was able to connect to the vendor.

It looks like TLS 1.1 and TLS 1.2 are supported only from OpenSSL version 1.0.1

This means we need to upgrade. I am not sure if this will get approved by Jim Cobane  😛

[Charlie Bursell]

Take a look here and see if it helps.

https://developer.salesforce.com/forums/?id=906F0000000BCs9IAG

Also you really need to get an HTTP analyzer like ieinspector, etc.

I always used something like this so I could see the traffic back and forwards via a browser then emulate that with cURL

[Rajesh Purushothaman]

Thanks for the link Charlie.

I verified that I am passing the parameters in the same fashion as described in the link.

I need to check to see if we can either upgrade the OpenSSL or if possible just try to install a patch so that our current version of CURL supports TLS 1.1

I will read more about ieInspector, I didnt know about this. It looks more like a Windows based tool which can be used for Analyzing HTTP request and response.

Since I was little familiar with .Net programming I was using it to check if my HTTP Post request format is correct.

[Author]

Replies