Help! Upgrade to 5.7 fileset-ftp — no newlines b/w msgs?

This topic has 11 replies, 5 voices, and was last updated 15 years, 12 months ago by Jim Kosloskey.

Creator

Topic
April 15, 2010 at 7:58 pm #51696
Jennifer Hardesty
Participant
We are in the process of upgrading from 5.4 to 5.7. I have set up one of our batch billing ftp processes exactly as it is set up in 5.4.

However, once the messages begin processing, the ones in a route that are checked in a tclproc for the correct length account # all get killed. The messages that go through the route that just has a translator make it to the next thread fine.

When I compare the SMAT files, the 5.7 inbound SMAT has no x0d and no x0dx0a (newline) characters. I have the inbound style set to “nl” for both 5.4 and 5.7…and these batch billing files worked perfectly fine in Production last night so I know it’s not something wrong with them.

Is there a setting we missed somewhere in 5.7?
Creator

Topic

Viewing 10 reply threads

Author

Replies
- April 19, 2010 at 2:55 pm #71322
  Sergey Sevastyanov
  Participant
  Jennifer,
  
  Do you send to the same FTP server from your new CL 5.7?
  
  We switched to 5.7 from 5.4 over a year ago and I don’t remember running into any problems with FTP threads.
  
  But several months ago we also switched to a different FTP server and we had problems related to the way the new server treats end of lines. I couldn’t fine any details of what exactly happened but I do remember that I had to change ASCII mode to Binary in several occasions to preserve end of lines in transferred files. Again, it wasn’t Cloverleaf related problem but rather the new FTP server, which runs a different FTP service (in fact it runs different OS).
- April 19, 2010 at 3:57 pm #71323
  Jennifer Hardesty
  Participant
  It’s the same servers. The only change is the upgrade.
- April 19, 2010 at 9:22 pm #71324
  Michael Hertel
  Participant
  Cloverleaf’s method of using ftp is different between 5.4 and 5.7
  
  They added Tclcurl.
  
  You probably want to start from scratch to create this interface instead of relying on what you had.
  
  We stayed on 5.4.1 for this reason.
- April 20, 2010 at 2:14 pm #71325
  Jennifer Hardesty
  Participant
  Well, I did delete all of the ftp processes and folders and begin rebuilding them from scratch because we noticed that some of the settings in 5.4 were carried over to 5.7 but weren’t editable or even seen in 5.7 and were causing errors.
  
  However, I’ve read all of the documentation. It doesn’t appear to require much different set up, certainly nothing that I can figure out that should be stripping out all of the newlines and other hex characters.
- April 20, 2010 at 4:16 pm #71326
  Michael Hertel
  Participant
  I never trusted the ascii/binary setting on past Cloverleaf versions.
  
  Do the 50/50 rule. If it doesn’t work as ascii, try binary.
  
  Also check any tclprocs that might be compensating.
  
  Obviously the end result is to get it to work.
- April 21, 2010 at 2:37 pm #71327
  Jennifer Hardesty
  Participant
  Well, that is just bizarre. I have switched the Data Type on the FTP Options tab from ASCII to Image/Binary.
  
  Here is what showed up in the log — it looks to me like it’s still using ASCII:
  
  Code: * About to connect() to 172.xx.x.xx port 21 (#0) * Trying 172.xx.x.xx… * connected * Connected to 172.xx.1.16 (172.xx.x.xx) port 21 (#0) < 220-Microsoft FTP Service USER xxxxxxxxxxxx PASS XXXXXXXX < 230-You are connected to the Maine Medical Center FTP site. PWD CWD /impac/TestFiles EPSV * Connect data stream passively PASV TYPE A nlst < 125 Data connection already open; Transfer starting. * Maxdownload = -1 * Remembering we are in dir "/impac/TestFiles/" < 226 Transfer complete. * Connection #0 to host 172.xx.x.xx left intact [pd :pdtd:INFO/1: fr_impac_ft:04/21/2010 09:21:21] Set driver status to PD_STATUS_UP [tps :tps :INFO/1: fr_impac_ft:04/21/2010 09:21:21] tds.string = tps_ftp_impac_dft {MSGID message0} {CONTEXT fileset_ibdirparse} {ARGS {}} {MODE run} {VERSION 3.0} ***
  
  Then, the files were picked up, read in properly, translated and written out just like they currently are in 5.4.
  
  While it’s nice that one is working, I don’t know why and I can’t be sure it’ll work that way when we move it into production. I’m setting up a test for another ftp batch billing site to see if this is a consistent thing, but this feels very wrong.
- April 21, 2010 at 3:31 pm #71328
  Michael Hertel
  Participant
  I agree.
- April 21, 2010 at 7:14 pm #71329
  Bob Schmid
  Participant
  Rather than trying to play the shell game….shouldnt lawson support be able to confirm whether this is broken?
- April 27, 2010 at 1:40 am #71330
  Jennifer Hardesty
  Participant
  I spoke to tech support today. They said that the setting should be binary and that it should have always been binary, even in 5.4. According to tech support, ASCII would have stripped my newlines and binary keeps them, as opposed to the opposite.
  
  That’s fine and dandy but later I got to studying the log and I really, really don’t think Cloverleaf is recognizing the checkboxes or radio buttons in the FTP Options box on that second tab. I have it set to delay until needed and close after; however, as soon as I start the process, it immediately attempts to ftp login and then maintains that quiet keepalive connection until it’s needed; if the connection is broken, Cloverleaf won’t know it and it will get stuck when the scheduled time kicks off.
  
  Honestly, even on 5.4, we’ve never been able to get those two options to work on inbound or outbound processes.
  
  Plus, the log still shows that the connection is “Type A” though the files I pick up have all of the x0ds and x0dx0as in all of the right places.
  
  If I had a more trustworthy, consistent, ftp protocol, I would use it.
- April 27, 2010 at 12:35 pm #71331
  Michael Hertel
  Participant
  I couldn’t agree more.
  
  I quit fighting with ftp a long time ago.
  
  It got to the point that I just compensated with tcl to add/remove
  
  the extra cr/lf’s. The bottom line was that I had to get the interface working.
  
  You would think that by now ftp and batch file processing would work flawlessly and naitively. I suppose it’s more fun to get soap and xml working.
- April 27, 2010 at 2:12 pm #71332
  Jim Kosloskey
  Participant
  Jennifer and Michael,
  
  Cloverleaf 5.6.
  
  For the United Health Care X12 278 project, I needed to FTPS from UHC.
  
  I used the native Cloverleaf protocol and while we were in use (we stopped the project after a couple of months for data issues not FTP issues) I did not have the issues with either Binary mode or ‘Delay Connection until needed’.
  
  The Binary mode was always set properly as I recall.
  
  As far as the ‘Delay Connection Until Needed’ I think that is only effective when FTPng to a destination not when Getting from a Server.
  
  So when attempting to Get, the Connection is attempted immediately upon startup as I recall and any available files at that time are presented. Once that set is exhausted, then at the next scan interval (or advanced scheduling trigger) the connection and scanning is done again.
  
  While I have yet to do a FTP Put on Cloverleaf 56. I expect the actual connection will be delayed until there is something to put should I have that turned on.
  
  My experience with the FTP protocol on Cloverelaf 5.6 was not problematic.
  
  email: jim.kosloskey@jim-kosloskey.com 30+ years Cloverleaf, 61 years IT – old fart.
Author

Replies

Viewing 10 reply threads

The forum ‘Cloverleaf’ is closed to new topics and replies.