.hl7 file type

  • Creator
    Topic
  • #54792
    Christy Stepp
    Participant

    I am trying to use fileset-ftp protocol in Cloverleaf to pick up a file named filename.hl7.  The file contains one HL7 ORU message with between each segment and at the end of the file.  I have tried multiple settings for the fileset-ftp properities and Cloverleaf will not read the file.

    We are using UNIX, so I tried to look at the file in Cloverleaf.

    When I type the more command with the filename I can see the message, it starts with MSH, and appears to be a complete HL7 message.  If I type the vi command with the filename the file appears as one @ per line with no other data.

    Does anyone have any experience with the .hl7 file type?  The fileset-ftp protocol works for me in other fileset-ftp threads using .txt file types.

Viewing 19 reply threads
  • Author
    Replies
    • #83007
      Robert Kersemakers
      Participant

      Hi Christy,

      The way you describe the file, it should work with fileset-ftp. Just make sure you are reading it ‘single’ style.

      Have you turned up the EO config to have some more logging on what is happening? And you say Cloverleaf will not read the file: is there an error given, or is the file just not picked up by ftp? The ‘Dir List Command’ is normally ‘nlst’, but if it’s something like ‘nlst *.txt’ it will only pick up *.txt files.

      Your log file (especially with your EO config turned up) should indicate what is happening.

      Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands

    • #83008
      Russ Ross
      Participant

      If the file is new-line terminated the setting fileset-ftp “nl” I would expect to worked.

      However, once I was told my single message file was new-line terminated and it wasn’t, so I ended up using eof to get it to work.

      I haven’t used the setting “single” which might have the same outcome.

      If you do hcihd on the file in question you can see if it is new-line terminated, because you can’t tell in vi when there is only one message in the file.

      You might also be able to use “wc” on the file and if number of lines is 0 then it isn’t new-line terminated.

      Today I noticed for the first time a setting called “hl7” which I have no experience with, but wonder how that setting will consume a file, especially those HL7 files with one segment per line.

      Russ Ross
      RussRoss318@gmail.com

    • #83009
      Christy Stepp
      Participant

      I was using style of hl7, and I also tried single.  The error in the thread is FTP operation failed: RETR response: 550.  I can see all of the files that the ftp parse tcl scanned in the directory, and I can see that it found 3 files that matched my regular expression.  I think error 550 is “cannot open file” (from google search!)  Not sure why Cloverleaf cannot open the files…

    • #83010
      Charlie Bursell
      Participant

      Have you triedthe same thing from the command line?

    • #83011
      James Cobane
      Participant

      Does the file have appropriate privileges for the ‘hci’ user to read/write from it?  Perhaps the user/process that is placing the file has different read/write privileges.

      Jim Cobane

      Henry Ford Health

    • #83012
      Christy Stepp
      Participant

      I can manually FTP the file from the command line.  I have tried viewing the file with the vi command, it shows one @ per line, no other data.  If I save the file as a .txt file Cloverleaf will read it using the nl style setting.  I would like to know why Cloverleaf will not read it as a .hl7 file using the hl7 style or single style.

    • #83013
      Russ Ross
      Participant

      I think I recall seeing one @ per line on files that have very long lines and can’t all be displayed in my vi view window.

      Still would be curious to run hcihd on the original .hl7 file to know what is in it, since an editor like vi will not give you the complete picture.

      When you say “saved the file as .txt” did you save the file using an editor like vi or “cp -p” from the command line or “mv” from the command line because they aren’t the same.

      So when you do “diff” and “ls -l” of your .hl7 file and .txt file, are they are identical and the permissions are the same, too?

      If so, that by itself is a head scratcher.

      If you rename the .txt file that works back to .hl7 what happens?

      I’m struggling to imagine the file extension would change the outcome of the fileset-ftp if the file contents, ownership, and permissions are identical.

      Russ Ross
      RussRoss318@gmail.com

    • #83014
      Christy Stepp
      Participant

      I can see the content of the file if I use the hcihd command.  I saved the file as .txt by opening it in NotePad++ and using the save as text command. The diff command shows no difference in the two files.

    • #83015
      Robert Milfajt
      Participant

      vi is showing @ because message is too long and vi has limitations.  I like to either use cat -v to show control characters or od -c (which is a file dump) showing characters, od -x shows hex values.

      Once you can look at the dump of the file, you might have better data on why Cloverleaf is failing to read it.

      Hope this helps,

      Robert Milfajt
      Northwestern Medicine
      Chicago, IL

    • #83016
      Christy Stepp
      Participant

      I was able to get Cloverleaf to read the files.  I have the style setting as single and the data type setting as Image/Binary.  Thanks to everyone for all of your help.

    • #83017
      Jim Kosloskey
      Participant

      Christy,

      Does this file have RTF or PDF imbedded in the messages?

      email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

    • #83018
      Christy Stepp
      Participant

      New problem.

      The Fileset-FTP thread passes a single filename.hl7 file from the directory to the Xlate and it processes fine, as long as there is only one .hl7 file in the directory.

      But if there are 2 files in the directory with .hl7 extensions I get an error:  550 filename.hl7 filename2.hl7: not a plain file.  I also get 550 filename.hl7 filename2.hl7: A file or directory in the path name does not exist.

      I was expecting each message to process in the order they were in the directory.  The log file shows that the ftp_dir_parse tcl is passing back  new_msg{filename.hl7 filename2.hl7}  Then I see the errors from the ftp process.

      Each file contains a single message.

      Thread set as fileset-ftp

      Style: single

      Directory Parse: ftp_dir_parse (passing in parameters {OP FETCH} {TYPE REGEXP} {FILES .*.hl7} {DEBUG 9} )

      Scheduling

      Read Interval: 5

      Scan Interval: 30

      Max Messages: 1

    • #83019
      Russ Ross
      Participant

      I wonder if setting messages to a higher number than one could be helpful and certainly not likely to hurt anything.

      I keep mine to a rate of below 4,000 per minute so as to give cloverleaf 15 seconds of rest every minute on my platform setup and for slower integrations it might be as low as 200 per minute when I don’t want to overwhelm a slow downstream interface.

      With such a frequent directory sweep, I wonder if the second file is still being written to when Cloverleaf attempts to retrieve it and might be causing the file not found error.

      Make sure the .hl7 file(s) are ready for consumption and not being written to when the directory sweep occurs.

      The simplest effect way to handle this I’ve run across so far is to have the file names whatever.hl7.pending while being written to and then have the source application rename them to whatever.hl7.ready or just whatever.hl7 to put Cloverleaf on notice the file creation has been completed.

      Russ Ross
      RussRoss318@gmail.com

    • #83020
      Jim Kosloskey
      Participant

      Could it be that both file names are being presented as one:

      550 filename.hl7 filename2.h7

      Perhaps your dirparse routine is not returning a proper list.

      email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

    • #83021
      Russ Ross
      Participant

      Understanding what typically gets change when transferring in ASCII mode is helpful when determining is ASCII or binary FTP transfer mode is needed.

      Different operating system use different line termination sequence and when transferring in ASCII mode those line terminators get convert.

      So on Microsoft Windows or DOS a line terminator is , while on a UNIX platform the line terminator is .

      Open a file up in an editor for viewing is okay but then saving the file can also super impose such undesired intelligence along with sending message through email and trying to use them.

      hcihd is your fried or an editor that has a hex edit mode will give you the true story.

      Since the hl7 segment termination character is a this can get in the way of a FTP ASCII transfer and might require a binary transfer.

      Other times the source system might have one segment per line and some message formatting to combine back to one line is a common approach I use before sending them to xlate.

      So when dealing with a new FTP integration I hcihd to see what I’m dealing with and that helps determine what approach I go with.

      I could ask the vendor to make it right but typically take what I get and preprocess it before xlate if I get some variation but at least with hcihd I will be aware of what I’m starting with and knowing what I need it to be I just close the gap, which sometimes I accomplish with a binary transfer like you did.

      I always do a manual FTP binary FTP the first time so I will be able to see with hcihd what I’m starting with.

      Russ Ross
      RussRoss318@gmail.com

    • #83022
      Russ Ross
      Participant

      Jim wrote:

      Could it be that both file names are being presented as one:

      550 filename.hl7 filename2.h7

      Perhaps your dirparse routine is not returning a proper list.

      So my question is are you using a dir parse routine or just letting Cloverleaf do the default built in dir parsing?

      Russ Ross
      RussRoss318@gmail.com

    • #83023
      Christy Stepp
      Participant

      If TCL ftp_dir_parse returns one file it works; if it returns 2 files it does not work – it actually passes a blank message to the Xlate.  

      When there is one .hl7 file in the directory

      process log:  ftp_dir: test_ftp_oru: new_msg|filename.hl7|

      msgDriverControl: {FILENAME /home/hci/temp/filename.hl7}

      msg: MESSAGE IS HERE

      When there are two .hl7 files in the directory

      process log: ftp_dir: test_ftp_oru: new_msg|{filename.hl7 filename2.hl7}

      msgDriverControl: {FILENAME /home/hci/temp/filename.hl7 filename2.hl7}

      msg: ‘ ‘

      I thought this was correct.  Shouldn’t it pass the first message in the list, process it, then pass the next message in the list and process it?

    • #83024
      Russ Ross
      Participant

      Okay sounds like your dir parse routine needs a fix to properly handle multiple files.

      Jim Kosloskey on our team has done all the dir parse work which we rarely need beyond the builtin cloverleaf dirparse so I’m not current on what dir parse needs to return.

      Since Jim asked if your dir pase was returning one item with 2 files, I imagine dir parse needs to return a list of files names when multiple files exist.

      I also recall the directory information returned to dirparse changed when we went from cloverleaf 5.6 to Cloverleaf 6.0 with expanded listing information that broke dir parse for us until Jim remediated.

      I’m sure others can chime in on if my guess is correct that dir parse needs to return a list of filenames when multiple files need to be processed.

      It is a common mistake to just concatenate things together separated by a space instead of using the list commands to create a list of items to return.

      Russ Ross
      RussRoss318@gmail.com

    • #83025
      Christy Stepp
      Participant

      The TCL is returning new_msg|{filename.hl7 filename2.hl7} which I think is a list.

    • #83026
      Jim Kosloskey
      Participant

      Have you echoed out whath the  list looks like you are receiving before you modify it?

      email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

Viewing 19 reply threads
  • The forum ‘Cloverleaf’ is closed to new topics and replies.

Forum Statistics

Registered Users
5,117
Forums
28
Topics
9,293
Replies
34,435
Topic Tags
286
Empty Topic Tags
10