Max message size Is there one ? What is it?

Homepage Clovertech Forums Read Only Archives Cloverleaf Cloverleaf Max message size Is there one ? What is it?

  • Creator
    Topic
  • #50318
    Huy Tran
    Participant

    I am about to create an HL7 interface that will be sending both ASCII and TIFF reports from HPF.

    The vendor is insistanting that Cloverleaf can’t handle messages over 97 K.

    Can someone confirm that for me please? I really don’t believe them. Better to be safe then sorry.

Viewing 15 reply threads
  • Author
    Replies
    • #65611
      Jim Kosloskey
      Participant

      Huy,

      That is not true.

      What is true is the default for some fields in an HL/7 message (independent of Integration tool) is 64K. I some cases that may mean the entire message will not be larger than some number above 64K – maybe 97K for the message in question.

      However, in Cloverleaf(R) inbound messages are not constrained by this field limit as specified in default variants (HL/7). Outbound messages will be – BUT – Cloverleaf allows one to specify a field size INCLUDING INFINITE!

      One thing you do need to be careful about however, is that image messages (which these seem to be) should be encoded (base-64 works just fine) and should be constructed properly following the HL/7 Standard (ED DataType). I am going to bet your vendor has either no clue about the HL/7 standard as it relates to binary type data or deploy it if they are aware of the standard.

      email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

    • #65612
      Dirk Engels
      Participant

      Huy,

      we encounter problems when sending large messages (>50k) with a pdl driver.

      Support told us that there seems to be a limitation in the pdl driver. Better us a file protocol instead.

      Dirk

    • #65613
      James Cobane
      Participant

      We have several interfaces that contain large base-64 encoded .pdf’s within the HL7; we haven’t had any issues with sending these via tcp/ip using the mlp pdl.  These transactions average over 100K.  We did  have to increase the field size in our outbound variant to handle the larger data; so you probably should look there first.  Theoretically, the engine can handle messages up to 4GB in size…..

      Jim Cobane

      Henry Ford Health

    • #65614
      Dirk Engels
      Participant

      Hi Jim,

      that’s curious. We have seen a pdl error from the pdl driver when sending these base-64 encoded pdf via pdl-tcpip. This only happens if the message is bigger then the normal messages and support told us about the possible limitation in the pdl driver.

      Dirk

    • #65615
      James Cobane
      Participant

      Dirk,

      We haven’t had any issues; we have several threads that are passing messages (over 100K in size) between threads using the tcp_acknak.pdl and also threads using the standard mlp_tcp.pdl without issue on messages similar (and larger) in size.  The tcp_acknak.pdl should be able to handle messages up to 4GB in size, as it uses the 4-byte length encoding (in hex) so the max value in the 4-bytes could be FF FF FF FF.  As for the mlp_tcp.pdl, it uses encapsulation so it looks for the beginning and ending characters to define the message.

      Jim Cobane

      Henry Ford Health

    • #65616
      Paul Johnston
      Participant

      Hi Jim C.,

      We are investigating the handle of ” base-64 encoded ” pdf files.

      You mentioned in this old thread you do not have any issues with passing messages over 100K in size.

      Do you know what is your average and maximum message size for the  based-64 encoded files/messages you have encountered.  ?

      Are you aware if there is any lag time in processing these files/ messages . ?

      Is there any OS limitations.  ?

      Thanks Jim

      If anyone else can add their experiences that would be great also.

    • #65617
      James Cobane
      Participant

      I can’t tell you what the max has been, but we have several interfaces that vary in average size; here is a few averages:

      2K

      764K

      99K

      22K

      94K

      Note that the volumes vary as well.  I believe there is an OS limitation (don’t know what it is), so I would assume that Cloverleaf would be limited to that.

      Hope this helps.

      Jim Cobane

      Henry Ford Health

    • #65618
      Alan Fleming
      Participant

      We pass base64 pdf in OBX #5 in QDX 5.4.

      We manually edited that field size to 500K. Our messages are generally over 150K. The main issue is storage of these bloated messages.

    • #65619
      Tom Patton
      Participant

      I have not seen a maximum yet – and I’ve sent some multi-MB (some double digit) files via TCP/IP with pdl driver with an encoded payload in OBX:5.

      And yes, there was a backup while the packets were sent.

      And no, it was never my intent to send files that large.  We have one department that, from time to time, forgets to reduce the picture quality setting on their 14 megapixel camera that they use to attach 5 pictures to the PDF report from….

    • #65620
      Paul Bishop
      Participant

      We average about 46K per message and 129 messages per day for the one vendor that is doing base64 PDFs in an HL7 message.  The largest we ever received was about 130K and cloverleaf handled it with no problem.  We don’t do much with the messages – break out the OBX-5.0 field, decode it, and write the PDF to a file in a directory which is then processed by a batch job.

      We are running QDX 5.6 on an AIX 5.3 box.

      Paul Bishop

      Carle Physician Group

      Urbana, IL

      Paul Bishop
      Carle Foundation Hospital
      Urbana, IL

    • #65621
      David Barr
      Participant

      I think the biggest messages that I’ve sent through Cloverleaf were about 60 megabytes, and the message size limit was based on the memory limit of the engine process which was based on a compile time flag when Cloverleaf was built.

      When you’re working with messages that big you need to avoid calling “msgget” (at least for the entire message; calling it to get chunks of the message is OK).  If you call “msgget” it makes another copy of the message in memory and doubles the memory requirements.

    • #65622
      Paul Johnston
      Participant

      Thanks Jim C and all who responded .

      It is greatly appreciated.

      And David B. Very good point on the “msgget” with the large files. Definitely keep this in mind.

    • #65623
      Kevin Crist
      Participant

      I know this thread is older but it goes with what i am looking for. We are getting ready to send embedded pdf holters through the engine and i’m told they will be 25-30 meg. Is there some issues or anything to watch out for with this size? It makes me a little nervous. The site that these are in have the messages saved for 5 days, wondering if we need to shorten that. I was told our size is 20 gb.

      Thanks.

    • #65624
      Jim Kosloskey
      Participant

      Kevin,

      You should be able to calculate that by multiplying your average message size by your average daily message count by 5 days.

      I would then allow at least 20% head room for expansion.

      That will tell you if you have sufficient archive space. Don’t forget you will need sufficient space for your current SMAT files and you logs will increase whenever you need to turn the EO up to see messages.

      email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

    • #65625
      Richard Hart
      Participant

      Hi Kevin.

      We work with PDF documents, although ours are tiny!

      When a document enters Cloverleaf, we strip it out of the message and replace it with a link.  When the document is sent out, we put it back in again.

      This way, we don’t have documents being passed though the engine.

      We have had 30mb documents sent to Cloverleaf – in error – so these can be handled.

    • #65626
      Kevin Crist
      Participant

      Thanks. Out of curiosity how do you store your file and pick it up with the link?

Viewing 15 reply threads
  • The forum ‘Cloverleaf’ is closed to new topics and replies.

Forum Statistics

Registered Users
5,126
Forums
28
Topics
9,296
Replies
34,439
Topic Tags
287
Empty Topic Tags
10