Max message size Is there one ? What is it?

[Huy Tran]

I am about to create an HL7 interface that will be sending both ASCII and TIFF reports from HPF.

The vendor is insistanting that Cloverleaf can’t handle messages over 97 K.

Can someone confirm that for me please? I really don’t believe them. Better to be safe then sorry.

Topic

[Jim Kosloskey]

Huy,

That is not true.

What is true is the default for some fields in an HL/7 message (independent of Integration tool) is 64K. I some cases that may mean the entire message will not be larger than some number above 64K – maybe 97K for the message in question.

However, in Cloverleaf(R) inbound messages are not constrained by this field limit as specified in default variants (HL/7). Outbound messages will be – BUT – Cloverleaf allows one to specify a field size INCLUDING INFINITE!

One thing you do need to be careful about however, is that image messages (which these seem to be) should be encoded (base-64 works just fine) and should be constructed properly following the HL/7 Standard (ED DataType). I am going to bet your vendor has either no clue about the HL/7 standard as it relates to binary type data or deploy it if they are aware of the standard.

email: jim.kosloskey@jim-kosloskey.com

[Dirk Engels]

Huy,

we encounter problems when sending large messages (>50k) with a pdl driver.

Support told us that there seems to be a limitation in the pdl driver. Better us a file protocol instead.

Dirk

[James Cobane]

We have several interfaces that contain large base-64 encoded .pdf’s within the HL7; we haven’t had any issues with sending these via tcp/ip using the mlp pdl.  These transactions average over 100K.  We did  have to increase the field size in our outbound variant to handle the larger data; so you probably should look there first.  Theoretically, the engine can handle messages up to 4GB in size…..

Jim Cobane

Henry Ford Health

[Dirk Engels]

Hi Jim,

that’s curious. We have seen a pdl error from the pdl driver when sending these base-64 encoded pdf via pdl-tcpip. This only happens if the message is bigger then the normal messages and support told us about the possible limitation in the pdl driver.

Dirk

[James Cobane]

Dirk,

We haven’t had any issues; we have several threads that are passing messages (over 100K in size) between threads using the tcp_acknak.pdl and also threads using the standard mlp_tcp.pdl without issue on messages similar (and larger) in size.  The tcp_acknak.pdl should be able to handle messages up to 4GB in size, as it uses the 4-byte length encoding (in hex) so the max value in the 4-bytes could be FF FF FF FF.  As for the mlp_tcp.pdl, it uses encapsulation so it looks for the beginning and ending characters to define the message.

Jim Cobane

Henry Ford Health

[Paul Johnston]

Hi Jim C.,

We are investigating the handle of ” base-64 encoded ” pdf files.

You mentioned in this old thread you do not have any issues with passing messages over 100K in size.

Do you know what is your average and maximum message size for the  based-64 encoded files/messages you have encountered.  ?

Are you aware if there is any lag time in processing these files/ messages . ?

Is there any OS limitations.  ?

Thanks Jim

If anyone else can add their experiences that would be great also.

[James Cobane]

I can’t tell you what the max has been, but we have several interfaces that vary in average size; here is a few averages:

2K

764K

99K

22K

94K

Note that the volumes vary as well.  I believe there is an OS limitation (don’t know what it is), so I would assume that Cloverleaf would be limited to that.

Hope this helps.

Jim Cobane

Henry Ford Health

[Alan Fleming]

We pass base64 pdf in OBX #5 in QDX 5.4.

We manually edited that field size to 500K. Our messages are generally over 150K. The main issue is storage of these bloated messages.

[Tom Patton]

I have not seen a maximum yet – and I’ve sent some multi-MB (some double digit) files via TCP/IP with pdl driver with an encoded payload in OBX:5.

And yes, there was a backup while the packets were sent.

And no, it was never my intent to send files that large.  We have one department that, from time to time, forgets to reduce the picture quality setting on their 14 megapixel camera that they use to attach 5 pictures to the PDF report from….

[Paul Bishop]

We average about 46K per message and 129 messages per day for the one vendor that is doing base64 PDFs in an HL7 message.  The largest we ever received was about 130K and cloverleaf handled it with no problem.  We don’t do much with the messages – break out the OBX-5.0 field, decode it, and write the PDF to a file in a directory which is then processed by a batch job.

We are running QDX 5.6 on an AIX 5.3 box.

Paul Bishop

Carle Physician Group

Urbana, IL

Paul Bishop
Carle Foundation Hospital
Urbana, IL

[David Barr]

I think the biggest messages that I’ve sent through Cloverleaf were about 60 megabytes, and the message size limit was based on the memory limit of the engine process which was based on a compile time flag when Cloverleaf was built.

When you’re working with messages that big you need to avoid calling “msgget” (at least for the entire message; calling it to get chunks of the message is OK).  If you call “msgget” it makes another copy of the message in memory and doubles the memory requirements.

[Paul Johnston]

Thanks Jim C and all who responded .

It is greatly appreciated.

And David B. Very good point on the “msgget” with the large files. Definitely keep this in mind.

[Kevin Crist]

I know this thread is older but it goes with what i am looking for. We are getting ready to send embedded pdf holters through the engine and i’m told they will be 25-30 meg. Is there some issues or anything to watch out for with this size? It makes me a little nervous. The site that these are in have the messages saved for 5 days, wondering if we need to shorten that. I was told our size is 20 gb.

Thanks.

[Jim Kosloskey]

Kevin,

You should be able to calculate that by multiplying your average message size by your average daily message count by 5 days.

I would then allow at least 20% head room for expansion.

That will tell you if you have sufficient archive space. Don’t forget you will need sufficient space for your current SMAT files and you logs will increase whenever you need to turn the EO up to see messages.

email: jim.kosloskey@jim-kosloskey.com

[Richard Hart]

Hi Kevin.

We work with PDF documents, although ours are tiny!

When a document enters Cloverleaf, we strip it out of the message and replace it with a link.  When the document is sent out, we put it back in again.

This way, we don’t have documents being passed though the engine.

We have had 30mb documents sent to Cloverleaf – in error – so these can be handled.

[Kevin Crist]

Thanks. Out of curiosity how do you store your file and pick it up with the link?

[Author]

Replies