Homepage › Clovertech Forums › Read Only Archives › Cloverleaf › Cloverleaf › Max message size Is there one ? What is it?
- This topic has 16 replies, 11 voices, and was last updated 12 years, 7 months ago by Kevin Crist.
-
CreatorTopic
-
September 25, 2008 at 7:33 pm #50318Huy TranParticipant
I am about to create an HL7 interface that will be sending both ASCII and TIFF reports from HPF. The vendor is insistanting that Cloverleaf can’t handle messages over 97 K.
Can someone confirm that for me please? I really don’t believe them. Better to be safe then sorry.
-
CreatorTopic
-
AuthorReplies
-
-
September 25, 2008 at 7:55 pm #65611Jim KosloskeyParticipant
Huy, That is not true.
What is true is the default for some fields in an HL/7 message (independent of Integration tool) is 64K. I some cases that may mean the entire message will not be larger than some number above 64K – maybe 97K for the message in question.
However, in Cloverleaf(R) inbound messages are not constrained by this field limit as specified in default variants (HL/7). Outbound messages will be – BUT – Cloverleaf allows one to specify a field size INCLUDING INFINITE!
One thing you do need to be careful about however, is that image messages (which these seem to be) should be encoded (base-64 works just fine) and should be constructed properly following the HL/7 Standard (ED DataType). I am going to bet your vendor has either no clue about the HL/7 standard as it relates to binary type data or deploy it if they are aware of the standard.
email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.
-
September 26, 2008 at 5:16 am #65612Dirk EngelsParticipant
Huy, we encounter problems when sending large messages (>50k) with a pdl driver.
Support told us that there seems to be a limitation in the pdl driver. Better us a file protocol instead.
Dirk
-
September 26, 2008 at 12:34 pm #65613James CobaneParticipant
We have several interfaces that contain large base-64 encoded .pdf’s within the HL7; we haven’t had any issues with sending these via tcp/ip using the mlp pdl. These transactions average over 100K. We did have to increase the field size in our outbound variant to handle the larger data; so you probably should look there first. Theoretically, the engine can handle messages up to 4GB in size….. Jim Cobane
Henry Ford Health
-
September 28, 2008 at 8:13 am #65614Dirk EngelsParticipant
Hi Jim, that’s curious. We have seen a pdl error from the pdl driver when sending these base-64 encoded pdf via pdl-tcpip. This only happens if the message is bigger then the normal messages and support told us about the possible limitation in the pdl driver.
Dirk
-
September 29, 2008 at 2:19 pm #65615James CobaneParticipant
Dirk, We haven’t had any issues; we have several threads that are passing messages (over 100K in size) between threads using the tcp_acknak.pdl and also threads using the standard mlp_tcp.pdl without issue on messages similar (and larger) in size. The tcp_acknak.pdl should be able to handle messages up to 4GB in size, as it uses the 4-byte length encoding (in hex) so the max value in the 4-bytes could be FF FF FF FF. As for the mlp_tcp.pdl, it uses encapsulation so it looks for the beginning and ending characters to define the message.
Jim Cobane
Henry Ford Health
-
April 26, 2011 at 2:59 pm #65616Paul JohnstonParticipant
Hi Jim C.,
We are investigating the handle of ” base-64 encoded ” pdf files.
You mentioned in this old thread you do not have any issues with passing messages over 100K in size.
Do you know what is your average and maximum message size for the based-64 encoded files/messages you have encountered. ?
Are you aware if there is any lag time in processing these files/ messages . ?
Is there any OS limitations. ?
Thanks Jim
If anyone else can add their experiences that would be great also.
-
April 26, 2011 at 3:29 pm #65617James CobaneParticipant
I can’t tell you what the max has been, but we have several interfaces that vary in average size; here is a few averages:
2K
764K
99K
22K
94K
Note that the volumes vary as well. I believe there is an OS limitation (don’t know what it is), so I would assume that Cloverleaf would be limited to that.
Hope this helps.
Jim Cobane
Henry Ford Health
-
April 26, 2011 at 5:23 pm #65618Alan FlemingParticipant
We pass base64 pdf in OBX #5 in QDX 5.4.
We manually edited that field size to 500K. Our messages are generally over 150K. The main issue is storage of these bloated messages.
-
April 26, 2011 at 9:49 pm #65619Tom PattonParticipant
I have not seen a maximum yet – and I’ve sent some multi-MB (some double digit) files via TCP/IP with pdl driver with an encoded payload in OBX:5.
And yes, there was a backup while the packets were sent.
And no, it was never my intent to send files that large. We have one department that, from time to time, forgets to reduce the picture quality setting on their 14 megapixel camera that they use to attach 5 pictures to the PDF report from….
-
April 26, 2011 at 9:52 pm #65620Paul BishopParticipant
We average about 46K per message and 129 messages per day for the one vendor that is doing base64 PDFs in an HL7 message. The largest we ever received was about 130K and cloverleaf handled it with no problem. We don’t do much with the messages – break out the OBX-5.0 field, decode it, and write the PDF to a file in a directory which is then processed by a batch job.
We are running QDX 5.6 on an AIX 5.3 box.
Paul Bishop
Carle Physician Group
Urbana, IL
Paul Bishop
Carle Foundation Hospital
Urbana, IL -
April 27, 2011 at 11:34 pm #65621David BarrParticipant
I think the biggest messages that I’ve sent through Cloverleaf were about 60 megabytes, and the message size limit was based on the memory limit of the engine process which was based on a compile time flag when Cloverleaf was built.
When you’re working with messages that big you need to avoid calling “msgget” (at least for the entire message; calling it to get chunks of the message is OK). If you call “msgget” it makes another copy of the message in memory and doubles the memory requirements.
-
April 28, 2011 at 11:50 am #65622Paul JohnstonParticipant
Thanks Jim C and all who responded .
It is greatly appreciated.
And David B. Very good point on the “msgget” with the large files. Definitely keep this in mind.
-
February 14, 2012 at 7:26 pm #65623Kevin CristParticipant
I know this thread is older but it goes with what i am looking for. We are getting ready to send embedded pdf holters through the engine and i’m told they will be 25-30 meg. Is there some issues or anything to watch out for with this size? It makes me a little nervous. The site that these are in have the messages saved for 5 days, wondering if we need to shorten that. I was told our size is 20 gb.
Thanks.
-
February 14, 2012 at 7:51 pm #65624Jim KosloskeyParticipant
Kevin,
You should be able to calculate that by multiplying your average message size by your average daily message count by 5 days.
I would then allow at least 20% head room for expansion.
That will tell you if you have sufficient archive space. Don’t forget you will need sufficient space for your current SMAT files and you logs will increase whenever you need to turn the EO up to see messages.
email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.
-
February 15, 2012 at 12:16 am #65625Richard HartParticipant
Hi Kevin.
We work with PDF documents, although ours are tiny!
When a document enters Cloverleaf, we strip it out of the message and replace it with a link. When the document is sent out, we put it back in again.
This way, we don’t have documents being passed though the engine.
We have had 30mb documents sent to Cloverleaf – in error – so these can be handled.
-
February 15, 2012 at 2:19 pm #65626Kevin CristParticipant
Thanks. Out of curiosity how do you store your file and pick it up with the link?
-
-
AuthorReplies
- The forum ‘Cloverleaf’ is closed to new topics and replies.