Transferring Images in the Engine

[Steve Holder]

We have a debate at our facility regarding image transfers within the engine.  It makes it convenient if the image is included depending on the method of delivery.  However, is it worth the risk of a process panic if the source system sends a very large image.  I know lots of sites incorporate embedding images and some don’t.   For those that do embed images, do you have a threshold script to check the image size on inbound?  Is there a size limit on a message or is it relative to the amount of memory?  Thanks for your advise.

Steve

Topic

[Daniel Lee]

Another thing for you to consider is what happens when the embeded image has the mlp end of message character embeded in it.

[Jim Kosloskey]

Daniel,

Typically imbeded images (ED Data Type) are encoded (usually base64) and there are even components of the ED DataType to indicate the type of binary (RTF,JPG,etc.) as well as the encoding technique deployed.

The encoding keeps the kind of issue you describe from happening.

Of course either the receiving system or perhaps an Outbound Tps proc will need to decode the binary.

So that is not a porblem.

Personally I think we will see less and less of the RP Data Types (Reference Pojnters to files) and more of the imbedded binary files as time goes on.

I have alerted my teammates and management to expect this and to begin to plan accordingly.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Steve Holder]

Jim,

So what is your policy on image transfers?  Do you check the inbound message size before processing?  Is there a maximum size a message can be?  Thanks.

[Jim Kosloskey]

Steve,

Thus far we do little related to images. What we have done in production is use the RP Data Type.

I have experimented with image transfer using the ED Data Type and an encoded, imbedded binary. The largest image I experimented with is 20M.

For that experiment I did not need to do anytyhing and the message traversed Cloverleaf in a reasonable time frame (I do not recall the elapsed time). I understand we have a relatively robust AIX environment on which Cloverleaf runs.

I have not done an analysis, but I suspect many of the binaries we would be interchanging would be less than 20M (probably more in the 2-5M range with some larger). Those sizes are with consideration given to the base64 encoding overhead.

Of course there is also the consideration as to whether the sending and receiving systems can comply with the standard. Also this would be dependent upon the ability to scale up the Cloverleaf platform sufficiently and also the capabilities of the underlying network.

The bottom line in my opinion is this: large messages (just consider the CCD/CDA if you want to see huge) whether as a result of imbedded binaries or not are coming. Now is the time to prepare to handle them. The RP Data Type in my opinion is a work-around whose usefullness is coming to an end – the ED data type with encoded imbedded binaries is what is coming.

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Scott Folley]

We embed PDFs (some upwards of 10-15 Meg).  As long as they are base64 encoded (or MIME encoded in some way) there is no real issue with interference with MLP.  We do not limit the size of an image that we will take in or deliver however some of our communications mechanisms do limit that.  Also, if we do not NEED to have the image ride all the way through with the message then we will put the image out to a file and then read it back in on the outbound-TPS.  It is a little extra work but the translate thread really slows down when messages of that size go through it so the throughput that we get back is well worth it.

One other thing, performance-wise, related to this is that you can win back some of the performance by changing to a length encoded transfer instead of using MLP.  It is a small difference but it does add up with messages of this size.

[Richard Hart]

Hi Guys.

We use a web service for documents – an architectural directive – but we parse out the document and store it out of Cloverleaf and then pick it up again on outbound.

We plan on using embedded documents for another project and again we plan to parse out the document and add on outbound delivery.

[Ian Morris]

We don’t actually send images through the engine.  Instead, in the message, we send their location on a file server.

[Author]

Replies