TCL 8.4 Variable, List Size Limits

[Bob Richardson]

Greetings,

I have a design question for a fileset dirparse TCL procedure that will read in an entire bill charge file (newline delimited) with a batch header, transaction details, batch trailer.  The fileset is configured to read these files as EOF style.

Ok… requirement is to count the transaction details and compare to the trailer count.  I figure to do this in a TCL TPS IB procedure that has the entire file in memory.  Basically, split the message (file) on newline and step through it to validate; if good then disposition each transaction detail with CONTINUE (copy existing message for metadata) then kill original message,  No problem here other than the following:

Question: what are the variable, array size limits in TCL 8.4?

(we are currently running CIS5.6R2 on AIX5.3 TL11).  In browsing the web and this forum I have come up empty.  I am concerned about exceeding the variable size limitations in the TCL interpreter.

Thanks for your responses – in the meantime, I will test this using the Edison method, that is, feed my TCL procedure huge files and see if it crashes.

Topic

[David Barr]

Take a look at this topic: <a href="http://clovertech.infor.com/viewtopic.php?t=2316&#8243; class=”bbcode_url”>http://clovertech.infor.com/viewtopic.php?t=2316.

I don’t think you have to worry about the amount of memory per variable, but you can’t exceed the memory limit per process.

If your files are smaller than 30 megs or so, you probably won’t have a problem.

[Bob Richardson]

David,

Thank you!  Forgot about the ulimit setting for the engine user accounts

– hci and hcitest.  The link has useful information on this topic.

Have a great day!

[Russ Ross]

Over a decade ago I inherited charge file integrations that read the entire file into memory which made me uncomfortable the minute I saw it much like the discomfort you are expressing now.

Just because you can do something doesn’t necessarily mean you should do it and I think that applies in this case.

I ended up redoing those batch integrations with a new batch model that preprocess the batch charge file(s) to create a one message per charge to be fed to the xlate one message at a time like a typical real-time message flow.

During the preprocessing phase with a standalone script (could be TCL, perl, or KSH) outside the cloverleaf engine I do basic data validation like you are asking about to compare the charges in the file with the trailer record to be sure they match.

I personally think it is a bad design and overuse of shared resources to read an entire file into memory during xlate before processing it.

When I was first faced with this challenge I had no choice due to limted resources that were being overconsumed.

Even though today I now have sufficient resources to waste, I still prefer not to because those pieces do add up and I don’t need any extra unecissary worries.

Russ Ross
RussRoss318@gmail.com

[Author]

Replies