x12parser……no return

[Bob Schmid]

-f nl – parses 🙂

-f eof…….x12 parser hangs?

file is rather large – 1.37 meg

Is there something I should consider in definiing these as EOF terminated ?

smaller files with no deliniation charater at the end…..parses fine as EOF

same format but large file….like above……does not return

increased xmx parm in client – no good

is there something in  /etc/security/limits that would address file size ?

Bob

running 585 xp client, aix server

Topic

[Elisha Gould]

Try leaving it running for some time (1/2 – 1 hour).

The cloverleaf parser isn’t the best for large files as it iterates through all the nodes till it reaches the node requested for every node. So it becomes exponentially slower for each node added.

When doing any translations for xml documents that are more than a few hundred kb, I’ve ditched the grm parsing, and used a separate library.

I’d be guessing its the same for x12.

[Bob Schmid]

Thanks for the workaround!…but still looking for a confirm on whether “LARGE Messages” are going to be untenable in Cloverleaf?

These days……everything is getting larger…..XML, CCD docs, X12…….

Why does “nl” file type “work” and eof does not?…might tend to confirm what you are saying……even though..there are no n’s in the file

Bob

[Jim Kosloskey]

Line Feed takes the file in chunks, eof takes in the whole file as one.

If your file is one message than eof is appropriate and performance in the case of larger messages is always a function of resources – the larger the average message set, the more resources required.

If your file is many messages and each one is terminated by Line Feed then that is appropriate and will have quicker flow through the tools.

I have processed 20M messages through in testing in acceptable time but with some slowness compared to 2k messages but then we have a lot of resources.

So I don’t think Cloverleaf per-se is constrained at the size you are talking about.

email: jim.kosloskey@jim-kosloskey.com 30+ years Cloverleaf, 60 years IT – old fart.

[Author]

Replies