Msg processing

[Todd Yingling]

Hey there,

Issue: Cloverleaf 6.1.2 slowness with processing large volumes of historical records using the fileset-local protocol.  It seems that the msg gets bogged down on the translation thread, even though, we are routing raw.  Have others had the same issue?

Is there a way to skip the translation thread?

Thank you,

Todd

Topic

[James Cobane]

Todd,

You may need to tweak your Scan and Read intervals on your inbound thread to optimize your throughput.  If you have the Scan & Read intervals set too frequent/fast/high, you may be spending all the time pulling in messages and not giving enough time to the translation thread.  The engine puts priority on consuming messages, so if you are not giving some time to translation, the messages will queue up in the translation thread/process and not get pushed out to the outbound thread.

Hope that helps.

Jim Cobane

Henry Ford Health

[Todd Yingling]

Hi Jim,

Thanks for your response.  

Yeah, we ran into that snag at first.  That was our initial change.  We now have 200 Max Messages with a Scan Interval of 15 and Read Interval of 5.  That did help greatly, but we still run into slowness issues eventually throughout the week.  I wasn’t sure what other options we had.  

Thanks again,

Todd

[James Cobane]

Other considerations may be:

1) Are there other threads in that process that may be bogging things down?

2) What do the system resources look like; i.e. is there a high “Wait %” or I/O bottleneck?

3) Potentially look at some of the translation throttling options, maybe…

You may want to open a formal incident with Infor to have them help evaluate.

Good luck.  Hopefully you’ll find a solution.

Jim Cobane

Henry Ford Health

[Charlie Bursell]

With this set up your max is 200 records in 15 seconds   Set the scan time much lower.  Then if you see the engine bogging down you can tweak it

[James Cobane]

I thought with a scan interval of 15, read interval of 5 and max messages of 200, the thread would read 200 messages every 5 seconds, and would scan the directory for new files every 15 seconds. Thus, throughput would actually be 600 messages every 15 seconds (200 msgs/every 5 seconds).

Jim Cobane

Henry Ford Health

[Charlie Bursell]

Jim:  From the docs:

Read Interval

Specify the minimum time (IBREADINTERVAL), in seconds, to wait between reading from a file.

[James Cobane]

Thanks Charlie; that is what I thought. Thus, if there is a file to process, the thread would read 200 messages from that file every 5 seconds until there are no messages to process.

[Charlie Bursell]

What I am say is it will read 200 every 5 seconds from a file.if the NL option is chosen.  If the EOF option ( forget exact wording) is chosen then it will process 200 messages every 15 seconds

So, the question is, is he processing files with many messages or single messages?

Hope this makes sense

[Todd Yingling]

The folks here are using NL.  I appreciate the replies.

[Charlie Bursell]

How many records in a file.  If not a lot greater than 200 then you need to adjust

Look at IB queue.  If not really backed up decrease scan time and maybe read time.

Its a balancing act  😀

[Todd Yingling]

Ha, yeah, we are finding that out 😀

I spoke to our 3 conversion analysts and found out only one of them has files with 1000 records or more per file (I thought they were all that way). The other 2 had their files built with 10 records or less from what I can tell after spot checking a few.

Thanks,

Todd

[Author]

Replies