message throttling

[Gary Atkinson]

Can others share what they have done with message throttling in their sites?  We are on 5.5 rev2 on AIX 5.3.

thx,

gary

Topic

[Mark Thompson]

Gary,

Cloverleaf 5.6 on AIX 5.3

Sometimes we use a “penalty box” thead to throttle messages.  It is a fileset-local thread that writes each outbound message into a separate file in the directory of your choice.  We use the numfile proc to name the message files so they stay in order.  (We call the directories “queue directories” because they queue up messages that have come in faster than we want to process them.)  The inbound side of that same penalty box thread reads messages at whatever rate you set in the fileset-local properties.  

Insert the penalty box thread between your inbound and outbound threads.  Decide where you want to do translation (based on whether you want to save translated or untranslated messages in the queue directory).

We group our queue directories together so it is easy to scan or write reports that monitor the queue depth in these penalty box queues.

- Mark Thompson
HealthPartners

[David Bolin]

Here is a very simple idea for throttling the messages coming in from any particular inbound thread. Copy the “create_ack” proc as “create_ack_delay” and then add a 1 second sleep delay to the new proc. Replace this  “create_ack_delay” proc to do the function of “create_ack” proc for each thread in need of throttling. Be advised that this will then hold the messages on the client as they stack up. For the Cloverleaf admin this is great but may be a problem for the client machine’s admin. We implemented this to prevent sporadic critical messages from getting stuck in the out bound queue behind a stack of low importance batch messages.

[Mark Thompson]

David,

You might want to add to your “Please be aware” …

Our understanding is the entire process is frozen during the 1 second sleep delay.  If there are other threads in the process, they also will be stopped during the 1 second delay.

- Mark Thompson
HealthPartners

[Rob Abbott]

Mark is correct.  If you’re going to do this it’s advised that you put any threads executing ‘sleep’ in their own process.

Rob Abbott
Cloverleaf Emeritus

[Richard Hart]

We have used throttling in production for large data loads.

We use one thread, fileset-local  reading and writing to the local disc.  

Inbound messages are written to hourly timestamped files, set with the DRIVERCTRL parameters.  We did not want a directory with 000’s of files containing individual messages!

The Inbound read TPS ensures that ONLY files that aren’t currently being written to are readable and returns the first file in the list.

The Inbound scheduling is set to the throttling values required.

With hourly files, there is a possibility of an hour delay in quiet times – which was not an issue for us – but may be an issue for others.  Obviously this can be changed.

[Troy Morton]

I’m not sure of your particular situation or reason for throttling, but Cloverleaf does have an Xlate Throttle setting in the NetConfig GUI.  This throttle DOES NOT throttle messages coming in, but it does throttle how many messages are Xlated and sent downstream at a time.  This Xlate throttle can help reduce the number of messages that build up in Recovery Db state 7 in high-volume situations.  

Cloverleaf engines tend to favor bringing in and Xlateing new messages over sending them out of the engine, so many times if Cloverleaf cannot keep up with message volume you end up with a large queue in State 7 and a longer delay in delivery to the downstream systems than if Cloverleaf would Xlate a batch of messages then push them out, then read another batch, push them out, and so on.

The Xlate Throttle in the Network Configure tool does just that, it forces Cloverleaf to Xlate and push out the number of messages you specify either by percent of the in-bound Pre-TPS queue or an exact number entered.  (see screen shot)

[Gary Atkinson]

Can the throttle be done at the thread level?

[Christopher Wells]

Thanks Troy, this is just the info I needed. Very helpful. 🙂

Regards,

Christopher Wells

[Troy Morton]

You’re welcome.  I’m glad it helped you.

To answer Gary’s question, I don’t think this can be done at the thread level, only at the process level.  You could also put a SLEEP in your in-bound Tclproc check_ack, but that would freeze the whole process for the SLEEP duration, which would be bad if there was more than just a single inbound and outbound thread in the process.

[Author]

Replies