IDX Radiology and ACK handling

[Joe Halbrook]

I’ve got an outbound thread that sends ADT and orders to an IDX Radiology 9.9 system.  All is fine until the application undergoes an msql backup at 12:30am each night.  From what I can gather from our Operations folks, the app is not brought down, at this time.  Only the msql backup is taken.

During this time, we frequently get an error in our check_ack proc, throwing this:

[tps :tps :ERR /0:  bn13idx_out] ‘ERROR ‘ (returned by ‘check_ack ‘) does not match { }

For whatever reason, the proc is losing the message handle prior to attempting to ERROR the message.  I suspect this is because of the wacky way the msql backup is occurring, thinking that it’s attempting to post data prior to ACK’ing the message.

I was just curious if anyone has seen this issue, or experienced similar ACK processing issues with IDX Radiology systems. 9.9 and above.

Thank you.

Topic

[Michael Hertel]

My guess would be that this will happen anytime that the engine has to resend a message due to a timeout, not just when the backup happens.

If you don’t receive this error on other threads when resends are invoked, then check this thread to make sure you have the recover33 procs configured correctly.

[Joe Halbrook]

That was the first thing I checked, and the recover procs are configured exactly as all my other outbound threads, which have no issues such as this one.

Joe

[Michael Hertel]

See https://usspvlclovertch2.infor.com/viewtopic.php?t=264&highlight=match+key+value

and

https://usspvlclovertch2.infor.com/viewtopic.php?t=572&highlight=match+key+value

Maybe all of your threads need “send ok” config added.

[Joe Halbrook]

Yes, I’ve got sendOK_save configured for all outbound threads.

[Michael Hertel]

Could you post the contents of your check_ack proc?

My proc has the following:

               set my_mh $ob_save

               set ob_save “”

               lappend dispList “KILLREPLY $mh”

               lappend dispList “ERROR $my_mh”

I’m thinking this might be where the problem lies.

[Joe Halbrook]

Sure:

           keylget args MSGID mh

           set my_mh $ob_save

           # Null out the global variable

           set ob_save “”

           ….

           return “{KILLREPLY $mh} {ERROR $my_mh}”

[Michael Hertel]

Well that didn’t help. Uncle! I give!  😉

[Joe Halbrook]

Yeah, it’s got me too!  I only post when I’m thoroughly stumped!  Thanks for brainstorming with me on this, Michael.

[Jim Kosloskey]

Joe,

Do you have anything which could have cycled the thread during this time period?

If a reply comes back after the recycling for a message sent before the recycling, the saved message object (which is in a global variable) might disappear. Maybe this is what is happening – because it sure appears you do not have a message handle when you are getting the error and your proc gets the message handle from the global.

Jim Kosloskey

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Joe Halbrook]

Jim,  I looked at that, as well.  I cycle one site at 23:05 and one site at 23:19 each night.  These errors are always occuring after the IDX Radiology msql backup kicks off which is after 00:30 each night.  

In the meantime, I think tonight I’ll bring down the outbound thread at 00:27 and have the night operators bring it back up after the see the IDX msql backup is complete, which only takes about 20 mins.  Hospital staff always call in any stat orders during that time of the morning (night).

[Russ Ross]

I can’t say for sure I’ve had the exact same problem but I may have.

I often experience issues during foriegn system outage of the database and even get NAKed on some of them.

The best solution I have so far has been to make a procedural requirement of the foreighn system owner to stop their interface listner before taking their database down and bring the listener back up once the database is available again.

It is even better if they can automate the process programtically becuase procedures are often not followed when done manually.

I think looking on the cloverleaf side to fix your problem will be like chasing your tail – alot of work and no satisfaction.

Russ Ross
RussRoss318@gmail.com

[Joe Halbrook]

Russ,

That’s exactly what we did – implemented taking the thread down to the foreign system, prior to the database backup, then restarting it (manually for now) after the backup is complete.

[Author]

Replies