Process to switch thread from TCPIP to DISK?

[Matt Stemple]

I am researching a new interface to an external repository.  This repository will be outside of our four walls connected via a VPN.  One of my concerns is that the when the external system’s interface is down, our engine will queue up transactions very quickly and likely impact other areas of our engine’s performance.

I am looking for ideas around a process that will at the time of external system disconnect, attempt to reconnect / resend some  number of times then if no connection, the interface will switch from a TCPIP connection to a disk connection.  At the time of the switch to disk mode, all of the recovery database queued transactions would be dumped to file first then continue to write all transactions to disk.  This process would continue until manual intervention and a resend from the disk file could be performed.

Thanks in advanced for your ideas.

Matt Stemple

Topic

[John Hamilton]

Have you thought maybe of writing some tcl to take care of that.

What you could do is set some global flag on your await replies. That if you hit this condition start writing them to a file and killing the messages.

You would of course have to handle all the issues of getting those that are saved to a file sent in the correct order as well as what to do on restarts if you have messages in that file.   You could write it in such a manner as to remove them the from the recovery database.  But all these different things would be very much thread specific

Changing the NetConfig to reflect a defend protocol could become a nightmare. I guess you could setup an alert to shutdown the thread copy the NetConfig.FILE_SETUP to the NetConfig then restart. But then you have to become worried about maintaining more then one NetConfig.

[Jim Kosloskey]

Matt,

If thread forwarding worked the way I would like it to, that is what I would use (along with some other functions).

What we do here (from a high level) is we set up another site (could be done in the same site, different processes).

In the other site (we call it a ‘mock’ site) we have a thread setup to listen via TCP/IP localhost on the port that the original thread is sending. Normally this thread is off.

We then static raw route all messages coming in to a thread which is a file protocol.

We have another set of threads in another process in this mock site which will read the file (fileset local) created by the first process and route (static raw again) to a TCP/IP thread configured same as the original Client.

During a down time, we:

   stop the original Client

   change the NetConfig to point that thread to localhost (this is where forwarding would come in handy)

   in the mock site start the receiving thread (so that we are now listening on the correct port localhost.

   start the original client (now sending to localhost – the mock thread).

If the downtime will be very long, we have scripts that cycle the file being created in the mock site so we don’t run into a failure due to the file getting too large.

Once the downtime is done:

   stop the orginal thread allowing messages to queue

   stop the receiving thread

   stop the file writing thread

   copy the files to the directory the file reading thread is scanning

   start the mock sending thread (sending to the real destination)

   allow all of the files to finish processing

   change the NetConfig for the original thread back to the correct Host (from localhost).

   start the original thread to deliver the messages queued during the startup/recovery.

No extra Tcl involved.

This is in use here for virtualy all of our real time integrations.

We have additional considerations when the original destination is not tcp/ip.

Jim Kosloskey

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Matt Stemple]

Thanks for the ideas Jim and John!  

ms

[Bryan Dort]

We have a big problem with one of our vendors sending a LARGE number of A08s all at once.  This would bog down some systems downstream.  So I setup a disk queue with two threads and fileset-local.  A08s get written to disk with one thread and throttled to the downstream systems with another thread.

Couldn

[Author]

Replies