Cloverleaf Alerts

[Mike Klemens]

Greetings all,

I wanted to poll all of you clovertechers to learn about different solutions you may have to getting quick and accurate alerts about your interfaces.

Currently at our hospital we are using custom alert files for each of the analysts (3 of us) – and we rotate the active alert file depending who is on-call.  When we get an alert it triggers to use of a custom “mailx” command (our cloverleaf service is running unix – HP UX) that emails our skytel pagers.  Prior to that we used a program called “TelAlert” that basically did the same thing – except we found it seemed to alert the SkyTel pages a few minutes faster (not sure if TelAlert had a different way of hooking up with SkyTel besides just emailing the pagers)…

Anyway, I am curious what type of alerting solutions everyone is using.  Overall I would prefer it if we could just get rid of our pagers.  However, I think ideally it would be great to implement a system with a lot of flexability where it might alert multiple places – maybe send an email, text message and/or even call us on several different available numbers.

Any help or advice any of you can give would be much appreciated.

Thanks,

Mike

Topic

[Tom Rioux]

Mike,

We too have custom code here that is similar to what you have there.   We don’t carry pagers per se but we all do carry blackberries.  The alerts send the oncall person an email.  Our blackberries are set up with audio alerts based on the subject or the sender of the email.  

Hope this helps…

[Paul Bishop]

We have checks in place (custom code) that runs every 15 minutes (the sites are staggered to even the load) and will tell us if there are messages queued but not sent.  We have a control file for each site/thread so that it can be set to page, email, both, or neither.  It can also be set to automatically cycle the connection once an hour if there are messages queued up (for those troublesome vendors that don’t always ACK a message or drop the connection in the middle of a send).

We are thinking of adding an option to allow us to specify the page to go out to the system owners, but haven’t written that part yet.

The four of us carry a pager and whoever is on call watches it.  The paging works through a remote shell to the communication center so that the page is logged in a database.

Generally, we know of a problem with a vendor’s system before the system administrators know about it.

Paul Bishop
Carle Foundation Hospital
Urbana, IL

[James Cobane]

With the ability to execute commands from the alert tool, you can get as creative/sophisticated as you dare.

[mike brown]

Hi Mike

[Baron Matthews]

We have a couple of scripts, written outside of the engine, that run via the cron.  They are just some tcl scripts (also, we have built in alerts to a lot of batch process that takes place outside of CL, but is kicked off when CL places a file in a certain directory) that check for various criteria every x number of minutes.  The advantage is that they can email as many or as few addresses as we want.  Since there are only 3 of us that get these pages, we all get them, as the person on call knows they are on call.  We all want to be notified if something is dead at 3AM though, just incase. 🙂

We are moving to 5.7r2 in a month or so and I think we will take a look at the built in alert system again at that point.

[Mike Klemens]

Thanks everyone for your feedback!

I am considering actually sending my alerts to my email address and then configuring rules that will automatically forward the alert via text message to multiple phones (I have a work and a personal cell phone).  If anyone is interested, after I set things up I will report back here.

We recently upgraded to 5.7rev2 as well and I am interested to see how I can better tweak the alerts.

[Author]

Replies