I've been asked to research ways of handling end user notifications when systems such as email are experiencing problems. Perhaps an example will make this a little clearer.
We have a number of sites in different countries. Recently email was impacted at one of the sites, but it could have been a complete network outage. Information was provided by phone to local IT managers at the site but onward communication was slower than some would have liked.
It seems like almost everyone at the site has a personal mobile phone which could receive text messages, and perhaps access a remote website with postings on the situation. However managing and supporting a system to text people on these relatively infrequent occasions would be very costly to do internally.
What are other people doing to handle situations like this?
Some things I've thought of include:
- Database of phone numbers to text. Seems costly and not very easy to maintain for an already stretched IT group. Is there an external service that would let you do this policies?
- Send voicemail message to all phones on site.
- Maintain an external website. This would not work in all situations (network failure), and there is a limit on the amount of info that can be posted externally. A site outage could be sensitive information in some situations. How could the site be password protected? Maybe OpenId/Facebook connect would work.
- Use a site like Yammer.com which is publicly accessible but only by people with a company email address. Anyone using this for IT outage notifications?
To me it looks like there is no clear answer, and that there are solutions for some subsets of users. To be comprehensive a number of solutions would need to be combined.
Any additional thoughts or recommendations? What worked or didn't work for your organization?
We subscribe to a sms "short code" system google sms notification services. Basicly we register mobile numbers in it. Authorized users can send a code to the service and it will send out a canned sms to all registerd numbers. remember sms is not "secure" so only general outage info is sent. At each site there is a whiteboard that in the event of an outage a manager is called with the more details and they write them on the board. This is over a very low tech and cheap system the way we are subscribed to the sms system pay a small monthly fee for the service and each time we send a notification we pay a small fee that is based on the number of registered numbers it is being sent to and billed on the next month.
Well, one option would be to get a 3G aircard, and hook it up to a "monitoring" server (Something like Nagios or OpenNMS). Then run a separate email server on that box, sending data over the aircard. You wouldn't use much data (as the aircard would be used for nothing but emergencies). You can send emails to cell phones (number@carrier's address), so even a total network outage would still give notifications. Only a power outage would kill the system. But if you have your UPS's connected to the monitoring server, it can detect the power failure and send a notice before it goes down...