Home / server / Questions / 241721
Accepted
Sebastian Hoitz
Asked: 2011-03-02 07:56:52 +0800 CST

Get notification from supervisord when a job exits

  • 772

Is there any way supervisord can automatically restart a failed/exited/terminated job and send me a notification email with a dump of the last x lines of log file?

logging monitoring supervisord

2 Answers

  1. Best Answer

    There is a plugin called superlance.

    You install it with pip install superlance or download it at: http://pypi.python.org/pypi/superlance

    The next thing you do is you go into your supervisord.conf and add the following lines:

    [eventlistener:crashmail]
command=/usr/local/bin/crashmail -a -m [email protected]
events=PROCESS_STATE

    This should be followed by a "supervisorctl update". When a process "exits" you will now get a notification sent to [email protected].

    If you only want to listen to some selected apps you can exchange the -a for a -p program1 or if it is a group group1:program2 One example would be

    [eventlistener:crashmail]
command=/usr/local/bin/crashmail -p program1 -p group1:program2  -m [email protected]
events=PROCESS_STATE

    Regarding the automatic restart: you should make sure that autorestart is set to true (it is set to unexpected by default). This way the package will be restarted 3 times. If after that it still exits, it gives up, but you can change that with startretries.

    Example program:

    [program:cat]
command=/bin/cat
autorestart=true
startretries=10
    • 52

  2. I tried installing superlance and running crashmail like this:

    sudo apt-get install python-pip
sudo pip install superlance

    after i do:

    sudo nano /etc/supervisor/supervisord.conf

    and after i added:

    [eventlistener:crashmail]
command=/usr/local/bin/crashmail -a -m [email protected]
events=PROCESS_STATE

    and I do not receive anything....

    My crashmail file is:

    #!/usr/bin/python

-- coding: utf-8 --
import re
import sys

from superlance.crashmail import main

if name == 'main':
sys.argv[0] = re.sub(r'(-script.pyw?|.exe)?$', '', sys.argv[0])
sys.exit(main())
    • 0