Logging a Daemon's Output with Upstart

This snippet will pipe the output of your service into logger, while still allowing you to exec the service process (thus replacing the shell process) so that upstart doesn't get confused. It also makes sure the logger process is reparented to init, so it's not a child of your service, and it avoids leaving cruft sitting around in the filesystem, even though it needs to create a fifo temporarily.

script
  mkfifo /tmp/myservice-log-fifo
  ( logger -t myservice </tmp/myservice-log-fifo & )
  exec >/tmp/myservice-log-fifo
  rm /tmp/myservice-log-fifo
  exec myservice 2>/dev/null
end script

Here's how it works:

1. mkfifo /tmp/myservice-log-fifo simply makes the fifo special file (aka named pipe). Type man 7 fifo for more info.
2. ( logger ... </tmp/myservice-log-fifo & ) starts logger reading from the fifo, in the background. The parens cause the logger process to be reparented to init, rather than remaining a child of the current shell process.
3. exec >/tmp/myservice-log-fifo redirects the current shell's stdout to the fifo. Now we have an open file descriptor for that fifo, and we don't actually need the filesystem entry any more...
4. rm /tmp/myservice-log-fifo so we'll remove it.
5. exec myservice 2>/dev/null simply runs the service in the usual way. Stdout is already going to the fifo, and that won't change when the new program executes.

UPDATE: set -e is not needed as Upstart runs scripts with this option by default (see http://upstart.ubuntu.com/cookbook/#develop-scripts-using-bin-sh)

For recent Ubuntu versions (12.04+), simply use

console log

And the daemon output (STDOUT & STDERR) will be appended to /var/log/upstart/<service>.log

http://upstart.ubuntu.com/cookbook/#console-log

If you use the console output stanza, and then pipe your script's output to logger (the shell command interface to the syslog(3) system log module) then that will work.

For example

console output
exec /my/script | logger

will log to /var/log/messages

For example

console output
exec /my/script | logger -t my-script

will log to /var/log/messages and tag each message with my-script

logger --help for logger usage options.

(I'm on the Amazon Linux AMI, which is Centos 5.x based; YMMV)

I did not get the mkfifo trick to work satisfactorily; it did not seem to capture stderr, and attempts to redirect caused Upstart to bail with no errors.

It also has an unfortunate side effect of making the logger process hang around as a child of init, so the information about who "owns" the logger is lost, and anyone not already aware of the mkfifo might assume it's a dangling process that can be killed.

Instead I ended up with the following solution, which resolves all of these issues. It causes logger to become a child process, while preserving the service as the root process. Unfortunately, it requires exec'ing bash, but it just looks dirty.

script
  # ... setup commands here, e.g. environment, cd, ...
  exec bash <<EOT
    exec 1> >(logger -t myservice) 2>&1
    exec myservice
EOT
end script

This uses a trick that redirects stdout and stderr to a command. Since we exec the service inside the bash command, this has the side effect of replacing the shell and magically making bash become a child process of the service, as shown by ps aufxw:

myservice 
 \_ bash -c exec 1> >(logger -t myservice) 2>&1 && exec myservice
    \_ logger -t myservice

For some reason the above command has to be wrapped in a bash -c. I assume this is because Upstart only pretends to run your script via Bash, but isn't actually. If anyone can suggest a way to avoid the extra bash shell, that would be awesome.

This is ugly but so far the best I have found

exec /path/to/server >> /tmp/upstart.log 2>&1

You can also redirect the output to syslog, e.g.

exec $SERVER 2>&1 | logger -t myservice -p local0.info

However, the pipeline may cause upstart to confuse the PID of the logging process with the PID of the daemon.

Another alternative is using tee like:

exec $SERVER 2>&1 | tee /dev/stderr | logger -t myservice

to get both upstart file, and syslog output