Home / server / Questions / 776732
Accepted
Alexis Wilke
Asked: 2016-05-14 20:00:44 +0800 CST

How long do daemons have to exit when the server shutdown signal is received?

  • 772

As we are developing a complex set of services on Linux, we developed a tool that starts them one by one. One of the many considerations for creating such a tool was the order in which items can be started, but also a way to ensure that on the death of a daemon, the daemon auto-restarts. There are also server wide parameters that are shared between all the services.

However, I have a problem now where shutting down such a system takes time. It can take as much as 10 seconds to shutdown everything.

What I'm wondering is: How long a script defined under /etc/init.d/... can take to shutdown daemons it is controlling?

Although I would imagine that if we were to break down all of those daemons in separate packages (since startup scripts can now include a list of dependencies...), we would bumped in the exact same problem. So at this point we prefer to keep things the way they are...

Is there a well defined/known amount of time that a shutdown must take at the most to be graceful to all daemons?

linux service shutdown

2 Answers

  1. Best Answer

    Is there a well defined/known amount of time that a shutdown must take at the most?

    No.

    • 5

  2. As I now tested a shutdown of various daemons on a system running with systemd, I can attest that the timeout is clearly defined for each daemon.

    From what I can tell, it also applies to daemons that are still started/stopped with a SysV script. When Cassandra is still working on its files, doing a systemctl restart cassandra will not work as expected. For such services, you probably want to do a systemctl stop cassandra and once you can be sure it was stopped, do systemctl start cassandra.

    So... You may define/change the TimeoutStopSec paramter on a per daemon basis. This allows you great granularity!

    [Unit]
...
TimeoutStopSec=120

    And you may change the system default: DefaultTimeoutStartSec (which is probably not advisable...)

    There is another important timing, which is the restart feature (shown in the last link.) It is very important because systemd wants to restart a process in 100ms by default!!! So if your daemon take up to 2 minutes to shutdown, it won't work right...

    For those interested, for Cassandra, I actually first run a script which stops Cassandra. Then I proceed with the shutdown.

    This can take how much time Cassandra needs (it can be quite long) but it will cleanly stop Cassandra. Note that it may feel like it is long to shutdown that way, but on a restart, Cassandra will be ready nearly instantaneously.

    In comparison, shutting down down really fast means killing Cassandra and on a restart it has to go back through its journals which is actually way longer. So that's a good trade off.

    • 1