TIL that docker kill
means kill in the sense of "make it dead", as opposed to the POSIX sense of "send a signal".
We have several containers that we need to send a SIGHUP
to in order to reload configuration, but this causes them to ignore their restart policy of "always", which isn't what we want.
What's the best way to send signals to these containers without affecting their ability to auto-restart?
To more clearly demonstrate the issue we're seeing, take the following example.
We have some container that has a restart policy set to always
$ docker inspect cloudwatch-exporter | jq .[].HostConfig.RestartPolicy
{
"Name": "always",
"MaximumRetryCount": 0
}
We reload the config at some point using docker kill
:
$ docker kill --signal=SIGHUP cloudwatch-exporter
cloudwatch-exporter
Some time later, something happens that kills the process. To simulate this, I'll send a signal within the container:
$ docker exec cloudwatch-exporter bash -c "kill 1"
At this point, the container is dead and won't restart:
$ docker ps -a | grep cloudwatch-exporter
c7827204bba5 prom/cloudwatch-exporter:cloudwatch_exporter-0.8.0 "java -jar /cloudwat…" 20 hours ago Exited (143) 3 minutes ago cloudwatch-exporter
What alternatives do we have for using docker kill
? For many cases docker exec
works, but it won't work for any container that only contains a single static-linked binary.
This is a known Docker bug.
See https://github.com/moby/moby/issues/11065 "Non-fatal signals break restart policies"
See https://github.com/moby/moby/issues/41302 "Signal breaks unless-stopped restart policy"
Bypass the problem by doing
docker exec "kill -HUP 1"
.With the
--signal
option todocker kill
?Or with
docker exec
?