We are just trying out Docker-Swarm with 3 Nodes, each node is acting as Master and as Worker. This works fine, but i don't know if the following behavior is "normal".
If I shutdown one of those 3 nodes, the swarm handles the reallocation of the services fine.
If I shutdown two of those 3 nodes, the services are still working but if i connect to the "last man standing" and try to get my swarm overview with:
docker node ls
i get the following error message:
Error response from daemon: rpc error: code = 4 desc = context deadline exceeded
I am not sure if the swarm can operate when only one node is left? i would guess .. yes?
You have an explanation here : https://docs.docker.com/engine/swarm/raft/
When you are running 3 manager nodes, your cluster can accept to have one node down, because the remaining 2 can agree that the third one is dead.
If you have only one node remaining, maybe it's the one that have a problem. So it will not accept any changes, and you will not have an inconsistent state if your network split.