Is it possible to wire up a small zero downtime deployment (*1) with two Amazon EC2 instances? I'd like to roll my services regularly to a new EC2 instance to avoid manual OS updates on the instances itself.
EC2-1: application serivces
EC2-2: database, consul registry for docker networking
EC2-1 would be the only public instance (bound to an Amazon Elastic IP). Shouldn't be a problem to replicate this one and change the Elastic IP to the new EC2 instance, right?.
However, I don't know if it is possible to switch EC2-2 as docker stores the docker-networking settings in the consul database. Can I start a replica of that instance and tell docker that it should now use the new consul instance for networking?
(*1) you can't guarantee zero-downtime in case of instance failures etc. with two instances. I mean zero-downtime while moving to new EC2 instances :)
With Docker Swarm in 1.12, for zero downtime, you need at least 3. Instead of the past high availability installs where one node takes over when heartbeats fail to the primary, and shared HA storage, Docker traditionally uses a majority or quorum model. Once you push a write to a majority of the nodes, you can be assured to have the most recent version of that content when you then read from a majority of the nodes.
In a Swarm cluster of 3, you can have one node go down and still have a majority of the nodes available for a quorum. Note that you still would need an application that can be run as multiple instances at each level so that if any one node goes down, the application is already running on another to handle the traffic.
The recommended setup is to have 5 nodes in a cluster. That allows one to be taken down for scheduled maintenance, and a second node to fail unexpectedly, without losing quorum.