I've done a little Docker Swarm in the past, and it was pretty uncomplicated - create a Swarm Manager on one node, create Swarm Workers on two more nodes, stick with a single Manager. I would like to learn more about Swarm, so I am following along the video series "Docker Swarm: Native Docker Clustering" by Nigel Poulton on Pluralsight.
However, these videos are a great deal more complicated than my Swarm experiences two years ago, possibly because they run on an old ("Standalone") version of Swarm. I would like to ask a clarifying question to see if the material I am now learning is rather too outdated to be worthwhile. (I will describe below what is on the video course, so that one does not need to be familiar with it to answer the question).
The video series, now four years old at this point, shows how to:
- Install a high-availability Discovery Service (Consul, Etcd and ZooKeeper are supported, and the author chooses Consul). It uses the progrium/consul image, which as of today has not been updated in five years
- Set up three Swarm Manager nodes on the Consul servers
- Set up three Swarm Worker nodes, on which Consul clients are also installed
The author acknowledges that this is all rather involved, and hints that, at the time of shooting the videos, core Docker might simplify this stuff in the future. He says (Building Your Swarm Cluster -> Installing an HA Discovery Service -> 5:39):
I would even put a wild punt out there, that at some point, Swarm might even not need us to do any of this discovery backend hand-crafting. Just because it's a bit of a pain, and if I know one thing about the folks at Docker, it's that they're all hell-bent on giving us this smooth, effortless, Docker experience, which this part kind of isn't at the moment.
And that feature appears to be what actually happened, if I am reading things right. From the manual:
You are viewing docs for legacy standalone Swarm. These topics describe standalone Docker Swarm. In Docker 1.12 and higher, Swarm mode is integrated with Docker Engine. Most users should use integrated Swarm mode.
Admittedly my previous experiences with Swarm were only using a single Manager, but from what I understand, one can add a Manager to an existing raft merely using docker swarm join-token
.
To my question, then: this feature evolution of Docker Swarm makes me wonder if Consul offers any value for managing the Swarm itself. Can I do it? Should I? Is there more features that it offers over the integrated raft consensus system? Is this learning material now harmful?
(Aside: I've watched a couple of videos from this author, and they are excellent - if this one is outdated then that is outside of his control. Should it be the case that the material needs reshooting, it is the course/platform owners that would need to arrange that).
There are two variants of Docker Swarm. The Classic/Legacy version that can use an external key/value store like Consul. This is unlikely to receive any updates in the future and the repo was recently renamed to "classicswarm" to help reduce some of the confusion.
The integrated version is Swarm Mode, based on the swarmkit project. This does not require any external key/value store, and attempting to configure one in the docker engine for overlay networking will fail when you enable Swarm Mode since it controls the key/value database internally with it's own raft based solution.
Therefore it's not necessary to use Consul, and any guide that doesn't start with
docker swarm init
should be ignored as outdated.I'm really sorry about this confusion... I'll look to get this older video either retired or updated.
For Swarm you should watch the relative sections in:
As both of these use the more current Swarm Mode that is integrated into the Docker Engine and doesn;t require any of that old consul madness.
Feel free to ping me @nigelpoulton with any questions.