We’ve spent a lot of time reviewing Docker and Amazon’s Elastic Cloud Compute (EC2) Container Service (ECS) as a strategy for clustering Docker containers. In this article we’re going to review the native clustering support that Docker provides through its Docker Swarm solution.
Introduction to Docker Swarm
Docker Swarm is Docker’s native clustering technology. It works very well with the Docker command line tools like docker and docker-machine, and provides the basic ability to deploy a Docker container to a collection of machines running the Docker Engine. Docker Swarm does differ in scope, however, from what we saw when reviewing Amazon ECS.
Amazon ECS leverages its own technology stack to run Docker containers. This includes EC2 instances to host the virtual machines, auto-scaling to scale those virtual machines up and down, Elastic Load Balancers (ELB) to distribute load to your Docker containers, and more.
Docker Swarm, on the other hand, is only a clustering technology: you register the servers that can run Docker Containers with Swarm and Swarm will deploy containers to those machines. It is your responsibility to start and stop machines, register and deregister machines with Swarm, and register and deregister your containers with your own load balancing solution.
While Amazon ECS is probably the preferred way of clustering Docker in AWS, Docker Swarm does have the ability to run anywhere, including outside of Amazon. Many organizations are embracing the cloud slowly and many organizations are running in a hybrid cloud environment, in which some applications or additional instances of applications are running in a public cloud while the remaining applications are running in a local data center.
Regardless of whether you have embraced a cloud platform, are running a hybrid cloud, or running solely in your own data center, Swarm will enable to you to take advantage of Docker in any environment.
So how does Docker Swarm work?
Docker Swarm is implemented using two different types of components:
A manager container, which runs on a virtual machine, manages the environment, deploys containers to the various agents, and reports the container status and deployment information for the cluster; it is your primary interface into Docker Swarm
Agents are containers running on virtual machines that register themselves with the manager and run the actual Docker containers.
Figure 1 shows how these concepts are related.
In this example, we have a Docker Swarm Manager that is managing two agents (Agent1 and Agent2). Those two agents are running two instances of an Nginx Container. Both the manager and agents are “docker machines” that contain the Docker Engine and are capable of running Docker containers. As we’ll see in the next section, Docker machines are very similar to Docker containers themselves, with the exception that they are started using the docker-machine command instead of the docker command.
The example in the next section will demonstrate how to setup a Docker Swarm cluster on your local machine (hence the reference to “Your Laptop” as the Docker Host in figure 1). In production, both the manager as well as the agents will run directly on their own virtual machines.
Getting Started with Docker Swarm
Now that you understand what Docker Swarm is and how it works, this section will walk you through setting up a local Docker Swarm cluster so that you can run it first hand.
The first step is to install Docker. You can find information on installing Docker on a Mac here, on Windows here, and on Linux here. When you install Docker, it will include the docker-machine command, which we need in order to create the Swarm manager and agents. The Docker Swarm documentation defines the docker machine as follows:
Docker Machine is a tool that lets you install Docker Engine on virtual hosts, and manage the hosts with docker-machine commands. You can use Machine to create Docker hosts on your local Mac or Windows box, on your company network, in your data center, or on cloud providers like AWS or Digital Ocean. Using docker-machine commands, you can start, inspect, stop, and restart a managed host, upgrade the Docker client and daemon, and configure a Docker client to talk to your host.
Star Docker and open a command line terminal. Execute the docker-machine ls command to view all Docker machines running on your computer:
$ docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM default * virtualbox Running tcp://192.168.99.100:2376
On my machine, I only have the standard “default” machine running on IP Address 192.168.99.100. Because we’re about to create and start three new machines (one manager and two agents), you might want to stop the default machine to conserve resources on your local computer:
$ docker-machine stop default
Next, let’s create the three Docker machines using the docker-machine create command:
$ docker-machine create -d virtualbox manager
$ docker-machine create -d virtualbox agent1
$ docker-machine create -d virtualbox agent2
At this point, these machines do not have anything to do with Swarm, they are simply virtual machines that we’re creating on our local computer. The -d argument specifies the driver that we want to use when creating the machine, which, when running locally, should be virtualbox. We can view our machine by executing the docker-machine ls command again:
$ docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
agent1 - virtualbox Running tcp://192.168.99.101:2376 v1.11.1
agent2 - virtualbox Running tcp://192.168.99.102:2376 v1.11.1 default - virtualbox Stopped Unknown
manager * virtualbox Running tcp://192.168.99.100:2376 v1.11.1
We have three running virtual machines that we can use, named manager, agent1, and agent2. Before we can start our Swarm cluster, we need to create a discovery token. The discovery token is a unique identifier for the Swarm cluster that we’re creating and we’ll use it to start the manager and give it to the agents so that they can register themselves with the manager.
Creating a Discovery Token
To create the discovery token, we run an instance of the Docker “swarm” container, passing it the “create” argument. An interesting point about Docker Swarm is that it truly builds on top of Docker’s foundational technologies. The actions that we’ll perform in this section to (1) create a discovery token, (2) start a Swarm manager, and (3) start Swarm agents, will all be performed by a preconfigured Docker container named “swarm”. It will make more sense as we go along.
The docker command line tool is used to send commands to the active machine. As you’ll notice above, the “manager” machine has an asterisk in its “ACTIVE” column, which means that all docker commands will be sent to the manager Docker machine. I demonstrate how to change the active machine below.
Let’s create a discovery token:
$ docker run --rm swarm create
Unable to find image 'swarm:latest' locally
latest: Pulling from library/swarm
eada7ab697d2: Pull complete
afaf40cb2366: Pull complete
7495da266907: Pull complete
a3ed95caeb02: Pull complete
Status: Downloaded newer image for swarm:latest
This command should look a little familiar to you:
- Docker run: run the specified container instance
- “swarm”: this is the name of the Docker image from which to create a container; note that because we do not specify a version, this is equivalent to “swarm:latest”
- “create”: this is the command that we’re sending to the “swarm” container
- --rm: this command line option tells Docker to delete the Docker image when it is finished; this means that if we need it we’ll need to download it again, but it conserves space
The last line in the output is your discovery token:
7c14cbf2a86ecd490a7ea7ae4b795a6b. Save this because you’re going to need it in the next step.
Hold that thought, and rejoin us in our next article where we start our Swarm Manager and agents.