No Dishes Studio – Marklux Source: marklux.cn/blog/55 All rights reserved

preface

After the launch of DOCKER technology, a wave of containerization technology has been set off. Containerization makes the deployment of services extremely easy, which provides great convenience for micro-services and distributed computing.

In order to give full play to the advantages of containerization technology, Docker company successively launched three technologies: Docker-machine, Docker-compose and Docker-swarm, which can be said to have almost realized all the underlying technical means that may be needed in containerization technology.

After the go language is used to realize the problem solving engine and package the Docker image, it is necessary to write distributed problem solving. This time, let us practice manually and try to deploy a multi-machine problem solving service cluster by using the three major killers of Docker.

The Three Musketeers

docker-machine

Docker technology is based on the Cgroup technology of the Linux kernel, so the question comes, whether docker technology can not be used on the non-Linux platform? The answer is yes, but it obviously requires a virtual machine to simulate a Linux environment.

Docker-machine is officially proposed by Docker company, which is used to quickly create virtual machines with Docker services on various platforms. It can even customize the realization principle of virtual machines (generally Virtualbox) by specifying driver.

docker-compose

Docker image after creation, often need to manually pull to obtain the image, and then execute the run command to run. The manual operation of deploying a service when it needs multiple containers, with dependencies and connections between them, can be quite annoying.

Dcoker-compose is composed by using a. Yml configuration file to write all the container deployment methods, file mapping, container connection, and a series of configurations into one configuration file. Finally, you only need to execute docker-compose up command to install containers one by one and automatically deploy them like executing scripts, greatly facilitating the deployment of complex services.

docker-swarm

Docker swarm is a cluster technology based on docker platform. It can quickly create a Docker cluster through a few simple instructions, and then deploy applications on the shared network of the cluster, and finally realize distributed services.

Compared with ZooKeeper and other cluster management frameworks, Swarm is very lightweight. As a tool, it condense complex operations such as node addition, management and discovery into a few simple commands. It also has algorithms for automatic node discovery and scheduling, and supports customization. Swarm’s technology is not yet very mature, but its power is already visible.

Brief introduction to Docker service architecture and remote API

Before formally deploying cluster applications using Docker technology, we should first understand some underlying principles of Docker work and docker remote call API, so as to get a general understanding of how clusters work.

daemon

As mentioned in previous Docker introductory articles, basic docker services, such as container creation, view, stop, and image management, are actually implemented by Docker daemons.

Every time a Docker directive is executed, it actually sends a request to the daemon.

Daemons operate in two main modes: Unix sockets (by default, but only locally accessible, which is safe) and listening for TCP addresses and ports (this can be used remotely to call docker services).

The remote API

In addition to accessing the Docker service on the remote host through the remote TCP protocol, Docker also provides a set of HTTP-based API. You can use curl to operate the Docker service on the remote host, which facilitates the development of web-based Docker service.

Example of remote Docker use

The final implementation of cluster is actually the use of docker remote call to connect different Docker hosts into a whole (through TCP protocol).

Let’s start by manually simulating the remote invocation of the Docker service.

First, you need to change the running mode of the docker to TCP on the host providing the service by changing the DOCKER_OPTS in /etc/default/docker to the following content

- H TCP: / / 127.0.0.1:4243 - H Unix: / / / var/run/docker. The sockCopy the code

The parameter after -h is the TCP address and port that you define to bind. After binding, restart the Docker service to access the Docker Daemon service on this port.

Unfortunately:

  • Docker daemon configuration file is not found on OSX platform
  • On the OSX platform, useDocker -h TCP: / / 0.0.0.0:4243 - H Unix: / / / var/run/docker. The sock - dAttempting to start the Docker daemon using TCP also fails and has no effect
  • At present, it is speculated that the configuration methods on other platforms are not quite the same except Linux platform. However, no solution has been found on the network for the time being, so I can only simulate remote call by creating multiple Docker-machines locally for the following operations.

Let’s say we have docker enabled on 192.168.1.123 and listen on port 2375. Then we can use docker -h TCP ://192.168.1.123:2345 on other hosts in the same network segment (for example, 192.168.1.233) to call the docker service on the host.

Such as

Docker-h TCP ://192.168.1.123:2345 ps docker-h TCP ://192.168.1.123:2345 images Docker-h TCP ://192.168.1.123:2345 run .Copy the code

When Swarm eventually built the cluster, it used such remote service calls to schedule individual nodes.

Clustering and distributed computing

Before we start practicing clustering in earnest, it is important to understand what clustering is and what distributed computing is.

First of all, they have one thing in common: they both use multiple service nodes, which, in layman’s terms, means multiple servers working together (not necessarily entities, but virtual machines).

The difference between the two is:

  • Cluster refers to multiple machines that perform the same service and search for the most appropriate node to perform the service according to the scheduling algorithm
  • Distributed computing is the process of breaking up a business into independent parts that are performed by multiple machines working together

The advantage of clustering is that when the service needs a large number of resources, it can avoid the burden of a server alone, and even if a node goes down, the service can still run normally. This is a bit like load balancing.

The advantage of distributed computing is that it can cooperate with multiple machines to exert the power of computing and carry out operations requiring ultra-high performance.

Build a cluster

Now let’s start building the cluster.

Use docker-machine to create nodes

Due to the lack of physical machines and the failure to start TCP docker service on OSX, we created multiple virtual machines based on Docker-machine as nodes in the cluster.

Execute the following command to create a new Docker-machine manager1

docker-machine create --driver virtualbox manager1Copy the code

After creating a VM, you can use the Docker-machine Env Manager1 command to view vm Manager1 information, including the IP address

Create worker1 and worker2 by using docker-machine ls

docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS manager1 - virtualbox Running TCP: / / 192.168.99.100:2376 v17.06.1 - ce worker1 - virtualbox Running TCP: / / 192.168.99.101: v17.06.1 - ce worker2-2376 Virtualbox Running TCP: / / 192.168.99.102:2376 v17.06.1 - ceCopy the code

After a Docker machine is created, you can use the Docker-machine SSH Manager1 to access the VM and run the command.

Swarm create swarm

Swarm swarm to initialize a swarm

docker swarm init --listen-addr <MANAGER-IP>:<PORT> --advertise-addr <IP>Copy the code

Listen-addr is the IP address of the docker service on the manager node :PORT, that is, the docker service on the manager node can be accessed through this combination.

— Advertise-addr indicates the broadcast address, that is, the IP address that other nodes need to access when joining the swarm cluster

Now create a swarm network in Manager1 and execute

Docker-machine SSH Manager1 docker swarm init --listen-addr 192.168.99.100:2377 --advertise-addr 192.168.99.100Copy the code

Return response:

Swarm initialized: current node (23lkbq7uovqsg550qfzup59t6) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join \
    --token SWMTKN-1-3z5rzoey0u6onkvvm58f7vgkser5d7z8sfshlu7s4oz2gztlvj-c036gwrakjejql06klrfc585r \
    192.168.99.100:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.Copy the code

This creates a Swarm cluster, and the Manager1 node is now added to the node as a manager.

Now we put worker1 and worker2 two nodes to join in the swarm clusters, respectively in the virtual machine of two nodes perform docker swarm join — token.. You can:

docker-machine ssh worker1 docker swarm join --token \ SWMTKN - 1-3 z5rzoey0u6onkvvm58f7vgkser5d7z8sfshlu7s4oz2gztlvj - c036gwrakjejql06klrfc585r \ 192.168.99.100:2377 This node joined a swarm as a worker.Copy the code
docker-machine ssh worker2 docker swarm join --token \ SWMTKN - 1-3 z5rzoey0u6onkvvm58f7vgkser5d7z8sfshlu7s4oz2gztlvj - c036gwrakjejql06klrfc585r \ 192.168.99.100:2377 This node joined a swarm as a worker.Copy the code

Docker node ls can be executed on any node to view all nodes in the current cluster:

docker-machine ssh manager1 docker node ls NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS manager1 - virtualbox Running TCP: / / 192.168.99.100:2376 v1.12.3 worker1 - virtualbox Running TCP: / / 192.168.99.101:2376 v1.12.3 worker2 - Virtualbox Running TCP: / / 192.168.99.102:2376 v1.12.3Copy the code

Create a cross-host network

Once the cluster is set up all we need to do is deploy our services on the cluster. But first, all the nodes should be in a shared network, so that when we deploy the service in the shared network, it is deployed in the entire cluster.

Docker network LS can be used to view all the networks that the current host participates in:

docker-machine ssh manager1 docker network ls
NETWORK ID         NAME            DRIVER          SCOPE
764ff31881e5        bridge          bridge          local                  
fbd9a977aa03        host            host            local               
6p6xlousvsy2        ingress         overlay         swarm            
e81af24d643d        none            null            localCopy the code

SCOPE is swarm, DRIVER is overlay is the shared network in the cluster node. There is a default ingress shared network after the cluster is created. Now let’s create another one:

docker-machine ssh manager1 docker network create --driver overlay swarm_testCopy the code

Deploy services across host networks

To deploy an application on a cluster, you need to deploy a service on a shared network.

However, first of all, it is necessary to ensure that each node has the required image and environment. This can be achieved by sharing the same docker-compose configuration file to each host and using docker-compose to download the image and build the environment on each node.

Since judge_server’s service architecture is simple, just a single image, I’ll just pull it down on each host:

Docker - machine SSH manager1 docker pull registry.cn-qingdao.aliyuncs.com/marklux/judge_server:1.0 docker - machine SSH Worker1 docker pull registry.cn-qingdao.aliyuncs.com/marklux/judge_server:1.0 docker - machine SSH worker2 docker pull registry.cn-qingdao.aliyuncs.com/marklux/judge_server:1.0Copy the code

The next step is to start our service on the shared network using the Manager1 node

docker service create --replicas 3 --name judge_swarm -p 8090:8090 --network=swarm_test registry.cn-qingdao.aliyuncs.com/marklux/judge_server:1.0Copy the code

Does this command look a lot like Docker run? Yes, swarm’s ultimate goal is to make operating clusters as easy as operating a single Docker server!

— Replicas specifies the number of nodes required by the service, i.e. the size of the cluster. This value is elastic and can be changed dynamically later.

When a node in the service fails, Swarm searches for the remaining available nodes in the cluster and takes its place. Swarm will schedule dynamically, always keeping the service running on three nodes.

-p is used to expose ports to the host so we can access them.

–network Specifies the network on which the service is deployed

Now use docker service ls in the Manager1 node to view the services in the cluster:

docker service ls ID NAME MODE REPLICAS IMAGE PORTS kofcno637cmq judge_swarm replicated 3/3 Registry.cn-qingdao.aliyuncs.com/marklux/judge_server:1.0 * : 8090 - > 8090 / TCPCopy the code

If you change the IP address to worker1 or worker2, the result is the same, because all nodes are in a common cluster network

Swarm dynamically schedules each request and selects a different node to process it.

legacy

At this point, the cluster deployment is complete, but we still have several problems left:

  • The dynamic addition and deletion of cluster nodes is not very convenient, which leads to some difficulty in managing the problem solving service machine on the Web side. Of course, it can be achieved through the REMOTE API of Docker, but the complexity is relatively high
  • File synchronization between cluster nodes is not easy to implement and may require script synchronization or usersyncAnd so on
  • Swarm is great for quickly building a large number of clusters to handle business, but it’s a bit of an overkill for just a few machines