Please go to Github to get the latest docker-compole.yml. README has corresponding operation steps. Github address: github.com/JacianLiu/d…

I’ve been studyingKafkaWhen preparing to test the cluster state, it was too much trouble to run three virtual machines or create three different port numbers in one virtual machine (HMMM. Mainly lazy).

Environment to prepare

A computer with Internet access and a CentOS7 virtual machine

Why virtual machines? Because of the laptop, the IP will change every time you connect to the network, and you have to always modify the configuration file, which is too tedious and inconvenient to test. (This problem can be avoided through Docker virtual network, which was not known in the experiment at that time)

Docker installation

If Docker has been installed, skip this step

Docker supports the following CentOS versions:
CentOS 7 (64-bit) : The operating system must be 64-bit and the kernel version must be later than 3.10.
CentOS 6.5 (64-bit) or later: The operating system must be 64-bit and the kernel version must be 2.6.32-431 or later.
CentOS only supports Docker kernels in distributions.

Yum install

Docker requires a CentOS kernel version higher than 3.10. Check the previous prerequisites to verify that your CentOS version supports Docker.

Check the kernel version
$ uname -a
Copy the code

# installation Docker
$ yum -y install docker
Copy the code

Start the Docker background service
$ service docker start
Copy the code

# Hello-world is not available locally, so a hello-world image will be downloaded and run inside the container.
$ docker run hello-world
Copy the code

Script installation

Log in to Centos with sudo or root permission.
Make sure the YUM package is up to date.

$ sudo yum update
Copy the code

Get and execute the Docker installation script.

$ curl -fsSL https://get.docker.com -o get-docker.sh
# Executing this script will add the docker.repo source and install the Docker.
$ sudo sh get-docker.sh
Copy the code

Start the Docker

$ sudo systemctl start docker
Copy the code

Verify that docker is installed successfully and run a test image in the container.
$ sudo docker run hello-world
$ docker ps
Copy the code

Mirror to accelerate

At the beginning, I refused to configure the domestic mirror source, but after using it, I found that the download speed of duang~ went up. Therefore, you are strongly advised to configure the domestic mirror source. Open/create /etc/docker/daemon.json and add the following:

{
  "registry-mirrors": ["http://hub-mirror.c.163.com"]}Copy the code

The Zookeeper cluster is created

Zookeeper Image: Zookeeper :3.4

The mirror to

$ docker pull zookeeper:3.4
Copy the code

To find images, go to hub.docker.com/ docker pull images:TAG // to pull the TAG version of the image image

Create an independent Zookeeper container

Let’s start by creating a standalone Zookeeper node in the simplest way possible, and then we’ll create other nodes based on this example.

$ docker run --name zookeeper -p 2181:2181 -dZookeeper: 3.4Copy the code

By default, the container configuration files are in /conf/zoo. CFG, and the data and log directories are in /data and /datalog by default, which can be mapped to the host if necessary. Parameter interpretation

–name: Specifies the container name

-p: allocates a port number to the ports exposed by the container

-d: Runs the container in the background and prints the container ID

The cluster structures,

The creation method of Zookeeper containers for other nodes is similar to that of independent containers. Note that to specify node IDS and modify the configuration of multiple nodes in the file, run the following commands:

Create a Docker network

$ docker network create zoo_kafka
$ docker network ls
Copy the code

Zookeeper container 1

$ docker run -d \
     --restart=always \
     -v /opt/docker/zookeeper/zoo1/data:/data \
     -v /opt/docker/zookeeper/zoo1/datalog:/datalog \
     -e ZOO_MY_ID=1 \
     -p 2181:2181 \
     -e ZOO_SERVERS="server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888"\ --name=zoo1 \ --net=viemall-zookeeper \ --privileged zookeeper:3.4Copy the code

Zookeeper container 2

$ docker run -d \
     --restart=always \
     -v /opt/docker/zookeeper/zoo2/data:/data \
     -v /opt/docker/zookeeper/zoo2/datalog:/datalog \
     -e ZOO_MY_ID=2 \
     -p 2182:2181 \
     -e ZOO_SERVERS="server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888"\ --name=zoo2 \ --net=viemall-zookeeper \ --privileged zookeeper:3.4Copy the code

Zookeeper container 3

$ docker run -d \
     --restart=always \
     -v /opt/docker/zookeeper/zoo3/data:/data \
     -v /opt/docker/zookeeper/zoo3/datalog:/datalog \
     -e ZOO_MY_ID=3 \
     -p 2183:2181 \
     -e ZOO_SERVERS="server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888"\ --name=zoo3 \ --net=viemall-zookeeper \ --privileged zookeeper:3.4Copy the code

This method also achieves what we want, but the steps are too tedious and troublesome to maintain (lazy cancer terminal), so we use docker-compose to achieve it.

Docker-compose builds a ZooKeeper cluster

Create a Docker network

$docker network create --driver bridge --subnet 172.23.0.0/25 --gateway 172.23.0.1 zoo_kafka $docker network lsCopy the code

Write the docker-comemess.yml script

Usage:

The installationdocker-compose

# fetch scripthttps://github.com/docker/compose/releases/download/1.25.0-rc2/docker-compose- $curl - L ` uname-s`-`uname -m` -o /usr/local/bin/docker-compose
# Grant execute permission
$chmod +x /usr/local/bin/docker-compose
Copy the code

Create one in any directorydocker-compose.ymlFile, copy the following
Execute the commanddocker-compose up -d

Command controls

The command	explain
docker-compose up	Start all containers
docker-compose up -d	The background starts and runs all containers
docker-compose up –no-recreate -d	Do not recreate stopped containers
docker-compose up -d test2	Start only the container test2
docker-compose stop	Stop the container
docker-compose start	Start the container
docker-compose down	Stop and destroy the container

Docker-comemage. yml: github.com/JacianLiu/d… Docker – compose. Yml for details

version: '2'
services:
  zoo1:
    image: Zookeeper: 3.4 # mirror name
    restart: always Restart automatically when an error occurs
    hostname: zoo1
    container_name: zoo1
    privileged: true
    ports: # port
      - 2181: 2181
    volumes: # Mount data volumes
      - ./zoo1/data:/data
      - ./zoo1/datalog:/datalog 
    environment:
      TZ: Asia/Shanghai
      ZOO_MY_ID: 1 Node ID #
      ZOO_PORT: 2181 # ZooKeeper port number
      ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888 # List of ZooKeeper nodes
    networks:
      default:
        ipv4_address: 172.23. 011.

  zoo2:
    image: Zookeeper: 3.4
    restart: always
    hostname: zoo2
    container_name: zoo2
    privileged: true
    ports:
      - 2182: 2181
    volumes:
      - ./zoo2/data:/data
      - ./zoo2/datalog:/datalog
    environment:
      TZ: Asia/Shanghai
      ZOO_MY_ID: 2
      ZOO_PORT: 2181
      ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
    networks:
      default:
        ipv4_address: 172.23. 012.

  zoo3:
    image: Zookeeper: 3.4
    restart: always
    hostname: zoo3
    container_name: zoo3
    privileged: true
    ports:
      - 2183: 2181
    volumes:
      - ./zoo3/data:/data
      - ./zoo3/datalog:/datalog
    environment:
      TZ: Asia/Shanghai
      ZOO_MY_ID: 3
      ZOO_PORT: 2181
      ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
    networks:
      default:
        ipv4_address: 172.23. 013.

networks:
  default:
    external:
      name: zoo_kafka
Copy the code

validation

As can be seen from the figure, there is a Leader and two flowers, so our Zookeeper cluster has been set up

Kafka cluster setup

Is it still a problem to build Kafka clusters once you have the basics? It’s just a couple of variables with different values.

Docker-compose is composed for docker-compose. Docker-compose is composed for docker-compose. Docker-compose is composed for docker-compose, docker-compose is composed for docker-compose. At this time, we do not need to build a new Docker network, directly use the network created when building the Zookeeper cluster before!