Docker-compose creates an ELK cluster

directory

• planning
• Format file
• Start the command
• The resources

All the layout files and configuration files can be accessed on Github. You can quickly set up a 3-instance ES cluster and a Kibana instance by changing the path in the configuration file.

planning

The plan is to create three ES instances to form a cluster and one Kibana instance to connect to the cluster. Each ES instance uses a local configuration file for easy configuration file saving and version management. Kibana configuration files are also stored locally and mounted into containers via file mapping.

The overall directory structure is as follows:

$tree. ├ ─ ─ docker - compose. Yml ├ ─ ─ kibana. Yml ├ ─ ─ node1 │ └ ─ ─ es1. Yml ├ ─ ─ 2 │ └ ─ ─ es2. Yml └ ─ ─ node3 └ ─ ─ es3. Yml 3 directories, 5 filesCopy the code

Format file

The main orchestration file is docker-comemage.yml

version: "2.1"
services:
  es-node1:
    image: Docker. Elastic. Co/elasticsearch/elasticsearch: 6.7.0
    hostname: es-node1
    expose:         Do not expose ports to out-of-container applications
      - "9001"
    ports:          Expose the port to the host
      - "9200:9200"
      - "9300:9300"
    volumes:
      - ~/Projects/sh-valley/docker-conf/elasticstack/cluster/node1/es1.yml:/usr/share/elasticsearch/config/elasticsearch.yml
    environment:
      - cluster.name=es-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms256m -Xmx256m"
    ulimits:
      memlock:
        soft: - 1
        hard: - 1
    networks:
      es-cluster-network:
        ipv4_address: 172.21. 010.
  es-node2:
    image: Docker. Elastic. Co/elasticsearch/elasticsearch: 6.7.0
    hostname: es-node2
    expose:         Do not expose ports to out-of-container applications
      - "9002"
    ports:          Expose the port to the host
      - "9201:9201"
      - "9301:9301"
    volumes:
      - ~/Projects/sh-valley/docker-conf/elasticstack/cluster/node2/es2.yml:/usr/share/elasticsearch/config/elasticsearch.yml
    environment:
      - cluster.name=es-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms256m -Xmx256m"
    ulimits:
      memlock:
        soft: - 1
        hard: - 1
    networks:
      es-cluster-network:
        ipv4_address: 172.21. 011.
  es-node3:
    image: Docker. Elastic. Co/elasticsearch/elasticsearch: 6.7.0
    hostname: es-node3
    expose:         Do not expose ports to out-of-container applications
      - "9003"
    ports:          Expose the port to the host
      - "9202:9202"
      - "9302:9302"
    volumes:
      - ~/Projects/sh-valley/docker-conf/elasticstack/cluster/node3/es3.yml:/usr/share/elasticsearch/config/elasticsearch.yml
    environment:
      - cluster.name=es-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms256m -Xmx256m"
    ulimits:
      memlock:
        soft: - 1
        hard: - 1
    networks:
      es-cluster-network:
        ipv4_address: 172.21. 012.
  kibana:
    image: Docker. Elastic. Co/kibana/kibana: 6.7.0
    ports:
      - "5601:5601"
    volumes:
      - ~/Projects/sh-valley/docker-conf/elasticstack/cluster/kibana.yml:/usr/share/kibana/config/kibana.yml
    environment:
      - ELASTICSEARCH_URL=http://es-node1:9200
    networks:
      - es-cluster-network
networks:
  es-cluster-network:
    driver: bridge
    ipam:
      driver: default
      config:
      - subnet: 172.21. 0. 0/ 16
        gateway: 172.21. 01.
Copy the code

The ES configuration file selects only one example as follows:

cluster.name: elasticsearch-cluster
node.name: es-node1
network.bind_host: 0.0. 0. 0
network.publish_host: 172.21. 010.
http.port: 9200
transport.tcp.port: 9300
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: true 
node.data: true  
discovery.zen.ping.unicast.hosts: ["172.21.0.10:9300"."172.21.0.11:9301"."172.21.0.12:9302"]
discovery.zen.minimum_master_nodes: 2
Copy the code

The configuration file for Kibana is as follows

server.name: kibana
server.host: "0"
elasticsearch.hosts: [ "http://es-node1:9200"."http://es-node2:9201"."http://es-node3:9202" ]
xpack.monitoring.ui.container.elasticsearch.enabled: false
Copy the code

Start the command

With the configuration file ready, you can start the cluster

$ docker-compose up -d
Copy the code

The startup process may be slow and cluster nodes can be seen from the command line

$curl http://localhost:9200/_cat/nodes 172.21.0.12 51 96 29 6.53 6.43 3.72 MD-es-node3 172.21.0.11 47 96 30 6.53 6.43 3.72 mDI-ES-node2 172.21.0.10 49 96 30 6.53 6.43 3.72 mdi * es-node1Copy the code

Docker-compose command can be used to start and stop the docker-compose service. If you do not want to keep related instances, you can use docker-compose down to shut down and delete the container.

The resources

1. elasticsearch document
2. Docker-compose custom network, fixed container IP address
3. Docker-compose Ports and Expose differences