Docker installation ElasticSearch

1 Installation Instructions

ElasticSearch (Docker, Docker, Docker, Docker, Docker, Docker, Docker, Docker) ElasticSearch (Docker, Docker) But Chinese search, will achieve word segmentation integration, can use IK word segmentation. ElasticSearch uses Kibana for data visualization analysis, so we need to install Kibana as well as ElasticSearch and IK splitters.

Installation practice:

1:ElasticSearch is installed on a standalone server

2:IK word divider installed

3:Kibana installation
Copy the code

 

2 Docker installs ElasticSearch

The current version of ElasticSearch is 8.0 and has a lot of new features, performance and functionality. We recommend using the later version of ElasticSearch (7.12.1).

 

2.1 Network Creation

Docker will install Kibana on the same network segment as ElasticSearch. Docker will install Kibana on the same network segment as ElasticSearch.

docker network create itmentu-net
Copy the code

 

2.2 ElasticSearch installation

Install ElasticSearch as follows:

docker run -d \ --name elasticsearch \ -e "ES_JAVA_OPTS=-Xms512m -Xmx512m" \ -e "discovery.type=single-node" \ -v es-data:/usr/share/elasticsearch/data \ -v es-plugins:/usr/share/elasticsearch/plugins \ --privileged \ --network Itmentu-net \ -p 9200:9200 \ -p 9300:9300 \ ElasticSearch :7.12.1
Copy the code

Command description:

  • -e "cluster.name=es-docker-cluster": Sets the cluster name
  • - e "HTTP. Host = 0.0.0.0": Listening address that can be accessed from the Internet
  • -e "ES_JAVA_OPTS=-Xms512m -Xmx512m": Memory size
  • -e "discovery.type=single-node": Non-cluster mode
  • -v es-data:/usr/share/elasticsearch/data: Mounts and binds logical volumeselasticsearchData directory of
  • -v es-logs:/usr/share/elasticsearch/logs: Mounts and binds logical volumeselasticsearchLog directory of
  • -v es-plugins:/usr/share/elasticsearch/plugins: Mounts and binds logical volumeselasticsearchPlugins directory for
  • --privileged: Grants access to the logical volume
  • --network itmentu-net: Adds a name nameditmentu-netIn the network
  • -p 9200:9200: Port mapping configuration

 

Docker ElasticSearch download will be slow to download.

 

After the installation is complete, enter in your browser: http://192.168.211.130:9200/ can see elasticsearch response results:

 

The installation Kibana

We can operate ElasticSearch based on Http request, but it is difficult to operate based on Http. We can use Kibana to implement visual operation.

 

3.1 Kibana is introduced

Kibana is a free and open user interface that allows you to visualize Elasticsearch data and navigate through the Elastic Stack. You can easily do everything from tracking query load to understanding how requests flow through your entire application.

Kibana gives you the freedom to choose how your data is presented. But with Kibana’s interactive visualizations, you can start with a problem and see what you can discover.

 

The visual interface is as follows:

 

3.2 Kibana installation

Install Kibana using Docker. You only need to run the following command to install Kibana: ElasticSearch

docker The run - d \ - name kibana \ \ - e ELASTICSEARCH_HOSTS = http://192.168.211.130:9200 - network itmentu -.net \ -p 5601:5601 \ Kibana: 7.12.1
Copy the code

Command description:

  • --network itmentu-net: Adds a name nameditmentu-netNetwork, andelasticsearchIn the same network
  • - e ELASTICSEARCH_HOSTS = http://192.168.211.130:9200"Set:elasticsearchBecausekibanaWith theelasticsearchOn a network, so it can be accessed directly with the container nameelasticsearch, you can also write an IP address for access.
  • -p 5601:5601: Port mapping configuration

 

If there is no image during the installation, the image will be downloaded, and the effect is as follows:

The installation of Kibana takes a lot of time and requires patience to wait for the completion of the download and installation. If you want to know the real-time installation and running status of the service, you can check the log as follows:

docker logs -f kibana
Copy the code

If http://0.0.0.0:5601 is displayed in the log, the Kibana background service can be accessed as follows:

 

Visit http://192.168.211.130:5601 results are as follows:

You can click Add Data to Add sample data, as shown in the picture below. You can choose any sample data at will. In fact, you can choose no sample data.

 

3.3 Kibana Chinese configuration

We found that Kibana is an English panel, which looks not very convenient, but Kibana supports Chinese configuration, so we can configure Kibana into Chinese version, which is convenient for us to operate.

Switch to Chinese as follows:

# Enter container
docker exec -it kibana /bin/bash
Go to the configuration file directory
cd /usr/share/kibana/config
# Edit the file kibana.yml
vi kibana.yml
Add the following configuration on the last line
i18n.locale: zh-CN
# save exit
exit
And restart the container
docker restart kibana
Copy the code

 

Waiting for Kibana container startup, visit http://192.168.211.130:5601/ again effect is as follows:

 

4 INSTALLATION of IK word divider

We opened Kibana, clicked on the development tool and did as follows:

Enter the following operations to query word segmentation:

The above test code is as follows:

GET /_analyze
{
  "analyzer": "standard",
  "text": "IT disciple, open the door to your dreams!"
}
Copy the code

Using standard to IT disciples, open the door to your dreams! Do participles.

Word segmentation: Extracting words from a sentence or an article.

For ElasticSearch, the default word splitter is The Standard word splitter, but the standard word splitter does not meet the Chinese word splitter standard, so we need to install an extra Chinese word splitter.

 

4.1 Introduction to IK word divider

IK Analyzer is an open source, lightweight Chinese word segmentation toolkit developed based on the Java language. IKAnalyzer has been available in several large versions since its 1.0 release in December 2006. At first, it is based on the open source project Luence as the main application, combining dictionary segmentation and grammar analysis algorithm of Chinese phrase segmentation. IK Analyzer was developed as a common java-oriented phrase divider, independent of the Lucene project and providing a default optimized implementation of Lucene.

 

ElasticSearch – IK word segmentation for ElasticSearch – ElasticSearch – IK word segmentation for ElasticSearch – IK word segmentation for ElasticSearch

 

4.2 IK word segmentation configuration

Download the elasticSearch-Analysis-IK-7.12.1. zip package and decompress it to the following directory:

Insert elasticSearch – analysis-IK-7.12.1 into the plugins directory for elasticSearch. Therefore, you need to copy the files to the plugins directory of the Docker container.

The operation is as follows:

# change elasticSearch-Analysis-IK-7.12.1 to IK folder for easy configuration
mv Elasticsearch - analysis - ik - 7.12.1 ik
Copy the IK folder to elasticSearch
docker cp ik elasticsearch:/usr/share/elasticsearch/plugins
# restart container
docker restart elasticsearch
Copy the code

The operation effect is as follows:

 

4.3 Word segmentation test

IK word divider contains two modes:

  • Ik_smart: minimum sharding

  • Ik_max_word: the finest slice

 

After installing the IK tokenizer, we can use the IK tokenizer test. The test code is as follows:

GET /_analyze
{
  "analyzer": "ik_max_word",
  "text": "IT disciple, open the door to your dreams!"
}
Copy the code

The test results are as follows:

We can find that the word segmentation effect of Chinese is relatively good, but there are some shortcomings, for example, we want the dream door to be a word, but we want it not to be recognized as a word, and how to achieve it?

 

4.4 IK User-defined Dictionary

The IK tokenizer supports custom dictionaries, including custom tokenizers and custom deactivation tokenizers, and is very simple to operate. Let’s implement custom dictionaries and deactivated dictionaries next.

 

4.4.1 User-defined Dictionary

To customize a dictionary, you need to create your own dictionary and then reference your own dictionary.

1:Create a dictionary
2:Refer to the dictionary
Copy the code

 

1) Create a dictionary

Create your own dictionary in config, such as itmentu_ext.dic, and add your own words to the file as follows:

In itmentu_ext.dic, we have added the custom dictionary dream door. This is to create a dictionary, if the next several custom times, need to add a new line, here must pay attention to the Chinese word segmentation encoding format utF-8.

 

2) Use a dictionary

Modify the config/IKAnalyzer. CFG. XML references to create your own itmentu_ext. Dic dictionary, the configuration is as follows:

The code above is as follows:


      
<! DOCTYPEproperties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
	<comment>IK Analyzer extension configuration</comment>
	<! Users can configure their own extended dictionary here -->
	<entry key="ext_dict">itmentu_ext.dic</entry>
	 <! -- Users can configure their own extension stop word dictionary here -->
	<entry key="ext_stopwords"></entry>
	<! -- User can configure remote extension dictionary here -->
	<! -- <entry key="remote_ext_dict">words_location</entry> -->
	<! -- Users can configure the remote extension stop word dictionary here -->
	<! -- <entry key="remote_ext_stopwords">words_location</entry> -->
</properties>
Copy the code

We will change good file upload again elasticsearch container/usr/share/elasticsearch/plugins directory, restart elasticsearch container.

The operation is as follows:

Copy the uploaded config to the server
Copy the IK folder to elasticSearch
docker cp ik elasticsearch:/usr/share/elasticsearch/plugins
# restart container
docker restart elasticsearch
Copy the code

 

In the use of Kibana test results are as follows: http://192.168.211.130:5601/app/dev_tools#/console

 

4.4.2 Customizing disabling terms

Like a custom dictionary, you need to create your own dictionary and reference your own dictionary.

1:Create a dictionary
2:Refer to the dictionary
Copy the code

 

1) Create a dictionary

Create your own outage dictionary in config, such as itmentu_stop.dic, and add custom outage words to the file as follows:

 

2) Use a dictionary

Modify the config/IKAnalyzer. CFG. XML references to create your own itmentu_stop. Dic stop using dictionary, configuration is as follows:

The code above is as follows:


      
<! DOCTYPEproperties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
	<comment>IK Analyzer extension configuration</comment>
	<! Users can configure their own extended dictionary here -->
	<entry key="ext_dict">itmentu_ext.dic</entry>
	 <! -- Users can configure their own extension stop word dictionary here -->
	<entry key="ext_stopwords">itmentu_stop.dic</entry>
	<! -- User can configure remote extension dictionary here -->
	<! -- <entry key="remote_ext_dict">words_location</entry> -->
	<! -- Users can configure the remote extension stop word dictionary here -->
	<! -- <entry key="remote_ext_stopwords">words_location</entry> -->
</properties>
Copy the code

We will change good file upload again elasticsearch container/usr/share/elasticsearch/plugins directory, restart elasticsearch container.

The operation is as follows:

Copy the uploaded config to the server
Copy the IK folder to elasticSearch
docker cp ik elasticsearch:/usr/share/elasticsearch/plugins
# restart container
docker restart elasticsearch
Copy the code

 

In the use of Kibana test results are as follows: http://192.168.211.130:5601/app/dev_tools#/console

We can see that there are no more participles with you, indicating that the suspension of participles is also in effect.

 

5 ElasticSearch cluster installation

Deploying an ES cluster can be done directly with Docker-compose, but your Linux VM must have at least 4 gb of memory.

Docker-compose docker-compose docker-compose docker-compose docker-compose docker-compose docker-compose docker-compose docker-compose docker-compose docker-compose

version: '3.8'
services:
  es01:
    image: Docker. Elastic. Co/elasticsearch/elasticsearch: 7.12.1
    container_name: elasticsearch01
    environment:
      - node.name=elasticsearch01
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=elasticsearch01,elasticsearch01
      - cluster.initial_master_nodes=elasticsearch01,elasticsearch01,elasticsearch01
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: - 1
        hard: - 1
    volumes:
      - data01:/usr/share/elasticsearch/data
    ports:
      - 9200: 9200
    networks:
      - elastic
  es02:
    image: Docker. Elastic. Co/elasticsearch/elasticsearch: 7.12.1
    container_name: elasticsearch02
    environment:
      - node.name=elasticsearch02
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=elasticsearch01,elasticsearch03
      - cluster.initial_master_nodes=elasticsearch01,elasticsearch02,elasticsearch03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: - 1
        hard: - 1
    volumes:
      - data02:/usr/share/elasticsearch/data
    networks:
      - elastic
  es03:
    image: Docker. Elastic. Co/elasticsearch/elasticsearch: 7.12.1
    container_name: elasticsearch03
    environment:
      - node.name=elasticsearch03
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=elasticsearch01,elasticsearch02
      - cluster.initial_master_nodes=elasticsearch01,elasticsearch02,elasticsearch03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: - 1
        hard: - 1
    volumes:
      - data03:/usr/share/elasticsearch/data
    networks:
      - elastic

volumes:
  data01:
    driver: local
  data02:
    driver: local
  data03:
    driver: local

networks:
  elastic:
    driver: bridge
Copy the code