This article source: making here | | GitEE, click here

I. Cluster environment construction

1. Environmental overview

ES version 6.3.2, cluster name ESMaster, VM centos7

ElasticSearch

• MySQL data is synchronized to the ES search engine in both full and incremental ways
• Build ElasticSearch middleware in Linux Centos7 environment
• SpringBoot2 integrates the ElasticSearch search engine framework

The ElasticSearch search engine needs to be managed in a cluster. It is common to search for billions of data in real time.

2. Cluster configuration

The configuration file

Vim/opt/elasticsearch - 6.3.2 / config/elasticsearch. YmlCopy the code

Active Node Configuration

Configure the cluster master node
cluster.name: esmaster
node.master: true

# node name
node.name: esnode1

# Development accessNetwork. The host: 0.0.0.0Copy the code

Secondary Node Configuration

Note that node.name is set to esnode2 and esnode3 respectively.

# cluster name
cluster.name: esmaster

# node name
node.name: esnode2

# Development accessNetwork. The host: 0.0.0.0IP address of the primary node
discovery.zen.ping.unicast.hosts: ["192.168.72.133"]
Copy the code

Memory access

vim /etc/sysctl.conf

# Add content
vm.max_map_count=262144

# to perform
sysctl -p
Copy the code

3. The cluster starts

Add and authorize user esroot.

/ opt/elasticsearch 6.3.2 / bin/elasticsearchCopy the code

Single service Viewing

ps -aux |grep elasticsearch
Copy the code

Viewing Cluster Status

http://localhost:9200/_cluster/health?pretty

{
  "cluster_name" : "esmaster".# cluster name
  "status" : "green".# Green: healthy, yellow: sub-healthy, red: sick
  "timed_out" : false.# timeout
  "number_of_nodes": 3.Number of nodes
}
Copy the code

Second, cluster mode test

1. Environment configuration

Dev environment

Configure a single node and select any node to test data writing.

spring:
  data:
    elasticsearch:
      # cluster name
      cluster-name: esmaster
      # single node
      # cluster-nodes: en-master:9300
      # cluster-nodes: en-node01:9300
      cluster-nodes: en-node02:9300
Copy the code

The test environment

Link the cluster environment for data reading tests.

spring:
  data:
    elasticsearch:
      # cluster name
      cluster-name: esmaster
      # Cluster node
      cluster-nodes: en-master:9300,en-node01:9300,en-node02:9300
Copy the code

Of course, all operations can be tested in a single node or cluster environment.

2. Instance object

Manage data object instances based on annotations.

import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.Document;

@Document(indexName = "usersearchindex",type = "usersearch")
public class UserSearch {

    // this column is the primary key of Elasticsearch
    @Id
    private Long id;
    private String userId;
    private String userName;
    private String sex;
}
Copy the code

3. Operating cases

Provides a data query operation and a data write operation.

import com.esearch.cluster.entity.UserSearch;
import org.springframework.stereotype.Service;
import javax.annotation.Resource;

@Service
public class UserSearchServiceImpl implements UserSearchService {

    @Resource
    private UserSearchRepository userSearchRepository ;

    @Override
    public String esInsert(Integer num) {
        for (int i = 0 ; i < num ; i++){
            UserSearch userSearch = new UserSearch() ;
            userSearch.setId(System.currentTimeMillis());
            userSearch.setUserId("Name"+i);
            userSearch.setUserName("ZSan"+i);
            userSearch.setSex("Male"+i);
            userSearchRepository.save(userSearch) ;
        }
        return "success" ;
    }

    @Override
    public Iterable<UserSearch> esFindAll (a){
        returnuserSearchRepository.findAll() ; }}Copy the code

Cluster console

Here is the cluster console based on the Kibana component.

1. Data list

You can view the list data in the Discover panel or continue your search.

A list of the query

A list of search

2. Development tools

You can run ElasticSearch commands on the dev_tools panel.

Check the cluster health status

GET /_cat/health? vCopy the code

Query all data

GET _search
{
  "query": {
    "match_all": {}}}Copy the code

Source code address

Making address GitEE, https://github.com/cicadasmile/data-manage-parent, https://gitee.com/cicadasmile/data-manage-parentCopy the code

Recommended reading: Data Source Management series