preface

This article describes how to solve various problems during the use of ElasticSearch.

The ElasticSearch environment is not installed properly

1, Max Virtual memory areas VM. Max_map_count [65530] is too low, increase to at least [262144]

Reason: The memory limit is too small! Solution: Change the maximum memory limit!

Modify the sysctl.conf file

vim /etc/sysctl.conf
Copy the code

Add the following configuration at the end:

vm.max_map_count = 655360
vm.swappiness=1
Copy the code

Then save and exit and enter the following command for it to take effect

   sysctl -p
Copy the code

Use the following command to view:

tail -3 /etc/sysctl.conf
Copy the code

Photo examples:

2, Max number of threads [2048] for user [elastic] is too low, increase to at least [4096]

Reason: Too few thread limits! Solution: Change the maximum number of threads limit!

Modify the 90-nproc.conf file

  vim /etc/security/limits.d/90-nproc.conf 
Copy the code

Note: The file name of 90-nproc.conf may be different on different Linux servers. You are advised to check the file name in /etc/security/limits.d/ before changing the file name.

The following content

    soft nproc 2048
Copy the code

Modified to

  soft nproc 4096
Copy the code

Use the following command to view:

tail -3 /etc/security/limits.d/90-nproc.conf
Copy the code

3， max file descriptors [65535] for elasticsearch process likely too low, increase to at least [65536]

Cause: Too few open files! Solution: Modify the most open file number!

Modify the limits. Conf

vim /etc/security/limits.conf
Copy the code

Add the following at the end:

   * hard nofile 65536
   * soft nofile 65536
   elastic soft memlock unlimited
   elastic hard memlock unlimited
Copy the code

4, ERROR:bootstrap checks failed

Cause: The memory is not locked. Solution: Add bootstrap.memory_lock: true configuration to elasticSearch. yml configuration file on the machine reporting the error!

ElasticSearch usage problem

1. The number of ES query subscripts is too large

Index * pagesiz is greater than the maximum value of ES (1w).

Solutions:

A, can be set through the URL, convenient and quick without restarting. As follows:

The curl - XPUT http://127.0.0.1:9200/book/_settings - d '{" index ": {" max_result_window" : 200000000}}'Copy the code

Note:

  • 1. The size cannot exceed the value of the index.max_result_window parameter. The default value is 10,000.
  • 2. To search for pages, use the from size combination. From indicates the number of lines to start from, and size indicates how many documents to query. From defaults to 0 and size defaults to 10

2. Set through the configuration file:

{ "order": 1, "template": "index_template*", "settings": { "index.number_of_replicas": "0", "index.number_of_shards": "1", "index.max_result_window": 2147483647 }
Copy the code

2, the shard of ES is not allocated

1. Locate problem sharding

The CAT API of ES can be used to analyze unallocated shard information and the cause of unallocated shard information

curl -XGETlocalhost:9200/_cat/shards? h=index,shard,prirep,state,unassigned.reason| grepUNASSIGNEDCopy the code

The returned information includes the index name, shard number, whether the shard is a master shard or a duplicate shard, and the reason for unallocation

If it is a shard with a deleted index, you can use the delete command to delete the index:

curl -XDELETE 'localhost:9200/index_name/'
Copy the code

2. Cluster purposeful deferred allocation

When a node leaves the cluster, the primary node temporarily delays redistributing fragments to reduce resource waste caused by rebalancing fragments. In this case, if the source node rejoins the cluster within a certain period of time (1 minute by default), the fragment information can be recovered. In this case, the log information is as follows:

[TIMESTAMP][INFO][cluster.routing] [MASTER NODE NAME]delaying allocation for [54] unassigned shards, next check in [1m]
Copy the code

You can manually modify the delay time:

curl -XPUT'localhost:9200/<INDEX_NAME>/_settings' -d '
{
    "settings": {
     "index.unassigned.node_left.delayed_timeout": "30s"
    }
}'
 
Copy the code

If you need to change thresholds for all indexes, you can replace <INDEX_NAME> with _all

3. The number of fragments is too high and the number of nodes is insufficient

The master node will not allocate the master shard and the replica shard to the same node, and also will not allocate two replica nodes to the same node, so when there are not enough nodes to allocate the shard, there will be an unallocated state. To avoid this situation, the relationship between the number of nodes and the number of copies should be N>=R+1 (where N is the number of nodes and R is the number of copies). This can be resolved by increasing the number of nodes or reducing the number of replicas.

4. Fragments need to be redistributed

Fragment reallocation is enabled by default, but it may be disabled but forgotten to be enabled for some reason. After this function is enabled, fragments are redistributed.

To enable the reassign command:

curl -XPUT 'localhost:9200/_cluster/settings'-d
'{ "transient":
  {"cluster.routing.allocation.enable" : "all" 
  }
}'
Copy the code

5. The fragmented data does not exist in the cluster

The data does not exist in the cluster.

  • 1. Restore the source node with 0 shards and add it to the cluster (no forced redistribution of primary shards)
  • 2. Use the Reroute API to force shard redistribution
curl -XPOST'localhost:9200/_cluster/reroute' -d '{ "commands" :
  [ { "allocate_empty_primary" : 
      { "index" :"constant-updates", "shard" : 0, "node":"<NODE_NAME>", "accept_data_loss": "true" }
  }]
}'
 
Copy the code

  • 3, rebuild index from original data or restore from backup snapshot

6. The disk space is insufficient

Generally, when the disk usage reaches 85%, the primary node does not allocate fragments to the node. You can run the following command to view the disk usage:

curl -s 'localhost:9200/_cat/allocation? v'Copy the code

If you have a lot of disk space and 85% utilization is a waste, Can be set by cluster. Routing. Allocation. Disk. Watermark. Low and (or) cluster. Routing. Allocation. The disk. The watermark. High to increase the value:

curl -XPUT 'localhost:9200/_cluster/settings'-d
'{
    "transient": {  
     "cluster.routing.allocation.disk.watermark.low":"90%"    
    }
}'
Copy the code

Note: If cluster restart is required to be effective, change TRANSIENT to Persistent; In the ES setting, percentage refers to used space and byte value refers to unused space

7. Multiple versions

Multiple VERSIONS of ES exist in the ES cluster, causing incompatibility

3, the ES index library state is only readable

Cause: Data was written to the ES index library

retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"block
Copy the code

The exception.

Cause: The ES cluster changed the state of these index libraries to make them read-only and unable to write, possibly because the service’s disk was nearing full. The root solution is to increase the disk or clean the disk of useless data.

Temporary method whose modifier state is readable.

PUT /_all/_settings
{
"index.blocks.read_only_allow_delete": null
}
Copy the code

4. ES cluster appears red

Run the GET /_cluster/health command to check the health status of the cluster. If unassigned Shards is displayed, the fragment is missing. You can run GET /_cat/shards to check the fragment status and find the missing fragment. If the main shard is missing because of cluster downtime, it can be solved by adding nodes and automatic shard. If because of lack of data, that is, the deputy shard was lost, in which case the data is unable to recover, can choose according to the situation, if it is a important index of the library data, you can use the reindex the data migration, can solve the clusters of red, but the lack of data cannot be recovered. If it is an unimportant index library, delete it and rebuild it.

5. GC collection in ES cluster fails

Solution: 1. Upgrade the JDK version to a higher version than 1.8_145. 2. Reduce GC collection frequency.

6, fissure

master not discovered or elected yet, an election requires a node with id
Copy the code

Solution: 1. Specify the master node in ES7.x configuration, and go to 2. Delete data of the original data and delete all data entries

Es7.x Configuration example:

Cluster. name: pancm node.name: node-3 network.host: 192.168.8.160 node.master: false node.data: True discovery.seed_hosts: ["192.168.9.238","192.168.8.181","192.168.8.160"] ["192.168.9.238"] network.tcp.keep_alive: true network.tcp.no_delay: true transport.tcp.press: true cluster.routing.allocation.cluster_concurrent_rebalance: 16 cluster.routing.allocation.node_concurrent_recoveries: 16 cluster.routing.allocation.node_initial_primaries_recoveries: 16 path.data: /home/elastic/masternode/data path.logs: /home/elastic/masternode/logsCopy the code

Logstash usage issues

1,logstash: Could not execute action: PipelineAction::Create, action_result: false

Solution: Oblique rod with “/”

2, logstash: object mapping for [host] tried to parse field [host] as object, but found a concrete value

Solution: Add to filter:

  #mutate {
 #   rename => { "[host][name]" => "host" }
 # }
mutate {
      rename => { "host" => "host.name" }
    }
Copy the code

ElasticSearch Java code issue

1.. Es7.x version query error:

org.elasticsearch.action.search.SearchRequest.isCcsMinimizeRoundtrips()Z
Copy the code

Solution: Missing jar package, complete POM configuration as follows:

<dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-high-level-client</artifactId> < version > 7.3.2 < / version > < exclusions > < exclusion > < groupId > org. Elasticsearch < / groupId > <artifactId>elasticsearch</artifactId> </exclusion> <exclusion> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-client</artifactId> </exclusion> </exclusions> </dependency> <dependency> < the groupId > org. Elasticsearch < / groupId > < artifactId > elasticsearch < / artifactId > < version > 7.3.2 < / version > < / dependency > <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-client</artifactId> The < version > 7.3.2 < / version > < / dependency >Copy the code

other

Reference: blog.csdn.net/laoyang360/… Blog.csdn.net/u013673976/… www.datadoghq.com/blog/elasti… Blog.csdn.net/kezhen/arti…

ElasticSearch Combat Series

  • Kinaba for ElasticSearch
  • ElasticSearch DSL statement for ElasticSearch
  • ElasticSearch: JAVA API for ElasticSearch
  • ElasticSearch: ElasticSearch
  • Metric Aggregations for ElasticSearch
  • ElasticSearch: Logstash Quick start
  • ElasticSearch: Logstash: ElasticSearch
  • Filebeat: ElasticSearch, ElasticSearch, ElasticSearch, ElasticSearch
  • Install the ELK log system for ElasticSearch
  • ElasticSearch: Cold/hot separation architecture for ElasticSearch

Original is not easy, if you feel good, I hope to give a recommendation! Your support is the biggest motivation for my writing! Copyright: www.cnblogs.com/xuwujing CSDN blog.csdn.net/qazwsxpcm Nuggets: juejin.cn/user/365003… Personal blog: www.panchengming.com