When after completing our ES cluster, how can I see the state of each node in the cluster and the master node and the health situation, the following interpretation of the use the curl command to interact with ES cluster, respectively to track the status of the master node and cluster, and create the index view index, view the fragmentation and on ES cluster operations such as query request.

CURL CURL

RESTful API:
		curl  -X<VERB> '
      
       ://
       
        :
        
         /
         
          ? 
          
           '
          
         
        
       
       -d '<BODY>'-x: specifies the request mode. <VERB> : GET,POST,PUT, or DELETE The request mode is GET, the submit change is POST, the upload file is PUT, and the DELETE operation is DELETE. <PROTOCOL> : indicates the PROTOCOL, which is usually HTTP. Hostname <PORT> : indicates the host PORT. <PATH> : indicates the PATH after the host PORT. The following paths are available: The /_cat, /_search, /_cluster /_cat path covers most of the information content. /_search searches for all indexes and document types. Request body in JSON formatCopy the code

Elasticseearch basic query statement

/ / check _cat support information kibana: GET / _cat bash: curl - XGET -u elastic: 26 tbktgolycyzd2ppisw'http://192.168.31.215:9201/_cat'Kibana: GET /_cat/master? v bash: curl -XGET -u elastic:26tBktGolYCyZD2pPISW'http://192.168.31.215:9201/_cat/master? v'Kibana: GET /_cat/nodes? v bash: curl -XGET -u elastic:26tBktGolYCyZD2pPISW'http://192.168.31.215:9201/_cat/nodes? v'Kibana: GET /_cat/indices? v bash: curl -XGET -u elastic:26tBktGolYCyZD2pPISW'http://192.168.31.215:9201/_cat/indices? v'Kibana: GET /_cat/ movies? v bash: curl -XGET -u elastic:26tBktGolYCyZD2pPISW'http://192.168.31.215:9201/_cat/indices/movies? v'Kibana: GET /_cat/shards? v bash: curl -XGET -u elastic:26tBktGolYCyZD2pPISW'http://192.168.31.215:9201/_cat/shards? v'Kibana: GET /_cat/shards/movies? v bash: curl -XGET -u elastic:26tBktGolYCyZD2pPISW'http://192.168.31.215:9201/_cat/shards/movies? v'Kibana: GET /_cat/health? v kibana: GET _cluster/health bash: curl -XGET -u elastic:26tBktGolYCyZD2pPISW'http://192.168.31.215:9201/_cat/health? v'
bash: curl -XGET -u elastic:26tBktGolYCyZD2pPISW 'http://192.168.31.215:9201/_cluster/health? pretty'Kibana: GET /_cat/plugins? v bash: curl -XGET -u elastic:26tBktGolYCyZD2pPISW'http://192.168.31.215:9201/_cat/plugins? v'/ / see all index documents kibana: total GET _all / _count bash: curl - XGET -u elastic: 26 tbktgolycyzd2ppisw'http://192.168.31.215:9201/_all/_count? pretty'/ / look at the specified index document total kibana: GET movies / _count bash: curl - XGET -u elastic: 26 tbktgolycyzd2ppisw'http://192.168.31.215:9201/movies/_count? pretty'/ / to see all the template kibana: GET _cat/templates bash: curl - XGET -u elastic: 26 tbktgolycyzd2ppisw'http://192.168.31.215:9201/_cat/templates? v'Kibana: GET /_cat/indices? v&health=green bash: curl -XGET -u elastic:26tBktGolYCyZD2pPISW'http://192.168.31.215:9201/_cat/indices? v&health=green'/ / see movies index metadata kibana: GET movies bash: curl - XGET -u elastic: 26 tbktgolycyzd2ppisw'http://192.168.31.215:9201/movies? pretty'Kibana: GET _cat/indices? v&s=docs.count:desc bash: curl -XGET -u elastic:26tBktGolYCyZD2pPISW'http://192.168.31.215:9201/_cat/indices? v&s=docs.count:desc'/ / check each index memory size and sorting kibana: bash: curl - XGET -u elastic: 26 tbktgolycyzd2ppisw'http://192.168.31.215:9201/_cat/indices? v&h=i,tm&s=tm:desc'
Copy the code

Elasticsearch CURD grammar

CURD Request way The main body instructions
Create PUT /index/_create/id Specify a Document ID to create the Document, or fail if the ID already exists
Create POST /index/_create/id Specify a Document ID to create the Document, or fail if the ID already exists
Create POST /index/_doc The ID is automatically generated and will not be repeated. Multiple documents will be created after repeated submission, and the document version is 1
Index PUT /index/_doc/id If the ID does not exist, create a new document. If the ID does exist, delete the existing document and create a new document, version +1, with the same ID
Index POST /index/_doc/id If the ID does not exist, create a new document. If the ID does exist, delete the existing document and create a new document, version +1, with the same ID
Read GET /index/_doc/id View the Document with Document ID 1
Update POST /index/_doc/id The document must exist; otherwise, the update fails. The field can only be incremented, not reduced. The value of the field can be changed at will, and the version can be increased by 1
Delete Delete /index/_doc/id Document must exist, otherwise delete returns “not_found”
Delete Delete /index If you delete an index, all documents in the index will be deleted. The index to be deleted must exist, otherwise “404” is returned.

Create Creates indexes and documents

When creating a document, Elasticsearch can automatically generate the document ID or specify the document ID.

  1. By calling thePOST /index/_docSyntax, the system automatically generates the Document ID. Elasticsearch will create multiple duplicate DOCUMENTS with different ids if the Document ID is not the same.
  2. By calling thePOST /index/_create/idorPUT /index/_create/idSyntax, then manually specify the generated Document ID, this method, if we specify the ID of the Document already exists, then create failed, create the Document anyway.

The instance automatically generates the Document ID

# Kibana grammar
POST /index/_doc
{
  "name": "WeiLiang Xu"."Blogs": "abcops.cn"."Is male": true."age"25} :# Bash syntax
curl -XPOST -u elastic:26tBktGolYCyZD2pPISW -H "Content-Type:application/json" 'http://192.168.31.215:9201/index/_doc? pretty' -d ' { "name": "WeiLiang Xu", "Blogs": "abcops.cn", "Is male": true, "age": 25 }Copy the code

The POST request specifies a Document ID

#Kibana POST request syntax
POST /index/_create/1
{
  "name": "WeiLiang Xu"."Blogs": "abcops.cn"."Is male": true."age"25} :#Bash POST request syntax
curl -XPOST -u elastic:26tBktGolYCyZD2pPISW -H "Content-Type:application/json" 'http://192.168.31.215:9201/index/_create/1? pretty' -d ' { "name": "WeiLiang Xu", "Blogs": "abcops.cn", "Is male": true, "age": 25 }'
Copy the code

PUT Specifies the Document ID

#Kibana PUT request syntax
PUT /index/_create/1
{
  "name": "WeiLiang Xu"."Blogs": "abcops.cn"."Is male": true."age"25} :#Bash PUT request syntax
curl -XPUT -u elastic:26tBktGolYCyZD2pPISW -H "Content-Type:application/json" 'http://192.168.31.215:9201/index/_create/1? pretty' -d ' { "name": "WeiLiang Xu", "Blogs": "abcops.cn", "Is male": true, "age": 25 }'
Copy the code

Index Creates indexed documents or updates

The/Index /_doc/id syntax is used to specify the Document ID. If the Document ID does not exist, it will be created as a new Document. If the Document ID does exist, it will be created as a new Document. Then delete the existing ID and create a new Document, Document Version will proceed to + 1. The Index mode works as follows: If a document already exists, delete the original document first and then add a new document with version +1.

The instance

# Kibana operation
POST /index/_doc/6              The #PUT syntax simply changes POST to PUT
{
  "name": "WeiLiang Xu"."Blogs": "abcops.cn"."Is male": true."age"25} :# Bash operation
curl -XPUT -u elastic:26tBktGolYCyZD2pPISW -H "Content-Type:application/json" 'http://192.168.31.215:9201/index/_doc/6? pretty' -d ' { "name": "WeiLiang Xu", "Blogs": "abcops.cn", "Is male": true, "age": 25 }'
Copy the code

Read Query document

Read reads the contents of a document. Read uses the HTTP request form GET.

# Kibana operation
GET /index/_doc/1                   Select * from index where Document ID = 1

# Bash operation
curl -XGET -u elastic:26tBktGolYCyZD2pPISW 'http://192.168.31.215:9201/index/_doc/1? pretty'

# See the top of this article for more GET syntax
Copy the code

Update Update documents

Unlike Index, the Update method does not delete the original document. Instead, it actually updates the data. Update The document to be updated must actually exist. The Update can only make incremental changes to the field value, but cannot reduce the Update of the document field. If you need to reduce fields, use Index to manipulate the document.

The instance

# Kibana operation
POST /weiliang/_update/1
{
  "doc": {
  "name": ["weiliang Xu"."xueiliang"]."JobS": "Linux DevOps"."Age": 25."gender": "man"}}# Bash operation
curl -XPOST -u elastic:26tBktGolYCyZD2pPISW -H "Content-Type:application/json" 'http://192.168.31.215:9201/weiliang/_update/1? pretty' -d ' { "doc": { "name": ["weiliang Xu","xueiliang"], "JobS": "Linux DevOps", "Age": 25, "gender": "man" } }'
Copy the code

Delete Deletes a document

Delete can specify the Document ID to Delete the Document, or can directly Delete the index, Delete the index, the Document is deleted.

# Kibana operation
DELETE /weiliang/_doc/1         Delete the specified document
DELETE /weiliang                Drop index

# Bash operation
curl -XDELETE -u elastic:26tBktGolYCyZD2pPISW -H "Content-Type:application/json" 'http://192.168.31.215:9201/weiliang/_doc/1? pretty'
curl -XDELETE -u elastic:26tBktGolYCyZD2pPISW -H "Content-Type:application/json" 'http://192.168.31.215:9201/weiliang? pretty'
Copy the code

Bulk API Operations are performed in batches

  1. Supports operation on different indexes in a single API call
  2. Four operations are supportedIndex,Create,Update,Delete
  3. You can specify an Index in the URI or in the requested Playoad
  4. The failure of a single statement does not affect subsequent operations
  5. The return result contains the execution result of each entry

The instance

#Kibana operation POST _bulk {"create" : { "_index" : "bulk_index"."_id" : "1"}}// The document bulk_index and ID 1 are created
{ "Job" : "Linux Ops" }                                             // Document contents are field "Job" value "Linux Ops"
{ "delete" : { "_index" : "bulk_index"."_id" : "2"}}// Delete the document whose index is Bulk_index with ID 2. Since we have no document whose ID is 2, this execution returns not_found, but it does not affect the subsequent statement execution
{ "update" : { "_id": "1"."_index" : "bulk_index"}}// Incrementally update the file whose ID is 1 in Bulk_index. Note that _id comes first and _index comes second
{ "doc" : {"name" : "xuweiliang"}} {"index" : {"_index" : "bulk_index"."_id" : "1"}}//Index operates the document whose ID is 1 of bulk_index and changes the content as follows
{ "name" : "xuweiliang" }
{ "create" : { "_index" : "bulk_index"."_id" : "2"}}// A document with ID 2 is created in bulk_index
{ "name" : "xuweiliang" , "Age" : 25 }
{ "delete" : { "_index" : "bulk_index"."_id" : "2"}}// The document whose ID is 2 in bulk_index is deleted
Copy the code

Each statement that operates in bulk using the BULK API returns the following result

MGET Reads data in batches

Elasticsearch provides a batch read mode for MGET, which reduces network connection overhead. To improve performance

The instance

GET _mget
{
  "docs": [//docs is in mGET format
    {
      "_index": "bulk_index".// Specify the index of the document to read
      "_id" : 1                             // Specify the ID of the document to read
    },
    {
      "_index": "bulk_index".// different ids in the same index are read together
      "_id" : 2
    },
    {
      "_index": "index".// different ids in different indexes are read together
      "_id" : 1}}]Copy the code

The results are as follows:

MSEARCH Batch query

Multi Serach API is a syntax for conditional matching query GET /

/_msearch

Mserach gets multiple search results from a single API. The request format is similar to the bulk API format and uses newline delimited JSON (NDJSON) format. The last line of data must end with a newline character \n. Each newline character can be preceded by a carriage return character \r. When sending a request to this endpoint, the Content-Type header should be set to Application/X-NDJSON

Path argument (optional, string) A comma-separated list or wildcard expression of index names used to restrict requests.

Request body 1. Aggregations (Optional, object) Specify aggregations.

2. From Specifies the offset from the (optional, integer) origin document. Default is 0.

Max_concurrent_searches maximum queries (optional, integer) specifies the maximum number of concurrent searches that the multi-search API will perform, based on the number of data nodes and the size of the default search thread pool.

Max_concurrent_shard_requests (optional, integer) Specifies the maximum number of concurrent sharding requests that will be executed on each node for each sub-search request. This parameter is used to protect a single request against cluster overload (for example, the default request will hit all indexes in the cluster, and if the number of shards per node is high, the shard request may be rejected). The default value is 5. In some cases, parallelism cannot be achieved for concurrent requests, so this protection will result in performance degradation. For example, in an environment where only a small number of concurrent search requests are expected, it might be helpful to increase this value to a larger number.

5. Preference (Optional, string) specifies the node or shard on which the operation should be performed. The default is random.

6. Query (Optional, query object) Uses the query DSL to define the search definition.

7. Routing (Optional, string) targets the specified primary shard.

8. Search_type (Optional, string) Type of search operation. Available options:

  • query_then_fetch
  • dfs_query_then_fetch

9. Size (optional, integer) Number of clicks to return. The default value is 10.

The response body Responses (array) includes the search response and status code for each search request that was sequentially matched in the original multiple search requests. If a particular search request fails completely, ERROR returns an object with a message and the corresponding status code in place of the actual search response.

The instance header section contains one or more indexes to search for, search_type, preferences, and routes. This body contains typical search body requests (including query, aggregation, source, size, and so on).

$ cat requests
{"index" : "test"."index"}              
{"query" : {"match_all" : {}}, "from" : 0."size" : 10}
{"index" : "test"."search_type" : "dfs_query_then_fetch"}
{"query" : {"match_all": {} {} {}}"query" : {"match_all" : {}}}

{"query" : {"match_all" : {}}}
{"search_type" : "dfs_query_then_fetch"}
{"query" : {"match_all" : {}}}
Copy the code

Common error return

The problem why
Unable to connect The network is faulty or the cluster is down
Connection cannot be closed The network is faulty or the node is faulty
429 The Cluster is too Busy
4xx Request size error
500 Cluster internal error

Does more articles and materials | click behind the text to the left left left 100 gpython self-study data package Ali cloud K8s practical manual guide] [ali cloud CDN row pit CDN ECS Hadoop large data of actual combat operations guide the conversation practice manual manual Knative cloud native application development guide OSS Operation and maintenance field manual Cloud native architecture white paper Zabbix enterprise-level distributed monitoring system source document Linux&Python self-study information package 10G interview questions