This is the 16th day of my participation in the August More Text Challenge. For details, see:August is more challenging

🌈 past review

Thank you for reading, I hope it can help you, if there are flaws in the blog, please leave a message in the comment area or add my private chat in the profile of the home page, thank you for your advice. I’m XiaoLin, a boy who can write bugs and rap

💘10 minutes, 6 points, SpringBoot integrated Swagger the whole thing 💘
🌹 the most comprehensive summary of the most common Linux back-end required commands (super comprehensive! Super detailed!) Collect this one is enough! 🌹
🌈 iS MySQL really CRUD? ✨ to see the difference between 2K and 12K (bottom)

Four, Kibana

4.1 overview,

Kibana is an open source analytics and visualization platform for Elasticsearch. With Kibana, you can query, view, and interact with data stored in the ES index. With Kibana, you can perform advanced data analysis and view data in the form of charts, tables, and maps. Kibana is strictly consistent with the Elasticsearch release.

4.2 Download and install Kibana

Download Kibana

Download link

Install and download Kibana

The RPM - the ivh kibana - 6.2.4 - x86_64. RPMCopy the code

Find out where Kibana was installed

find / -name kibana
Copy the code

Edit the Kibana profile

vim /etc/kibana/kibana.yml
Copy the code

Modify the configuration

#ES server host addressServer host: "192.168.202.200"#ES server AddressElasticsearch. Hosts: [" http://192.168.202.200:9200 "]Copy the code

Start the kibana

#Start the kibana
systemctl start kibana
#Stop kibana
systemctl stop  kibana
#Check 1Kibana status
systemctl status kibana
Copy the code

Access to the test

The default port for Kibana is 5601. Use host: port for direct access.

Basic operation of Kibana

5.1 Basic Index Operations

5.1.1 creating an index

put /student/
Copy the code

5.1.2. Deleting an Index

delete /student
Copy the code

5.1.3. Delete all indexes

delete /*
Copy the code

5.1.4 Viewing All Index Information

get /_cat/indices? vCopy the code

5.2 Basic Operations of the type

5.2.1 Create a type

Create the /shop index and create the type (product). This way of creating the type requires that the index not exist.

PUT /shop             
{
  "mappings": {
"product": { "properties": { "title": { "type": "text" }, "name": { "type": "text" }, "age": { "type": "integer" }, "created": { "type": "date" } } } } }Copy the code

5.2.1 Viewing types

# Syntax formatGet/Index name /Get /shop/_mapping/product
Copy the code

5.3 Basic Operations of documents

5.3.1 Adding documents

# / index/type /idPUT /school/student/1 {"name":"xiaolin", "age":23, "bir":"2012-12-12", "content":" this is a better student "}Copy the code

5.3.2 Querying documents

GET /school/student/1

# Here is the return result{"_index": "school", "_type": "student",
  "_id": "1", "_version": 1,
  "found": true,
  "Xiaolin _source ": {" name" : ""," age ": 23," bir ":" 2012-12-12 ", "content" : "this is a better student"}}Copy the code

5.3.3 Deleting a Document

DELETE /school/student/1

# Here is the return result{"_index": "school", "_type": "student",
  "_id": "1", "_Version ": 2, "result": "deleted", "# deleted"_shards": { "total": 2, "successful": 1, "failed": 0 }, "_seq_no": 1, "_primary_term": 1
}
Copy the code

5.3.4 Update the document

5.3.4.1 First Method (Update the original data)

POST /school/student/1/_update { "doc":{ "name":"xiaohei" } }Copy the code

5.3.4.2 Second Method (Adding New Data)

 POST /school/student/1/_update { "doc":{ "name":"xiaohei", "age":11, "dpet":"hello world" } }Copy the code

Query advanced retrieval

6.1 Retrieval mode

ES officially provides two retrieval methods:

One is to search by URL parameters, such as GET/index/type /_search? parameter
Domain Specified Language: GET/index/type /_search {}

Officially, the second way is more recommended. The second way is based on the transmission of JSON as the request body format to interact with ES, which is more powerful and concise

6.2 Preparing Data

# create index and specify type PUT /ems {"mappings": {"emp": {"properties": {"name": {"type":"text"
        },
        "age": {"type":"integer"
        },
        "bir": {"type":"date"
        },
        "content": {"type":"text"
        },
        "address": {"type":"keyword"}}}}} # PUT /ems/emp/_bulk {"index": {}} {"name":"Black"."age":23."bir":"2012-12-12"."content":"Choosing a good MVC framework for your development team is difficult, and it takes a high level of experience to choose among the many viable alternatives."."address":"Beijing"}
  {"index": {}} {"name":"Wang Xiaohei"."age":24."bir":"2012-12-12"."content":"The Spring framework is a layered architecture made up of seven well-defined modules. The Spring module is built on top of a core container that defines how beans are created, configured, and managed."."address":"Shanghai"}
  {"index": {}} {"name":"Zhang Xiao Wu"."age":8."bir":"2012-12-12"."content":"Spring Cloud is a Java language microservices framework that relies on Spring Boot for rapid development, continuous delivery, and easy deployment. "The Spring Cloud has a huge number of components that cover all aspects of microservices, and it's being perfected by the open source community Spring and Netflix and Pivotal."."address":"Wuxi"}
  {"index": {}} {"name":"win7"."age":9."bir":"2012-12-12"."content":"Spring aims to simplify Java development across the board. This is bound to lead to more explanations of how Spring simplifies Java development."."address":"Nanjing"}
  {"index": {}} {"name":"Mei Chao Feng"."age":43."bir":"2012-12-12"."content":"Redis is an open source, net-based, memory-based, persistent, logging, key-value database written in ANSI C, and provides apis in multiple languages."."address":"Hangzhou"}
  {"index": {}} {"name":"Zhang Wuji"."age":59."bir":"2012-12-12"."content":"ElasticSearch is a Lucene-based search server. It provides a distributed multi-user capability full-text search engine based on a RESTful Web interface."."address":"Beijing"}
Copy the code

6.3 URL retrieval

GET /ems/emp/_search? q=*&sort=age:ascCopy the code

_search: Search API
Q =* : Matches all documents
Sort: sort by the specified fields in the result
Asc: Sort mode (reverse order or order)

6.4. DSL Advanced Query

GET /ems/emp/_search { "query": {"match_all": {}},
"sort": [ { "age": { "order": "desc" } } ] }Copy the code

6.4.1, match_all

This keyword means that 1 of all documents in the index is returned.

GET /ems/emp/_search { "query": { "match_all": {} }
}	
Copy the code

6.4.2, size

The size keyword specifies the number of query results. By default, 10 query results are returned.

GET /ems/emp/_search { "query": { "match_all": {} },
	"size": 1
}	
Copy the code

6.4.3, from

From: specifies the starting return location. Used with size for paging.

GET /ems/emp/_search { "query": {"match_all": {}},
"sort": [ { "age": { "order": "desc" } } ], "size": 2, "from": 1 }
# # # 6.4.4,_source _Source is an array that specifies to return the specified field in the query result.

```markdown GET /ems/emp/_search { "query": { "match_all": {} }, "_source": ["account_number", "balance"] }Copy the code

6.4.5, term

Term is used to use keyword queries.

GET /ems/emp/_search {" query ": {" term" : {" address ": {" value" : "Beijing"}}}}Copy the code

Notes:

Through using term query, we know that StandardAnalyzer is the default word segmentation in ES, which is very friendly to English word segmentation, but very unfriendly to Chinese word segmentation.
Keyword, date, INTEGER, long, double, Boolean or IP in the Mapping Type of ES.

6.4.6, range,

Range is used to query documents within a specified range.

GET /ems/emp/_search { "query": { "range": { "age": { "gte": 8, "lte": 30 } } } }Copy the code

6.4.7, prefix

The prefix is used to retrieve the relevant documents containing the keywords with the specified prefix.

GET /ems/emp/_search { "query": { "prefix": { "content": { "value": "redis" } } } }Copy the code

6.4.8, wildcard

Wildcard is used for wildcard queries:

? : matches an arbitrary character.
* : Used to match any number of characters.

GET /ems/emp/_search { "query": { "wildcard": { "content": { "value": "re*" } } } }Copy the code

6.4.9, ids

The IDS keyword is used to get multiple documents based on a set of ids. Its value is an array type.

GET  /ems/emp/_search { "query": { "ids": { "values": ["lg5HwWkBxH7z6xax7W3_","lQ5HwWkBxH7z6xax7W3_ "]}}}Copy the code

6.4.10, fuzzy

Fuzzy is used to query documents with specified keywords. It has a maximum fuzzy error, which must be in the range 0~2:

The search term length is 2, no ambiguity is allowed, and the maximum ambiguity error is 0.
Search keywords are 3-5 in length, allow one blur, and the maximum blur error is 0 and 1.
Search keyword length greater than 5, the maximum fuzzy error is 2

GET /ems/emp/_search { "query": { "fuzzy": { "content":"spring" } } }Copy the code

6.4.11, bool

The bool keyword is used to combine multiple conditions to achieve complex queries.

GET /ems/emp/_search { "query": { "bool": { "must": [ { "range": { "age": { "gte": 0, "lte": 30 } } } ], "must_not": [
{"wildcard": { "content": { "value": "redi?" } }} ] } }, "sort": [ { "age": { "order": "desc" } } ] }Copy the code

6.4.12, highlight

The highlight keyword is used to highlight the key words in the documents that match the criteria. It does not highlight the original data. Instead, it queries the documents that match the highlight and prefixes them. We can customize the highlighting of HTML tags:

Pre_tags: prefix
Post_tags: suffix

GET /ems/emp/_search {" query ": {" term" : {" content ":" framework "}}, "highlight" : {" pre_tags": ["<span style='color:red'>"]."post_tags": [""], "fields": { "*":{} } } }Copy the code

If multi-field highlighting is required, it can be enabled by setting its value to false using the require_field_match keyword.

 GET /ems/emp/_search {" query ": {" term" : {" content ":" framework "}}, "highlight" : {" pre_tags": ["<span style='color:red'>"]."post_tags": [""], "require_field_match":false, "fields": { "*":{} } } }Copy the code

6.4.13, multi_match

Multi_match: multi_match: multi_match: multi_match

If the search field is segmented, the query is segmented first and then searched.
If the search field is not a word, he will directly use the query as a whole to search for the field. It is recommended to retrieve the field in the word segmentation field

GET /ems/emp/_search { "query": { "multi_match": {
["name","content"] ["name","content"]Copy the code

6.4.14, query_string

Query_string Used for multi-field segmentation queries.

GET /dangdang/book/_search { "query": { "query_string": {
"Query", "China's voice", "analyzer" : "ik_max_word", "fields" : (" name ", "content")}}}
Copy the code

6.5,Underlying principles of ElasticSearch

The underlying core of ElasticSearch is inverted index tables.

Index area: the result of document segmentation, for example: name:[sheet :0:1] (the keyword “sheet” appears once in document no. 0).

Metadata area: the original place of the individual documents.

6.5.1 Index

The forward list records the position information of each word in the document with the ID of the document as the keyword. When searching, the information of each word in the document is scanned until all documents containing the query keyword are found. We usually look for value by key.

His structure is: ID of document 1 → word 1: number of occurrences, list of occurrence locations; Word 2: number of occurrences, list of locations.

When a user searches for the keyword “Huawei mobile phone” on the home page and assumes that only the forward index exists, all documents in the index database need to be scanned to find all documents containing the keyword “Huawei mobile phone”. Then, the documents are graded according to the scoring model, and the ranking is presented to the user. Because the number of documents included in search engines on the Internet is astronomical, such an index structure simply cannot meet the requirements of real-time ranking results.

6.5.2 Inverted index

The search engine will rebuild the forward index into an inverted index, that is, the mapping of the file ID to the keyword into the mapping of the keyword to the file ID. Each keyword corresponds to a series of files in which the keyword appears. He came to use the keywords of the word to find the document.

His structure is: “Keyword 1” : ID of “document 1”, ID of “Document 2”

💖 Take out 20 minutes a day to bring you to the ElasticSearch full-text search engine 2️ with 3k salary increase