Elasticsearch: Documents and indexes

Original author, public number [programmer reading], welcome to pay attention to the public number, reprint the article please indicate the source oh.

In the last post, we looked at how Elastic Stack works and how it’s installed. The most important component is ElasticSearch (ES), so in this post we’re going to cover some of the core concepts of ES.

The core concepts of ES are index(index), Document(Document), Clusters(cluster), Node(Node) and instance. Let’s take a look at Document and index first.

RESTful APIs

Before we get into the concept of Document and Index, let’s take a look at RESTful APIs, because we’ll be using them when we talk about Document and Index.

Once we have the ES server up, how do we call it?

In fact, ES provides RESTful APIS based on HTTP protocol, that is, we can operate ES server by sending HTTP requests to es server, such as reading and writing documents, querying document API, searching API, and creating and deleting indexes. Es uses port 9200 to receive HTTP requests by default.

Curl curl curl curl curl curl curl curl curl curl curl curl curl curl curl

Use curl to create a document
curl http://localhost:9200/my_test/1 -H "Content-Type:application/json" \
-X POST -d '{"uid":1,"username":"test"}'
Copy the code

The following image shows the process of sending a request to ES and the RESPONSE from the ES server:

Dev Tools in Kibana allows you to use es RESTful apis. Dev Tools in Kibana allows you to use eS RESTful apis. Dev Tools in Kibana allows you to use eS RESTful apis. We will use this format in the next demonstration.

PUT /my_test/_doc/1
{
    "uid": 1,"username":"test"
}
Copy the code

Actually, for the convenience of the call to different programming languages, es offers a variety of programming languages of the libraries (Java, PHP, Ruby, Go, Python, JavaScript,.net, etc.), but these programming languages are based on es provide RESTful APIs of encapsulation.

The Document (the Document)

Es is document-oriented, and a document is the smallest searchable unit in ES. The document of ES consists of one or more fields, similar to a row of records in a relational database, but the document of ES is serialized and saved in JSON. Each JSON object consists of one or more fields, and the field type can be Boolean, value, String, binary, date, and other data types.

Es Each document has a unique ID, which can be specified by us or automatically generated by ES.

Metadata for the document

Es Each document, in addition to storing the original data of the document that we write, also has its own metadata, which identifies the relevant information of the document.

Here is a plain ES document:

{
  "_index" : "test_logs2"."_type" : "_doc"."_id" : "1"."_version" : 1,
  "_seq_no": 0."_primary_term" : 1,
  "found" : true."_source" : {
    "uid" : 1,
    "username" : "test"}}Copy the code

From the above document, we can see that the metadata fields of the document are as follows:

• _index: indicates the index name of the document
• _source: original JSON data
• _type: document type. After ES7.0, the value can only be _doc
• _version: indicates the document version. If the document is modified, this field will be added
• _score: correlation score
• Id: indicates the unique ID of the document

Es provides CURD and other operations through RESTful apis.

Create

With the ES RESTful API, we can create a document in an index using the HTTP PUT method. In Kibana Dev Tools, we can create a document using the following statement:

Create a document in my_test index
PUT /my_test/_create/1
{
    "uid": 1,"username":"one"
}
Copy the code

Index

In ES, indexes have the following three meanings and functions:

1. The verb, a way of creating documents in ES, is what we’re talking about here.
2. The way es organizes documents is described below.
3. Verb, to participle and store the fields of a document, as we’ll talk about later

The way to use Index
PUT /my_test/_doc/1
{
    "uid": 1,"username":"test"
}
Copy the code

Index is used to Create an ES document in the same way as Create, except that if the specified document ID already exists, the original document will be deleted and a new document will be created with the _version field of the document incremented by one

Update

To update the data in a document, use the HTTP POST method, and the modified field information must be in doc, as follows:

Using Update directly updates the data, as opposed to creating a document with Index, which deletes the document as it exists and then recreates it.

# update
POST /my_test/_doc/1
{
    "doc": {"username":"this is a document"}}Copy the code

Delete

An ES document can be deleted using the HTTP DELETE method as shown in the following example:

# delete file
DELETE /my_test/_doc/1
Copy the code

Read

Reading an ES document is simple, using the HTTP GET method, as follows:

# read
GET /my_test/_doc/1
Copy the code

Bulk Api

The above on the Index about the document, Create, Update, Delete and other operations, but every time can only be a document of an Index, and we know that every request to the server during operation, the network request round-trip time overhead is a big consumption, if each request only do an operation, That would be a bit wasteful.

So the bulk of the API document support es in a request for the different indexes of documents in the Index, the Create, Update, Delete and other operations, known as batch processing, in the process, even if one of the operating error, will not affect other operations, as follows:

POST _bulk
{"create": {"_index":"my_test2"."_id": {4}}"uid": 2."username":"333333333333333333"}
{"index": {"_index":"my_test2"."_id": 10}} {"uid": 10,"username":"tttt"}
{"delete": {"_index":"my_test2"."_id": {1}}"update": {"_index":"my_test2"."_id": {2}}"doc": {"uid": 2."username":"hhhhhhhhhhhhhhhhh"}}
Copy the code

The above is a simple example of the BULK API, but if you want to familiarize yourself with the syntax, look more closely at the official ES documentation.

The Index (Index)

Es index, the way es organizes documents, is a collection of documents with phase structure. The index of ES can be likened to a data table of a relational database.

Let’s take a look at RESTful APIs for various operations on indexes, as follows:

Create

An index can be created using the HTTP PUT method, and in Kibna’s Dev Tools, an index can be created using the following statement:

PUT /my_test
Copy the code

Also specify mapping and setting when creating the index as follows:

PUT /my_test
{
    "settings" : {
        "index" : {
            "number_of_shards": 3."number_of_replicas": 2}}}Copy the code

Exists

You can use the HTTP HEAD method to determine whether an index exists, as follows:

Check whether the index exists
HEAD /my_test
Copy the code

The HTTP status code returns 200 if the index exists, or 404 if it does not.

Get

The HTTP GET method can be used to obtain the index setting and mapping information as follows:

GET /index
Copy the code

The following result is returned:

{
  "my_test" : {
    "aliases": {},"mappings": {},"settings" : {
      "index" : {
        "creation_date" : "1564757617415"."number_of_shards" : "1"."number_of_replicas" : "1"."uuid" : "z6zGhu_ERA-R1c0m2fQrvg"."version" : {
          "created" : "7020099"
        },
        "provided_name" : "my_test"}}}}Copy the code

Delete

The index drop API in es allows us to drop an existing index in the following cases:

Delete one or more indexes using index names

# remove my_test
DELETE /my_test

# drop multiple indexes, separated by commas
DELETE /my_test,my_test1,my_test2
Copy the code

Use wildcard * to drop multiple indexes (with caution)

# delete index prefixed with my_test
DELETE /my_test*
Copy the code

Delete all indexes on the ES server using _all

Drop all indexes
DELETE /_all
Copy the code

Destructive_requires_name = true in es config/ elasticSearch. yml, for example:

Action. Destructive_requires_name:true
Copy the code

In this case, the above operations cannot be performed.

summary

Document and index are the most basic and core concepts in ES. Being familiar with the operation of document and index is the foundation for further learning ES. In fact, if you have the knowledge of relational database, you can compare index to data table in database, while document can be understood as a row record in data table.

---

Your attention is the biggest encouragement on my writing road!