Thanks to the whole section for ElasticSearch, most of which came from here

ElasticSearch

MySQL vs. ElasticSearch

MySQL	ElasticSearch
Database(Database)	The Index (Index)
The Table (Table).	Type (Type)
The Row (line)	Document (Document)
The Column (Column)	The Field (Field)
Schema (solution)	The Mapping (Mapping)
The Index (Index)	Everthing Indexed by Default (All fields Indexed)
SQL(Structured Query Language)	Query DSL(Query Specific Language)

Document APIs

Index API

The Index API allows us to store a DOCUMENT in JSON format, making the data searchable. The document is uniquely identified by index, type, and ID. The ID can either provide an ID itself or generate one for us using the Index API.

There are four different ways to generate JSON documents

• Manually, using native byte[] or String
• Using Map mode, the JSON equivalent is automatically converted
• Use third-party libraries to generate serialized Beans, such as JackJSON, FastJSON, and so on
• Use the built-in help class XContentFactory. JsonBuilder ()

manually

    /** * Manual mode *@throws UnknownHostException
     */
    @Test
    public void JsonDocument(a) throws UnknownHostException {
        String json = "{" +
                "\"user\":\"deepredapple\"," +
                "\"postDate\":\"2018-01-30\"," +
                "\"message\":\"trying out Elasticsearch\"" +
                "}";
        IndexResponse indexResponse = client.prepareIndex("fendo"."fendodate").setSource(json).get();
        System.out.println(indexResponse.getResult());
    }
Copy the code

The Map way

    /** * Map mode */
    @Test
    public void MapDocument(a) {
        Map<String, Object> json = new HashMap<String, Object>();
        json.put("user"."hhh");
        json.put("postDate"."2018-06-28");
        json.put("message"."trying out Elasticsearch");
        IndexResponse response = client.prepareIndex(INDEX, TYPE).setSource(json).get();
        System.out.println(response.getResult());
    }
Copy the code

Serialization mode

    /**
     * 使用JACKSON序列化
     */
    @Test
    public void JACKSONDocument(a) throws JsonProcessingException {
        Blog blog = new Blog();
        blog.setUser("123");
        blog.setPostDate("2018-06-29");
        blog.setMessage("try out ElasticSearch");

        ObjectMapper mapper = new ObjectMapper();
        byte[] bytes = mapper.writeValueAsBytes(blog);
        IndexResponse response = client.prepareIndex(INDEX, TYPE).setSource(bytes).get();
        System.out.println(response.getResult());
    }
Copy the code

XContentBuilder help class approach

    /** * Use the XContentBuilder helper class */
    @Test
    public void XContentBuilderDocument(a) throws IOException {
        XContentBuilder builder = XContentFactory.jsonBuilder().startObject()
                .field("user"."xcontentdocument")
                .field("postDate"."2018-06-30")
                .field("message"."this is ElasticSearch").endObject();
        IndexResponse response = client.prepareIndex(INDEX, TYPE).setSource(builder).get();
        System.out.println(response.getResult());
    }
Copy the code

Composite sample

package com.deepredapple.es.document;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.InetSocketTransportAddress;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentFactory;
import org.elasticsearch.transport.client.PreBuiltTransportClient;
import org.junit.Before;
import org.junit.Test;

import java.io.IOException;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;

/ * * *@author DeepRedApple
 */
public class TestClient {

    TransportClient client = null;

    public static final String INDEX = "fendo";

    public static final String TYPE = "fendodate";

    @Before
    public void beforeClient(a) throws UnknownHostException {
        client = new PreBuiltTransportClient(Settings.EMPTY)
                .addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("127.0.0.1"), 9300));
    }

    /** * Manual mode *@throws UnknownHostException
     */
    @Test
    public void JsonDocument(a) throws UnknownHostException {
        String json = "{" +
                "\"user\":\"deepredapple\"," +
                "\"postDate\":\"2018-01-30\"," +
                "\"message\":\"trying out Elasticsearch\"" +
                "}";
        IndexResponse indexResponse = client.prepareIndex(INDEX, TYPE).setSource(json).get();
        System.out.println(indexResponse.getResult());
    }

    /** * Map mode */
    @Test
    public void MapDocument(a) {
        Map<String, Object> json = new HashMap<String, Object>();
        json.put("user"."hhh");
        json.put("postDate"."2018-06-28");
        json.put("message"."trying out Elasticsearch");
        IndexResponse response = client.prepareIndex(INDEX, TYPE).setSource(json).get();
        System.out.println(response.getResult());
    }

    /**
     * 使用JACKSON序列化
     */
    @Test
    public void JACKSONDocument(a) throws JsonProcessingException {
        Blog blog = new Blog();
        blog.setUser("123");
        blog.setPostDate("2018-06-29");
        blog.setMessage("try out ElasticSearch");

        ObjectMapper mapper = new ObjectMapper();
        byte[] bytes = mapper.writeValueAsBytes(blog);
        IndexResponse response = client.prepareIndex(INDEX, TYPE).setSource(bytes).get();
        System.out.println(response.getResult());
    }

    /** * Use the XContentBuilder helper class */
    @Test
    public void XContentBuilderDocument(a) throws IOException {
        XContentBuilder builder = XContentFactory.jsonBuilder().startObject()
                .field("user"."xcontentdocument")
                .field("postDate"."2018-06-30")
                .field("message"."this is ElasticSearch").endObject(); IndexResponse response = client.prepareIndex(INDEX, TYPE).setSource(builder).get(); System.out.println(response.getResult()); }}Copy the code

Get API

The GET API can view documents by ID

GetResponse getResponse = client.prepareGet(INDEX, TYPE, "AWRJCXMhro3r8sDxIpir").get();
Copy the code

The parameters are index, type, and _id

Configuration of the thread

If setOperationThreaded is set to true, perform this operation on a different thread

    /** * Get API */
    @Test
    public void testGetApi(a) {
        GetResponse getResponse = client.prepareGet(INDEX, TYPE, "AWRJCXMhro3r8sDxIpir").setOperationThreaded(false).get();
        Map<String, Object> map = getResponse.getSource();
        Set<String> keySet = map.keySet();
        for(String str : keySet) { Object o = map.get(str); System.out.println(o.toString()); }}Copy the code

Delete API

Delete by ID:

DeleteResponse deleteResponse = client.prepareDelete(INDEX, TYPE, "AWRJCXMhro3r8sDxIpir").get();
Copy the code

The parameters are index, type, and _id

Configuration of the thread

If setOperationThreaded is set to true, perform this operation on a different thread

    /** * deleteAPI */
    @Test
    public void testDeleteAPI(a) {
        GetResponse getResponse = client.prepareGet(INDEX, TYPE, "AWRJCXMhro3r8sDxIpir").setOperationThreaded(false).get();
        System.out.println(getResponse.getSource());
        DeleteResponse deleteResponse = client.prepareDelete(INDEX, TYPE, "AWRJCXMhro3r8sDxIpir").get();
        System.out.println(deleteResponse.getResult());
    }
Copy the code

Delete By Query API

Delete the vm by search criteria

    /** * Delete */ by query criteria
    @Test
    public void deleteByQuery(a) {
        BulkByScrollResponse response = DeleteByQueryAction.INSTANCE.newRequestBuilder(client)
                .filter(QueryBuilders.matchQuery("user"."hhh")) // Query conditions
                .source(INDEX).get();/ / index name
        long deleted = response.getDeleted();// Delete document number
        System.out.println(deleted);
    }
Copy the code

Parameter Description QueryBuilders. MatchQuery (“user”, “HHH “) takes fields and query conditions, and source(INDEX) takes the INDEX name

An asynchronous callback

If the deletion takes a long time, you can use an asynchronous callback to perform the deletion and obtain the result from the callback

    / * * * to delete the callback Suitable for large amount of data delete * /
    @Test
    public void DeleteByQueryAsync(a) {
        for (int i = 1300; i < 3000; i++) {
            DeleteByQueryAction.INSTANCE.newRequestBuilder(client)
                .filter(QueryBuilders.matchQuery("user"."hhh " + i))
                .source(INDEX)
                .execute(new ActionListener<BulkByScrollResponse>() {
                    public void onResponse(BulkByScrollResponse response) {
                        long deleted = response.getDeleted();
                        System.out.println("Number of documents removed ="+deleted);
                    }

                    public void onFailure(Exception e) {
                        System.out.println("Failure"); }}); }}Copy the code

When the application stops, the ElasticSearch console is still performing the delete operation, asynchronously

The listener callback method is the execute method

    .execute(new ActionListener<BulkByScrollResponse>() { // Callback method
      public void onResponse(BulkByScrollResponse response) {
        long deleted = response.getDeleted();
        System.out.println("Number of documents removed ="+deleted);
      }

      public void onFailure(Exception e) {
        System.out.println("Failure"); }});Copy the code

Update API

Update the index

There are two main ways to perform the update operation

• Create an UpdateRequest and send it through the client
• Use the prepareUpdate() method.

Using UpdateRequest

    /** * Update with UpdateRequest */
    @Test
    public void testUpdateAPI(a) throws IOException, ExecutionException, InterruptedException {
        UpdateRequest updateRequest = new UpdateRequest();
        updateRequest.index(INDEX);
        updateRequest.type(TYPE);
        updateRequest.id("AWRFv-yAro3r8sDxIpib");
        updateRequest.doc(jsonBuilder()
                .startObject()
                    .field("user"."hhh")
                .endObject());
        client.update(updateRequest).get();
    }
Copy the code

Use the prepareUpdate ()

    /** * use PrepareUpdate */
    @Test
    public void testUpdatePrepareUpdate(a) throws IOException {
        client.prepareUpdate(INDEX, TYPE, "AWRFvA7k0udstXU4tl60")
                .setScript(new Script("ctx._source.user = \"DeepRedApple\"")).get();
        client.prepareUpdate(INDEX, TYPE, "AWRFvA7k0udstXU4tl60")
                .setDoc(jsonBuilder()
                .startObject()
                    .field("user"."DeepRedApple")
                .endObject()).get();
    }
Copy the code

Different versions of the setScript method in Client. prepareUpdate have different parameters. The values are directly passed in, or the script stored in the file can be directly inserted, and the data in the script can be directly updated.

Update By Script

Update documents with scripts

    /** * Update */ via script
    @Test
    public void testUpdateByScript(a) throws ExecutionException, InterruptedException {
        UpdateRequest updateRequest = new UpdateRequest(INDEX, TYPE, "AWRFvLSTro3r8sDxIpia")
                .script(new Script("ctx._source.user = \"LZH\""));
        client.update(updateRequest).get();
    }
Copy the code

Upsert

Update the document, if it exists, or insert it if it doesn’t

    /** * Update the document if there are updates, otherwise insert */
    @Test
    public void testUpsert(a) throws IOException, ExecutionException, InterruptedException {
        IndexRequest indexRequest = new IndexRequest(INDEX, TYPE, "AWRFvLSTro3r8sDxIp12")
                .source(jsonBuilder()
                    .startObject()
                        .field("user"."hhh")
                        .field("postDate"."2018-02-14")
                        .field("message"."ElasticSearch")
                    .endObject());
        UpdateRequest updateRequest = new UpdateRequest(INDEX, TYPE, "AWRFvLSTro3r8sDxIp12")
                .doc(jsonBuilder()
                    .startObject()
                        .field("user"."LZH")
                    .endObject())
                .upsert(indexRequest); // If not, add indexRequest
        client.update(updateRequest).get();
    }
Copy the code

If the _id parameter is present (index/type/_id), then UpdateRequest is executed. If the index/type/_id parameter is not present, then UpdateRequest is inserted

Multi Get API

Get multiple documents at once,

    /** * get multiple documents at once */
    @Test
    public void TestMultiGetApi(a) {
        MultiGetResponse responses = client.prepareMultiGet()
                .add(INDEX, TYPE, "AWRFv-yAro3r8sDxIpib") // An ID
                .add(INDEX, TYPE, "AWRFvA7k0udstXU4tl60"."AWRJA72Uro3r8sDxIpip")// Multiple ids
                .add("blog"."blog"."AWG9GKCwhg1e21lmGSLH") // Fetch from another index
                .get();
        for (MultiGetItemResponse itemResponse : responses) {
            GetResponse response = itemResponse.getResponse();
            if (response.isExists()) {
                String source = response.getSourceAsString(); //_source
                JSONObject jsonObject = JSON.parseObject(source);
                Set<String> sets = jsonObject.keySet();
                for (String str : sets) {
                    System.out.println("key -> " + str);
                    System.out.println("value -> "+jsonObject.get(str));
                    System.out.println("= = = = = = = = = = = = = = ="); }}}}Copy the code

Bulk API

The Buli API enables bulk inserts

    /** * batch insert */
    @Test
    public void testBulkApi(a) throws IOException {
        BulkRequestBuilder requestBuilder = client.prepareBulk();
        requestBuilder.add(client.prepareIndex(INDEX, TYPE, "1")
            .setSource(jsonBuilder()
                .startObject()
                    .field("user"."Zhang")
                    .field("postDate"."2018-05-01")
                    .field("message"."zhangSan message")
                .endObject()));
        requestBuilder.add(client.prepareIndex(INDEX, TYPE, "2")
            .setSource(jsonBuilder()
                .startObject()
                    .field("user"."Bill")
                    .field("postDate"."2016-09-10")
                    .field("message"."Lisi message")
                .endObject()));
        BulkResponse bulkResponse = requestBuilder.get();
        if (bulkResponse.hasFailures()) {
            System.out.println("error"); }}Copy the code

Useing Bulk Processor

With the Bulk Processor, the Bulk Processor provides a simple interface to automate timed batch requests on a given size of quantity

Create the Bulk Processor instance

Start by creating the Bulk Processor instance

    /** * Create Processor instance */
    @Test
    public void testCreateBulkProcessor(a) {
        BulkProcessor bulkProcessor = BulkProcessor.builder(client, new BulkProcessor.Listener() {
            // Perform this before calling Bulk, for example numberOfActions can be known by the request.numberoFactions () method
            public void beforeBulk(long l, BulkRequest request) {}// Execute after calling Bulk, for example by using the Response.hasfailures () method
            public void afterBulk(long l, BulkRequest request, BulkResponse response) {}// Call failure throws throwable
            public void afterBulk(long l, BulkRequest bulkRequest, Throwable throwable) {

            }
        }).setBulkActions(10000) // 10000 requests at a time
          .setBulkSize(new ByteSizeValue(5, ByteSizeUnit.MB)) // Split into 5MB pieces
          .setFlushInterval(TimeValue.timeValueSeconds(5))// Request every 5 seconds, regardless of the number of requests
          .setConcurrentRequests(1)// Set the number of concurrent requests. A value of 0 means that only one request is allowed. A value of 1 means that one concurrent request is allowed
          .setBackoffPolicy(
                  BackoffPolicy.exponentialBackoff(TimeValue.timeValueMillis(100), 3))
                / / custom repeat request mechanism, the most began to wait for 100 milliseconds, then multiply, retry 3 times, when one or more repeat request failed because not enough computing resources thrown EsRejectedExecutionException
                Backoffpolicy.nobackoff () to turn off the retry mechanism
          .build();
    }
Copy the code

BulkProcess is the default design

• bulkActions 1000
• bulkSize 5mb
• Don’t set flushInterval
• ConcurrentRequests (1) is executed asynchronously
• BackoffPolicy retries eight times and waits for 50 milliseconds

    /** * Create Processor instance */
    @Test
    public void testCreateBulkProcessor(a) throws IOException {
        BulkProcessor bulkProcessor = BulkProcessor.builder(client, new BulkProcessor.Listener() {
            // Perform this before calling Bulk, for example numberOfActions can be known by the request.numberoFactions () method
            public void beforeBulk(long l, BulkRequest request) {}// Execute after calling Bulk, for example by using the Response.hasfailures () method
            public void afterBulk(long l, BulkRequest request, BulkResponse response) {}// Call failure throws throwable
            public void afterBulk(long l, BulkRequest bulkRequest, Throwable throwable) {

            }
        }).setBulkActions(10000) // 10000 requests at a time
          .setBulkSize(new ByteSizeValue(5, ByteSizeUnit.MB)) // Split into 5MB pieces
          .setFlushInterval(TimeValue.timeValueSeconds(5))// Request every 5 seconds, regardless of the number of requests
          .setConcurrentRequests(1)// Set the number of concurrent requests. A value of 0 means that only one request is allowed. A value of 1 means that one concurrent request is allowed
          .setBackoffPolicy(
                  BackoffPolicy.exponentialBackoff(TimeValue.timeValueMillis(100), 3))
                / / custom repeat request mechanism, the most began to wait for 100 milliseconds, then multiply, retry 3 times, when one or more repeat request failed because not enough computing resources thrown EsRejectedExecutionException
                Backoffpolicy.nobackoff () to turn off the retry mechanism
          .build();

        //增加requests
        bulkProcessor.add(new IndexRequest(INDEX, TYPE, "3").source(
                jsonBuilder()
                        .startObject()
                            .field("user"."Fifty")
                            .field("postDate"."2019-10-05")
                            .field("message"."wangwu message")
                        .endObject()));
        bulkProcessor.add(new DeleteRequest(INDEX, TYPE, "1"));
        bulkProcessor.flush();
        / / close bulkProcessor
        bulkProcessor.close();
        client.admin().indices().prepareRefresh().get();
        client.prepareSearch().get();
    }
Copy the code

Search API

The search API supports search queries and returns matching results. It can search for one index/type or multiple index/ types using the Query Java API as Query criteria

Java provides two search Types QUERY_AND_FETCH and DFS_QUERY_AND_FETCH by default, but this mode should be chosen by the system rather than manually specified by the user

The instance

    @Test
    public void testSearchApi(a) {
        SearchResponse response = client.prepareSearch(INDEX).setTypes(TYPE)
                .setQuery(QueryBuilders.matchQuery("user"."hhh")).get();
        SearchHit[] hits = response.getHits().getHits();
        for (int i = 0; i < hits.length; i++) {
            String json = hits[i].getSourceAsString();
            JSONObject object = JSON.parseObject(json);
            Set<String> strings = object.keySet();
            for(String str : strings) { System.out.println(object.get(str)); }}}Copy the code

Using scrolls in Java

The Scrolls API allows us to retrieve large amounts of data, or even all of it, since a typical search request will return a page of data regardless of the size. The Scroll API allows us to do an initial search page and continue to batch pull results from ElasticSearch until there are none left. The Scroll API was not created for real-time user response, but for processing large amounts of data.

  /** **@throws ExecutionException
   * @throws InterruptedException
   */
  @Test
  public void testScrollApi(a) throws ExecutionException, InterruptedException {
      MatchQueryBuilder qb = matchQuery("user"."hhh");
      SearchResponse response = client.prepareSearch(INDEX).addSort(FieldSortBuilder.DOC_FIELD_NAME,
              SortOrder.ASC)
              .setScroll(new TimeValue(60000)) // In order to use scroll, the initial search request should specify the scroll parameter in the query to tell ElasticSearch how long it needs to hold the search context
              .setQuery(qb)
              .setSize(100).get();
      do {
          for (SearchHit hit : response.getHits().getHits()) {
              String json = hit.getSourceAsString();
              JSONObject object = JSON.parseObject(json);
              Set<String> strings = object.keySet();
              for (String str : strings) {
                  System.out.println(object.get(str));
              }
          }
          response = client.prepareSearchScroll(response.getScrollId()).setScroll(new TimeValue(60000)).execute().get();
      } while(response.getHits().getHits().length ! =0);
  }
Copy the code

If you continue to search for data using the scroll ID after the scroll time, an error is reported

Although the scrolling time has expired, the search context will be automatically cleared, but it will cost a lot to keep scrolling, so we should use the Clear-Scroll API to Clear it as soon as possible when we are not using scrolling.

Clear scroll ID

        ClearScrollRequestBuilder clearBuilder = client.prepareClearScroll();
        clearBuilder.addScrollId(response.getScrollId());
        ClearScrollResponse scrollResponse = clearBuilder.get();
        System.out.println("Knowing success:"+scrollResponse.isSucceeded());
Copy the code

MultiSearch API

The MultiSearch API allows multiple search requests to be executed in the same API. Its endpoint is _msearch

    @Test
    public void testMultiSearchApi(a) {
        SearchRequestBuilder srb1 = client.prepareSearch().setQuery(QueryBuilders.queryStringQuery("elasticsearch")).setSize(1);
        SearchRequestBuilder srb2 = client.prepareSearch().setQuery(QueryBuilders.matchQuery("user"."hhh")).setSize(1);
        MultiSearchResponse multiSearchResponse = client.prepareMultiSearch().add(srb1).add(srb2).get();
        long nbHits = 0;
        for (MultiSearchResponse.Item item : multiSearchResponse.getResponses()) {
            SearchResponse response = item.getResponse();
            nbHits += response.getHits().getTotalHits();
        }
        System.out.println(nbHits);
    }
Copy the code

Using Aggregations

The aggregation framework helps provide data based on search queries. It is based on simple building blocks also known as integrations, which put complex data summaries together in an orderly fashion. Aggregation can be thought of as a collective term for a series of efforts to extract analytical information from a set of files. The implementation of the aggregation is the process of defining this set of documents

    @Test
    public void testAggregations(a) {
        SearchResponse searchResponse = client.prepareSearch()
                .setQuery(QueryBuilders.matchAllQuery())
                .addAggregation(AggregationBuilders.terms("LZH").field("user"))
                .addAggregation(AggregationBuilders.dateHistogram("2013-01-30").field("postDate")
                        .dateHistogramInterval(DateHistogramInterval.YEAR)).get();
        Terms lzh = searchResponse.getAggregations().get("user");
        Histogram postDate = searchResponse.getAggregations().get("2013-01-30");

    }
Copy the code

Terminate After

Sticks to the maximum number of documents in our SearchResponse object. If this is set, the isterMinatesticks () in our SearchResponse object checks whether the maximum number of documents returned is reached

    @Test
    public void TestTerminate(a) {
        SearchResponse searchResponse = client.prepareSearch(INDEX)
                .setTerminateAfter(2) // If this number is reached, terminate early
                .get();
        if(searchResponse.isTerminatedEarly()) { System.out.println(searchResponse.getHits().totalHits); }}Copy the code

Aggregations

Aggregation. ElasticSearch provides a full Java API to use aggregation. Use AggregationBuilders to build objects to add to the search request.

SearchResponse response = client.prepareSearch().setQuery(/ * * /).addAggregation(/ * aggregation * /).execute().actionGet();
Copy the code

Structuring aggregations

Structured aggregation.

Metrics aggregations

Such aggregation operations in the computed measures class are based on using a method or extracting the values to be aggregated from the document.

The main class used in this process is **AggregationBuilders**, which contains a large number of the following aggregate method calls that can be used directly

Min Aggregation Indicates minimal Aggregation

    MinAggregationBuilder aggregation = AggregationBuilders.min("agg").field("age");

    SearchResponse sr = client.prepareSearch("twitter").addAggregation(aggregation).get();
    Min agg = sr.getAggregations().get("agg");
    String value = agg.getValueAsString();// This count is the date, generally use the following method to obtain the minimum value
    System.out.println("min value:" + value);
Copy the code

The debug mode

The first line of MinAggregationBuilder toString() does the following

{
  "error": "JsonGenerationException[Can not write a field name, expecting a value]"
}
Copy the code

SearchResponse sr = client.prepareSearch("twitter").addAggregation(aggregation).get();
Copy the code

The content of toString() in SearchResponse is as follows. This content is the JSON result of the query. The STRUCTURE of the JSON result is used in conjunction with the API operation of SearchResponse to obtain each value in it.

{
  "took": 1."timed_out": false."_shards": {
    "total": 5."successful": 5."skipped": 0."failed": 0
  },
  "hits": {
    "total": 4."max_score": 1.0."hits": [{"_index": "twitter"."_type": "tweet"."_id": "10"."_score": 1.0."_source": {
          "user": "kimchy"."postDate": "2018-06-29 T09:1. 396 z"."age": 30."gender": "female"."message": "trying out Elasticsearch"}}, {"_index": "twitter"."_type": "tweet"."_id": "2"."_score": 1.0."_source": {
          "user": "kimchy"."postDate": "The 2018-06-29 T09:05:33. 943 z"."age": 20."gender": "female"."message": "trying out Elasticsearch"}}, {"_index": "twitter"."_type": "tweet"."_id": "1"."_score": 1.0."_source": {
          "user": "kimchy"."postDate": "The 2018-06-29 T08:59:00. 191 z"."age": 10."gender": "male"."message": "trying out Elasticsearch"}}, {"_index": "twitter"."_type": "tweet"."_id": "11"."_score": 1.0."_source": {
          "user": "kimchy"."postDate": "The 2018-06-29 T09:10:54. 386 z"."age": 30."gender": "female"."message": "trying out Elasticsearch"}}},"aggregations": {
    "agg": {
      "value": 10.0}}}Copy the code

Sr.getaggregations ().get(“agg”); The method is to get the aggregated statistics data, where the parameter AGG throughout the code can be customized

Max Aggregation maximum Aggregation

    MaxAggregationBuilder aggregation = AggregationBuilders.max("agg").field("readSize");

    SearchResponse sr = client.prepareSearch("blog").addAggregation(aggregation).get();
    Max agg = sr.getAggregations().get("agg");
    String value = agg.getValueAsString();

    System.out.println("max value:" + value);
Copy the code

The specific analysis method is the same as Min Aggregation, but the maximum and minimum values of which data cannot be calculated

Sum Aggregation

    SumAggregationBuilder aggregation = AggregationBuilders.sum("agg").field("readSize");

    SearchResponse sr = client.prepareSearch("blog").addAggregation(aggregation).get();
    Sum agg = sr.getAggregations().get("agg");
    String value = agg.getValueAsString();

    System.out.println("sum value:" + value);
Copy the code

Avg Aggregation means average Aggregation

AvgAggregationBuilder aggregation = AggregationBuilders.avg("agg").field("age");
SearchResponse searchResponse = client.prepareSearch("twitter").addAggregation(aggregation).get();
Avg avg = searchResponse.getAggregations().get("agg");
String value = avg.getValueAsString();
System.out.println("avg value: "+ value);
Copy the code

Stats Aggreagtin Statistics aggregation

Statistical aggregation – Calculates statistics (min, Max, sum, count, AVG) based on a value in the document. The value can be a specific numeric field or calculated from a script.

        StatsAggregationBuilder aggregation = AggregationBuilders.stats("agg").field("age");
        SearchResponse searchResponse = client.prepareSearch("twitter").addAggregation(aggregation).get();
        Stats stats = searchResponse.getAggregations().get("agg");
        String max = stats.getMaxAsString();
        String min = stats.getMinAsString();
        String avg = stats.getAvgAsString();
        String sum = stats.getSumAsString();
        long count = stats.getCount();
        System.out.println("max value: "+max);
        System.out.println("min value: "+min);
        System.out.println("avg value: "+avg);
        System.out.println("sum value: "+sum);
        System.out.println("count value: "+count);
Copy the code

This aggregate statistic will add up to the usual statistic above. You can use this method when you need to count most of the above values

Extended Stats Aggregation Extends statistical Aggregation

Extended statistics aggregation – Calculates statistics based on a value of the document (more than the normal STATS aggregation sum_of_squares, variance, STd_deviation, std_deviation_bounds). The values used to calculate can be specific numerical fields. It can also be calculated from a script. The main result values are maximum, minimum, variance, square variance and other statistical values

        ExtendedStatsAggregationBuilder aggregation = AggregationBuilders.extendedStats("agg").field("age");
        SearchResponse response = client.prepareSearch("twitter").addAggregation(aggregation).get();
        ExtendedStats extended = response.getAggregations().get("agg");
        String max = extended.getMaxAsString();
        String min = extended.getMinAsString();
        String avg = extended.getAvgAsString();
        String sum = extended.getSumAsString();
        long count = extended.getCount();
        double stdDeviation = extended.getStdDeviation();
        double sumOfSquares = extended.getSumOfSquares();
        double variance = extended.getVariance();
        System.out.println("max value: " +max);
        System.out.println("min value: " +min);
        System.out.println("avg value: " +avg);
        System.out.println("sum value: " +sum);
        System.out.println("count value: " +count);
        System.out.println("stdDeviation value: " +stdDeviation);
        System.out.println("sumOfSquares value: " +sumOfSquares);
        System.out.println("variance value: "+variance);
Copy the code

Value Count Aggregation Indicates the Aggregation

Value count aggregation – Count the number of values in an aggregate document. The value used for the calculation can be a specific numeric field, or it can be calculated from a script. This aggregation is commonly used in conjunction with other single-value aggregations, such as calculating the average value of a field, and perhaps paying attention to how many values the average value is calculated from.

ValueCountAggregationBuilder aggregation = AggregationBuilders.count("agg").field("age");
SearchResponse response = client.prepareSearch("twitter").addAggregation(aggregation).get();
ValueCount count = response.getAggregations().get("agg");
long value = count.getValue();
System.out.println("ValueCount value: " +value);
Copy the code

Precentile Aggregation is 100%

    PercentilesAggregationBuilder aggregation = AggregationBuilders.percentiles("agg").field("age");
    SearchResponse response = client.prepareSearch("twitter").addAggregation(aggregation).get();
    Percentiles agg = response.getAggregations().get("agg");
    for (Percentile entry : agg) {
        double percent = entry.getPercent();
        double value = entry.getValue();
        System.out.println("percent value: " + percent + "value value: " + value);
    }
Copy the code

Cardinality Aggreagion Cardinality aggregation

The base of a number of duplicates removed

CardinalityAggregationBuilder aggregation = AggregationBuilders.cardinality("agg").field("age");
SearchResponse response = client.prepareSearch("twitter").addAggregation(aggregation).get();
Cardinality agg = response.getAggregations().get("agg");
long value = agg.getValue();
System.out.println("value value: "+ value);
Copy the code

Top Hits Aggregation Indicates the highest matched weight Aggregation

Query the number of matched document fields

TermsAggregationBuilder aggregation = AggregationBuilders.terms("agg").field("gender.keyword")
.subAggregation(AggregationBuilders.topHits("top").explain(true).size(1).from(10));
SearchResponse response = client.prepareSearch("twitter").addAggregation(aggregation).get();
Terms agg = response.getAggregations().get("agg");
        for (Terms.Bucket bucket : agg.getBuckets()) {
            String key = (String) bucket.getKey();
            long docCount = bucket.getDocCount();
            System.out.println("key value: " + key + " docCount value: " + docCount);
            TopHits topHits = bucket.getAggregations().get("top");
            for (SearchHit searchHitFields : topHits.getHits().getHits()) {
                System.out.println("id value: " + searchHitFields.getId() + " source value: "+ searchHitFields.getSourceAsString()); }}Copy the code

Bucket aggregations

Global Aggregation Indicates Global Aggregation

Query a global quantity statistic

        AggregationBuilder aggregation = AggregationBuilders
                .global("agg")
                .subAggregation(
                        AggregationBuilders.terms("users").field("user.keyword")); SearchResponse sr = client.prepareSearch("twitter")
                .addAggregation(aggregation)
                .get();
        System.out.println(sr);
        Global agg = sr.getAggregations().get("agg");
        long count = agg.getDocCount(); // Doc count

        System.out.println("global count:" + count);
Copy the code

Filter Aggreagion Filter aggregation

Filter statistics

AggregationBuilder aggregation = AggregationBuilders.filters("aaa".new FiltersAggregator.KeyedFilter("men", QueryBuilders.termQuery("gender"."male")));

SearchResponse sr = client.prepareSearch("twitter").setTypes("tweet").addAggregation(aggregation).get();
Filters agg = sr.getAggregations().get("aaa");
for (Filters.Bucket entry : agg.getBuckets()) {
  String key = entry.getKeyAsString();            // bucket key
  long docCount = entry.getDocCount();            // Doc count

  System.out.println("global " + key + " count:" + docCount);
}
Copy the code

These are aggregated

Filter multiple conditions and query the number

AggregationBuilder aggregation = AggregationBuilders.filters("aaa".new FiltersAggregator.KeyedFilter("men", QueryBuilders.termQuery("gender"."male")),new FiltersAggregator.KeyedFilter("women", QueryBuilders.termQuery("gender"."female")));

SearchResponse sr = client.prepareSearch("twitter").setTypes("tweet").addAggregation(aggregation).get();
Filters agg = sr.getAggregations().get("aaa");
for (Filters.Bucket entry : agg.getBuckets()) {
  String key = entry.getKeyAsString();            // bucket key
  long docCount = entry.getDocCount();            // Doc count

  System.out.println("global " + key + " count:" + docCount);
}
Copy the code

Missing Aggregation A single-bucket Aggregation based on field data

Nested Aggregation Indicates Aggregation of Nested types

Reverse nested Aggregation

Children Aggregation

Terms Aggregation

        TermsAggregationBuilder fieldAggregation = AggregationBuilders.terms("genders").field("gender.keyword")
                .order(Terms.Order.term(true));
        SearchResponse response = client.prepareSearch("twitter").setTypes("tweet").addAggregation(fieldAggregation).get();

        Terms terms = response.getAggregations().get("genders");
        for (Terms.Bucket bucket : terms.getBuckets()) {
            System.out.println("key value: " + bucket.getKey());
            System.out.println("docCount value: " + bucket.getDocCount());
        }
Copy the code

The Order sorting

        TermsAggregationBuilder fieldAggregation = AggregationBuilders.terms("genders").field("gender.keyword")
                .order(Terms.Order.term(true));
Copy the code

Significant Terms Aggregation

Range Aggregation

Date Range Aggregation Date Aggregation

Ip Range Aggregation Ip Range Aggregation

Histogram Aggregation

Date Histogram Aggregation Date range Histogram Aggregation

Geo Distance Aggregation

GeoHash Grid Aggregation GeoHash Grid Aggregation

Query DSL

Match All Query

Match all documents

QueryBuilder qb = matchAllQuery();
Copy the code

Full text Query

Match Query Match Query

Fuzzy matching and field phrase query

QueryBuilder qb = matchQuery("gender"."female");
Copy the code

Multi_mathc query Multi-field query

Multiple fields. You can have multiple fields

QueryBuilder qb = multiMatchQuery("female"."gender"."message");
Copy the code

Common_terms query Query of common terms

Some more professional partial door words for more professional inquiry

QueryBuilder qb = commonTermsQuery("gender"."female");
Copy the code

Query_string query Query statement

A combined with Lucene query syntax queries, allows the use of special conditions to the query (AND | OR | NOT)

QueryBuilder qb = queryStringQuery("+male -female");
Copy the code

Simple_string query Simple query statement

A simple query syntax

QueryBuilder qb = queryStringQuery("+male -female");
Copy the code

Term level Query

Term Query Item Query

Queries a document for the exact value in the specified field

QueryBuilder qb = termQuery("gender"."male");
Copy the code

Terms Query Specifies multiple queries

Query multiple exact values within a field

QueryBuilder qb = termsQuery("age"."10"."20");
Copy the code

Range Query Range Query

Range queries

• Gte () : Range queries will match documents with field values greater than or equal to this parameter value
• Gt () : Range query will match documents with field values greater than this parameter value
• Lte () : A range query will match documents with field values less than or equal to this parameter value
• Lt () : Range queries will match documents with field values less than this parameter value
• From () start value to() result value, these two functions are used with includeLower() and includeUpper()
• IncludeLower (true) indicates that the from() query will match documents with field values greater than or equal to this parameter value
• IncludeLower (false) indicates that the FROM () query will match documents with field values greater than this parameter value
• IncludeUpper (true) indicates that the to() query will match documents whose field values are less than or equal to this parameter value
• IncludeUpper (false) indicates that the to() query will match documents with field values less than this parameter value

QueryBuilder qb = QueryBuilders.rangeQuery("age").gte(10).includeLower(true).lte(20).includeUpper(true);
Copy the code

Methods includeLower() and includeUpper() indicate whether the range includes a query

Exists Query Exists a Query

Queries whether the specified field exists

QueryBuilder qb = existsQuery("user");
Copy the code

Prefix Query Prefix Query

Query by the specified field name and the specified exact prefix

QueryBuilder qb = prefixQuery("gender"."m");
Copy the code

Wildcard Query Query of the Wildcard

Wildcard query, specifying field name and wildcard. Among them? * indicates the single-character wildcard, and * indicates the multi-character wildcard. Wildcard query fields are unparsed fields

QueryBuilder qb = wildcardQuery("gender"."f? *");
Copy the code

Regexp Query Regular expression Query

Queries based on the specified field name and regular expression. The query fields are also unparsed fields

QueryBuilder qb = regexpQuery("gender"."f.*");
Copy the code

Fuzzy Query Fuzzy Query

Fuzzy query: Specify the exact field name and misspelled query content

QueryBuilder qb = fuzzyQuery("gender"."mala").fuzziness(Fuzziness.ONE);
Copy the code

Type Query Indicates the Query Type

Queries documents of the specified type

QueryBuilder qb = typeQuery("tweet");
Copy the code

Ids Query ID Query

Based on the type and ID query, the type may not be written

QueryBuilder qb = idsQuery("tweet").addIds("1"."11");
Copy the code