ElasticSearch integration SpringBoot

ES is an open-source, highly extensible distributed full text search engine that is at the heart of the Elastic Stack. It can store and retrieve data in near real time; Its scalability is very good, can be extended to hundreds of servers, processing PB level of data. Elasticsearch is developed in The Java language, so you can access Elasticsearch using the Java API. The operation method is basically the same as the Http operation

This article has been included on Github: github.com/chenliang15…

Maven rely on

Add ES dependencies to POM.xml

<dependencies>
    <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch</artifactId>
        <version>7.8.0</version>
    </dependency>
    <! -- ElasticSearch client -->
    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>elasticsearch-rest-high-level-client</artifactId>
        <version>7.8.0</version>
    </dependency>
    <! -- ElasticSearch relies on 2.x log4j -->
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-api</artifactId>
        <version>2.8.2</version>
    </dependency>
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-core</artifactId>
        <version>2.8.2</version>
</dependency>
	<dependency>
		<groupId>com.fasterxml.jackson.core</groupId>
		<artifactId>jackson-databind</artifactId>
		<version>2.9.9</version>
	</dependency>
    <! -- Junit Unit Test -->
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.12</version>
    </dependency>
</dependencies>
Copy the code

Error if running
```
java.lang.NoSuchMethodError: org.elasticsearch.client.Request.addParameters
Copy the code
```
Create a New Maven project. It is better not to create a SpringBoot project directly. There may be dependency conflicts, so add the above dependencies to the Maven project POM

Link to the ES client

Because the highLevel ES client is used, it is very easy to connect to ES, just through the host and port can be directly connected

public static void main(String[] args) throws IOException {
    RestHighLevelClient esClient = new RestHighLevelClient(
        RestClient.builder(new HttpHost("localhost".9200."http"))); esClient.close(); }Copy the code

If the code executes correctly and no error is reported, it is properly connected to the ES node and can be performed using the RestHighLevelClient object obtained

The index operation

With maven dependencies, you can link to ES via a Java client, starting with the basic operations of indexing

Create indexes

RestHighLevelClient esClient = new RestHighLevelClient(
    RestClient.builder(new HttpHost("localhost".9200."http")));@Test
public void index_create(a) throws IOException {
    // Create index - request object
    CreateIndexRequest request = new CreateIndexRequest("test");
    // Send the request and get the response
    CreateIndexResponse response = esClient.indices().create(request, RequestOptions.DEFAULT);
    boolean acknowledged = response.isAcknowledged();
    // Response status
    System.out.println("Operating state =" + acknowledged);
}
Copy the code

Index of the query

RestHighLevelClient esClient = new RestHighLevelClient(
    RestClient.builder(new HttpHost("localhost".9200."http")));@Test
public void index_get(a) throws IOException {

    GetIndexRequest request = new GetIndexRequest("test");

    GetIndexResponse response = esClient.indices().get(request, RequestOptions.DEFAULT);

    System.out.println(Arrays.toString(response.getIndices()));
    System.out.println(response.getMappings());
    System.out.println(response.getSettings());
}
Copy the code

Remove the index

RestHighLevelClient esClient = new RestHighLevelClient(
    RestClient.builder(new HttpHost("localhost".9200."http")));@Test
public void index_delete(a) throws IOException {

    DeleteIndexRequest request = new DeleteIndexRequest("test");

    AcknowledgedResponse response = esClient.indices().delete(request, RequestOptions.DEFAULT);

    System.out.println(response.isAcknowledged());
}
Copy the code

The document operation

Documents, as objects of data, can be manipulated directly through ES’s client objects

The RestHighLevelClient object is created in the same way as the preceding one and is omitted here

Basic Document Operations

Create a document

@Test
public void doc_create(a) throws IOException {
    IndexRequest request = new IndexRequest();
    // Sets the index of the operation and the ID of the currently inserted document
    request.index("people").id("1001");

    User user = new User();
    user.setName("Zhang");
    user.setAge(10);
    user.setSex("Male");

    ObjectMapper mapper = new ObjectMapper();
    String userJson = mapper.writeValueAsString(user);

    request.source(userJson, XContentType.JSON);

    IndexResponse response = esClient.index(request, RequestOptions.DEFAULT);

    System.out.println(response);

    System.out.println("_index" + response.getIndex());
    System.out.println("_id" + response.getId());
    System.out.println("_result" + response.getResult());
}
Copy the code

Update the document

@Test
public void doc_update(a) throws IOException {
    UpdateRequest request = new UpdateRequest();
    request.index("people").id("1001");

    request.doc(XContentType.JSON, "sex"."Female");

    UpdateResponse response = esClient.update(request, RequestOptions.DEFAULT);

    System.out.println(response);
}
Copy the code

Query the document

@Test
public void doc_get(a) throws IOException {

    GetRequest request = new GetRequest();
    request.index("people").id("1001");

    GetResponse response = esClient.get(request, RequestOptions.DEFAULT);

    System.out.println(response);
    System.out.println(response.getSource());
}
Copy the code

Delete the document

@Test
public void doc_delete(a) throws IOException {
    DeleteRequest request = new DeleteRequest();
    request.index("people").id("1001");

    DeleteResponse response = esClient.delete(request, RequestOptions.DEFAULT);
    System.out.println(response);
}
Copy the code

Batch Operation of documents

In general, the data of ES is saved or deleted in batches. If the operation is carried out one by one, the efficiency will be very low. The efficiency can be improved through the API of batch operation

Batch saving

@Test
public void doc_insert_batch(a) throws IOException {
    BulkRequest request = new BulkRequest();
    request.add(new IndexRequest().index("people").id("1001").source(XContentType.JSON, "name"."Zhang"."sex"."Male"."age".10));
    request.add(new IndexRequest().index("people").id("1002").source(XContentType.JSON, "name"."Bill"."sex"."Male"."age".11));
    request.add(new IndexRequest().index("people").id("1003").source(XContentType.JSON, "name"."Fifty"."sex"."Male"."age".12));
    request.add(new IndexRequest().index("people").id("1004").source(XContentType.JSON, "name"."Fifty and 1"."sex"."Male"."age".8));
    request.add(new IndexRequest().index("people").id("1005").source(XContentType.JSON, "name"."Fifty 2"."sex"."Male"."age".9));
    request.add(new IndexRequest().index("people").id("1006").source(XContentType.JSON, "name"."Fifty and 33"."sex"."Male"."age".7));

    BulkResponse response = esClient.bulk(request, RequestOptions.DEFAULT);

    System.out.println(response.getTook());
    System.out.println(Arrays.toString(response.getItems()));
}
Copy the code

Batch delete

@Test
public void doc_delete_batch(a) throws IOException {
    BulkRequest request = new BulkRequest();
    request.add(new DeleteRequest().index("people").id("1001"));
    request.add(new DeleteRequest().index("people").id("1002"));
    request.add(new DeleteRequest().index("people").id("1003"));

    BulkResponse response = esClient.bulk(request, RequestOptions.DEFAULT);

    System.out.println(response.getTook());
    System.out.println(response);
}
Copy the code

Advanced Document Query

Advanced query, group query, paging, sorting, aggregation operations, as common query operations of ES, THE Java client of ES also provides a user-friendly API for operation

Query all data for the specified index

@Test
public void match_all(a) throws IOException {
    SearchRequest request = new SearchRequest();
    request.indices("people");

    SearchSourceBuilder builder = new SearchSourceBuilder();
    builder.query(QueryBuilders.matchAllQuery());

    request.source(builder);

    SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);

    SearchHits hits = response.getHits();
    System.out.println("took:" + response.getTook());
    System.out.println("timeout:" + response.isTimedOut());
    System.out.println("total:" + hits.getTotalHits());
    System.out.println("MaxScore:" + hits.getMaxScore());
    System.out.println("hits========>>");
    for (SearchHit hit : hits) {
        // Displays the results of each query
        System.out.println(hit.getSourceAsString());
    }
    System.out.println("< < = = = = = = = =");
}
Copy the code

The term query

@Test
public void query_term(a) throws IOException {
    SearchRequest request = new SearchRequest();
    request.indices("people");

    SearchSourceBuilder builder = new SearchSourceBuilder();
    builder.query(QueryBuilders.termQuery("age".10));

    request.source(builder);

    SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);

    SearchHits hits = response.getHits();
    System.out.println("took:" + response.getTook());
    System.out.println("timeout:" + response.isTimedOut());
    System.out.println("total:" + hits.getTotalHits());
    System.out.println("MaxScore:" + hits.getMaxScore());
    System.out.println("hits========>>");
    for (SearchHit hit : hits) {
        // Displays the results of each query
        System.out.println(hit.getSourceAsString());
    }
    System.out.println("< < = = = = = = = =");
}
Copy the code

Paging query

@Test
public void query_page(a) throws IOException {
    SearchRequest request = new SearchRequest();
    request.indices("people");

    SearchSourceBuilder builder = new SearchSourceBuilder();
    builder.query(QueryBuilders.matchAllQuery());

    // from specifies the offset from page number, (current page number -1) * size
    builder.from(0);
    builder.size(3);

    request.source(builder);

    SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);

    SearchHits hits = response.getHits();
    System.out.println("took:" + response.getTook());
    System.out.println("timeout:" + response.isTimedOut());
    System.out.println("total:" + hits.getTotalHits());
    System.out.println("MaxScore:" + hits.getMaxScore());
    System.out.println("hits========>>");
    for (SearchHit hit : hits) {
        // Displays the results of each query
        System.out.println(hit.getSourceAsString());
    }
    System.out.println("< < = = = = = = = =");
}
Copy the code

Sorting query

@Test
public void query_sort(a) throws IOException {
    SearchRequest request = new SearchRequest();
    request.indices("people");

    SearchSourceBuilder builder = new SearchSourceBuilder();
    builder.query(QueryBuilders.matchAllQuery());

    builder.sort("age", SortOrder.DESC);
    request.source(builder);

    SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);

    SearchHits hits = response.getHits();
    System.out.println("took:" + response.getTook());
    System.out.println("timeout:" + response.isTimedOut());
    System.out.println("total:" + hits.getTotalHits());
    System.out.println("MaxScore:" + hits.getMaxScore());
    System.out.println("hits========>>");
    for (SearchHit hit : hits) {
        // Displays the results of each query
        System.out.println(hit.getSourceAsString());
    }
    System.out.println("< < = = = = = = = =");
}
Copy the code

Query and filter fields

@Test
public void query_filter(a) throws IOException {
    SearchRequest request = new SearchRequest();
    request.indices("people");

    SearchSourceBuilder builder = new SearchSourceBuilder();
    builder.query(QueryBuilders.matchAllQuery());

    String[] excludes = {};
    String[] includes = {"name"};
    // Filter fields, which specify that only certain fields are included or that certain fields need to be excluded
    builder.fetchSource(includes, excludes);

    request.source(builder);

    SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);

    SearchHits hits = response.getHits();
    System.out.println("took:" + response.getTook());
    System.out.println("timeout:" + response.isTimedOut());
    System.out.println("total:" + hits.getTotalHits());
    System.out.println("MaxScore:" + hits.getMaxScore());
    System.out.println("hits========>>");
    for (SearchHit hit : hits) {
        // Displays the results of each query
        System.out.println(hit.getSourceAsString());
    }
    System.out.println("< < = = = = = = = =");
}
Copy the code

Bool Conditional query, multiple conditions Must

@Test
public void query_bool_must(a) throws IOException {
    SearchRequest request = new SearchRequest();
    request.indices("people");

    SearchSourceBuilder builder = new SearchSourceBuilder();

    BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();

    // Must -> must be matched
    boolQueryBuilder.must(QueryBuilders.matchQuery("age".10));
    boolQueryBuilder.must(QueryBuilders.matchQuery("sex"."Female"));


    builder.query(boolQueryBuilder);

    request.source(builder);

    SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);

    SearchHits hits = response.getHits();
    System.out.println("took:" + response.getTook());
    System.out.println("timeout:" + response.isTimedOut());
    System.out.println("total:" + hits.getTotalHits());
    System.out.println("MaxScore:" + hits.getMaxScore());
    System.out.println("hits========>>");
    for (SearchHit hit : hits) {
        // Displays the results of each query
        System.out.println(hit.getSourceAsString());
    }
    System.out.println("< < = = = = = = = =");
}
Copy the code

Bool conditional query, multi-condition should

@Test
public void query_bool_should(a) throws IOException {
    SearchRequest request = new SearchRequest();
    request.indices("people");

    SearchSourceBuilder builder = new SearchSourceBuilder();

    BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();

    // select * from 'should'; // select * from 'should'
    boolQueryBuilder.should(QueryBuilders.matchQuery("age".10));
    boolQueryBuilder.should(QueryBuilders.matchQuery("sex"."Female"));

    builder.query(boolQueryBuilder);

    request.source(builder);

    SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);

    SearchHits hits = response.getHits();
    System.out.println("took:" + response.getTook());
    System.out.println("timeout:" + response.isTimedOut());
    System.out.println("total:" + hits.getTotalHits());
    System.out.println("MaxScore:" + hits.getMaxScore());
    System.out.println("hits========>>");
    for (SearchHit hit : hits) {
        // Displays the results of each query
        System.out.println(hit.getSourceAsString());
    }
    System.out.println("< < = = = = = = = =");
}
Copy the code

Range queries

@Test
public void query_range(a) throws IOException {
    SearchRequest request = new SearchRequest();
    request.indices("people");

    SearchSourceBuilder builder = new SearchSourceBuilder();

    // Build a range query based on a field
    RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age");

    // Specify the range of query conditional data
    rangeQueryBuilder.gte(9);
    rangeQueryBuilder.lte(12);

    builder.query(rangeQueryBuilder);

    request.source(builder);

    SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);

    SearchHits hits = response.getHits();
    System.out.println("took:" + response.getTook());
    System.out.println("timeout:" + response.isTimedOut());
    System.out.println("total:" + hits.getTotalHits());
    System.out.println("MaxScore:" + hits.getMaxScore());
    System.out.println("hits========>>");
    for (SearchHit hit : hits) {
        // Displays the results of each query
        System.out.println(hit.getSourceAsString());
    }
    System.out.println("< < = = = = = = = =");
}
Copy the code

Fuzzy query

@Test
public void query_fuzzy(a) throws IOException {
    SearchRequest request = new SearchRequest();
    request.indices("people");

    SearchSourceBuilder builder = new SearchSourceBuilder();

    // Fuzzy query, specify the field name for fuzzy query, the following specifies the number of fuzzy matching characters
    builder.query(QueryBuilders.fuzzyQuery("name"."Fifty").fuzziness(Fuzziness.ONE));

    request.source(builder);

    SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);

    SearchHits hits = response.getHits();
    System.out.println("took:" + response.getTook());
    System.out.println("timeout:" + response.isTimedOut());
    System.out.println("total:" + hits.getTotalHits());
    System.out.println("MaxScore:" + hits.getMaxScore());
    System.out.println("hits========>>");
    for (SearchHit hit : hits) {
        // Displays the results of each query
        System.out.println(hit.getSourceAsString());
    }
    System.out.println("< < = = = = = = = =");
}
Copy the code

Highlighting the query

@Test
public void query_highlight(a) throws IOException {
    SearchRequest request = new SearchRequest();
    request.indices("people");

    SearchSourceBuilder builder = new SearchSourceBuilder();
    TermsQueryBuilder termsQueryBuilder = QueryBuilders.termsQuery("sex"."Female");

    // Set the query mode
    builder.query(termsQueryBuilder);

    // Build the highlighted field
    HighlightBuilder highlightBuilder = new HighlightBuilder();
    highlightBuilder.preTags("<font color='red'>"); // Set the label prefix
    highlightBuilder.postTags("</font>"); // Set the label suffix
    highlightBuilder.field("sex"); // Set the highlighted field

    // Set the highlighted query
    builder.highlighter(highlightBuilder);

    // Set the request body
    request.source(builder);

    SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);
    System.out.println(response);
    SearchHits hits = response.getHits();
    System.out.println("took::"+response.getTook());
    System.out.println("time_out::"+response.isTimedOut());
    System.out.println("total::"+hits.getTotalHits());
    System.out.println("max_score::"+hits.getMaxScore());
    System.out.println("hits::::>>");
    for (SearchHit hit : hits) {
        String sourceAsString = hit.getSourceAsString();
        System.out.println(sourceAsString);
        // Prints the highlighted resultMap<String, HighlightField> highlightFields = hit.getHighlightFields(); System.out.println(highlightFields); }}Copy the code

Aggregate query – maximum value

@Test
public void query_aggs_max(a) throws IOException {
    SearchRequest request = new SearchRequest();
    request.indices("people");

    SearchSourceBuilder builder = new SearchSourceBuilder();

    // Aggregate query to get the maximum number of age fields
    builder.aggregation(AggregationBuilders.max("maxAge").field("age"));

    // Set the request body
    request.source(builder);

    SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);
    System.out.println(response);
}
Copy the code

Aggregate query – Group query

@Test
public void query_aggs_group(a) throws IOException {
    SearchRequest request = new SearchRequest();
    request.indices("people");

    SearchSourceBuilder builder = new SearchSourceBuilder();

    // Aggregate query to get the maximum number of age fields
    builder.aggregation(AggregationBuilders.terms("age_groupby").field("age"));

    // Set the request body
    request.source(builder);

    SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);
    System.out.printl	n(response);
}
Copy the code

Document advanced query, JavaAPI operation mode and basic grammar and Http operations, just replace the corresponding Http to API, ES can be better in the program for operation, the next together and see how to integrate the SpringBoot and ElasticSearch, Easier manipulation of ES with index data

Welcome to the wechat public account “code on the finger tip”

Original is not easy, click a praise and then go ~ welcome to pay attention to, to bring you more wonderful articles!

Your likes and attention are the biggest motivation for writing articles

ElasticSearch integration SpringBoot

Maven rely on

Link to the ES client

The index operation

The document operation

Basic Document Operations

Batch Operation of documents

Advanced Document Query

Related Posts

The most complete SpringBoot tutorial in the history of learning from scratch (2)

SpringCloud Upgrade Path 2020.0.x release -10. Use Log4j2 and some core configurations

Cloud native chaotic engineering tools | ChaosBlade Operator Pod