Elasticsearch (ES for short) is a distributed, scalable, real-time search and data analysis engine based on Lucene. This section describes how to integrate Elasticsearch with Spring Boot applications. You can access Elasticsearch using the Spring-packaged API.

Spring Data Elasticsearch is a Spring Data subproject that provides support for Elasticsearch integration and access. Note that Spring Data Elasticsearch version 3.2.x is not compatible with 4.x, so you should evaluate it when upgrading to Spring Boot version. This chapter is based on Spring Data Elasticsearch 4.x.

Welcome to pay attention to my B station account

B station account

If the content helps you, welcome everyone to like, favorites + attention

Learning exchange group

Communication group

Rely on

Introduce Spring Data Elasticsearch

<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId>  </dependency>Copy the code

Spring Data Elasticsearch 3.x is only compatible with Elasticsearch 6.x, Elasticsearch 7.x requires Spring Data Elasticsearch 4.x. This is due to the fact that Spring Data Elasticsearch relies on the ES official Client, which is partially incompatible between major versions. Therefore, it is better to select the version with ES server version 1.

If using Rest Client need to introduce the spring – the web, or you will quote Java lang. ClassNotFoundException: org. Springframework. HTTP. HttpHeaders

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
Copy the code

configuration

In the application. Yml plus

Spring: ElasticSearch: rest: connection-timeout: 1s # Connection timeout username: # connection username password: # connection password Read-timeout: Uris :${ES_HOST:localhost}:${ES_PORT:9200} # es SPECIFIES the REST interface addresses, which are separated by commasCopy the code

Create ElasticsearchRestTemplate

@Configuration @EnableElasticsearchRepositories public class ElasticsearchConfiguration extends AbstractElasticsearchConfiguration { @Value("${spring.elasticsearch.rest.uris}") private String uris; @Override @Bean public RestHighLevelClient elasticsearchClient() { final ClientConfiguration clientConfiguration = ClientConfiguration.builder() .connectedTo(uris.split(",")) .withConnectTimeout(Duration.ofSeconds(5)) .withSocketTimeout(Duration.ofSeconds(30)) .build(); return RestClients.create(clientConfiguration).rest(); } @Bean("esRestTemplate") public ElasticsearchRestTemplate esRestTemplate() { return new ElasticsearchRestTemplate(elasticsearchClient()); }}Copy the code

@ EnableElasticsearchRepositories annotations can be accessed using JPA ES, basePackages scanning for Spring package this code to configure Elasticsearch Client, Create ElasticsearchRestTemplate object.

Configuration can be used in any local injection ElasticsearchRestTemplate

@Autowired
private ElasticsearchRestTemplate esRestTemplate;
Copy the code

use

Spring Data Elasticsearch wraps the OFFICIAL ES client API, making it easier to access ES.

ElasticsearchTemplate ElasticsearchOperations uses the Transport Client interface. Starting from version 4.0 is not recommended, please switch to ElasticsearchRestTemplate.

ElasticsearchRestTemplate ElasticsearchOperations using advanced REST client interface implementation.

The default implementation ReactiveElasticsearchOperations ReactiveElasticsearchTemplate responsive,.

This chapter use ElasticsearchRestTemplate, configuration of the previous code, for example, created ElasticsearchRestTemplate object, below illustrate some code is used to implement KQL statement function.

  • Define LogEntity class

    package com.engrz.lab.springboot.elasticsearch;

    import lombok.Data; import org.springframework.data.annotation.Id; import org.springframework.data.elasticsearch.annotations.Document; import org.springframework.data.elasticsearch.annotations.Field; import org.springframework.data.elasticsearch.annotations.FieldType;

    import java.time.LocalDateTime;

    / * *

    • @author Engr-Z
    • @since 2021/2/5

    */ @Document(indexName = “demo-log”) @Data @AllArgsConstructor @NoArgsConstructor public class LogEntity {

    @Id private String id; /** * @field (type = fieldtype.keyword) private String level; /** * @field (type = fieldtype.text) private String title; /** * @field (type = fieldtype.text) private String content; @field (FieldType = FieldType.Date, format = dateformat.date_time) private LocalDateTime recordTime;Copy the code

    }

The @document annotation is used to declare a Document in ES, and indexName is the indexName. Each Field can be declared as a type using the @field annotation

  • insert

    String id = UUID.randomUUID().toString().replaceAll(“-“, “”); LogEntity LogEntity = new LogEntity(ID, “info”, “Add “,” Insert data “, localDateTime.now ()); esRestTemplate.save(logEntity);

  • Modify the

    Document doc = Document.create(); doc.put(“level”, “warn”); doc.put(“recordTime”, LocalDateTime.now()); UpdateQuery.Builder builder = UpdateQuery.builder(“6cba54cbc56a49cc8deee516e18b0121”) .withDocument(doc); UpdateQuery updateQuery = builder.build(); esRestTemplate.update(updateQuery, IndexCoordinates.of(“demo-log”))

Specifies the id of the modified document

  • delete

    esRestTemplate.delete(“6cba54cbc56a49cc8deee516e18b0121”, IndexCoordinates.of(“demo-log”))

  • The query

Because there is no index, the.keyword is added to the following query condition fields. Elasticsearch KQL will be covered separately

  1. Requirement Description: Query the total number of logs

KQL:

GET demo-log/_count
Copy the code

JAVA:

NativeSearchQueryBuilder searchQueryBuilder = new NativeSearchQueryBuilder();
long count = esRestTemplate.count(searchQueryBuilder.build(), IndexCoordinates.of("demo-log"));
log.info("count -> {}", count);
Copy the code

  1. Requirement description: Search for logs whose level is error within 3 days, and select the first 20 logs in reverse chronological order

KQL:

GET demo-log/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "level": "error"
          }
        },
        {
          "range": {
            "recordTime.keyword": {
              "gte": "2021-02-05T10:00:00",
              "format": "yyyy-MM-dd HH:mm:ss"
            }
          }
        }
      ]
    }
  },
  "sort": [
    {
      "recordTime.keyword": {
        "order": "desc"
      }
    }
  ],
  "size": 20
}
Copy the code

JAVA:

BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder(); // Query boolQueryBuilder.must().add(new TermQueryBuilder("level.keyword", "error")); RangeQueryBuilder RangeQueryBuilder = QueryBuilders. RangeQuery (" recordtime.keyword "); rangeQueryBuilder.gte(LocalDateTime.now().minusDays(3)); boolQueryBuilder.must().add(rangeQueryBuilder); ↓ = PageRequest. Of (0, 20, sort.by (" recordtime.keyword "). Descending ()); NativeSearchQueryBuilder searchQueryBuilder = new NativeSearchQueryBuilder(); searchQueryBuilder.withQuery(boolQueryBuilder) .withPageable(pageRequest); Query searchQuery = searchQueryBuilder.build(); SearchHits<LogEntity> hits = esRestTemplate.search(searchQuery, LogEntity.class); List<SearchHit<LogEntity>> hitList = hits.getSearchHits(); log.info("hit size -> {}", hitList.size()); ForEach (hit -> {log.info(" return data: ", hit.getContent().toString())); });Copy the code

The ES query returns a maximum of 10000 entries by default. If the value exceeds 10000 entries, you need to modify ES Settings or use rolling query. ES paging is covered in a new section

  1. Requirement Description: Displays the number of logs at each log level

KQL:

GET demo-log/_search
{
  "aggs": {
    "termLevel": {
      "terms": {
        "field": "level.keyword"
      }
    }
  }
}
Copy the code

JAVA:

TermsAggregationBuilder termsAggregationBuilder = AggregationBuilders.terms("aggTerms").field("level.keyword");

NativeSearchQueryBuilder searchQueryBuilder = new NativeSearchQueryBuilder();
searchQueryBuilder.addAggregation(termsAggregationBuilder);

Query searchQuery = searchQueryBuilder.build();
SearchHits<LogEntity> hits = esRestTemplate.search(searchQuery, LogEntity.class);
Terms aggTerms = hits.getAggregations().get("aggTerms");
for (Terms.Bucket bucket : aggTerms.getBuckets()) {
    log.info("level={}, count={}", bucket.getKey(), bucket.getDocCount());
}
Copy the code

The AggregationBuilders class can create other types of aggregate queries, such as the AggregationBuilders. Sum count

Integrate JPA

We can also access ES through Spring JPA, making it as easy to manipulate ES as to access the database. Using JPA need to use @ EnableElasticsearchRepositories annotations, front at the time of configuration.

Create the LogRepository interface to inherit ElasticsearchRepository

public interface LogRepository extends ElasticsearchRepository<LogEntity, String> {
}
Copy the code

Spring Boot will automatically scan and assemble and inject directly into the code for use.

@Autowired
private LogRepository logRepository;
Copy the code

LogRepository has common methods by inheriting the ElasticsearchRepository interface.

  • insert

    String id = UUID.randomUUID().toString().replaceAll(“-“, “”); LogEntity LogEntity = new LogEntity(ID, “info”, “JPA add “, “JPA insert a data “, localDatetime.now ()); logRepository.save(logEntity);

  • Modify the

Did not provide. Update data to rebuild the index, ES is not recommended to update a large amount of data.

  • delete

    logRepository.deleteById(“cd5c2ac848724acf8d66522e743eed87”);

  • count

    long count = logRepository.count(); log.info(“count -> {}”, count);

  • The query

Find by id:

Optional<LogEntity> entity = logRepository.findById("b64446b58492454c8576504b68767332");
log.info("size -> {}", entity.orElse(null));
Copy the code

  • Custom conditions

Spring JPA allows conditions to be written in method names, and the framework automatically resolves rules without having to write implementation methods manually.

  1. Requirement: Query data at a specified log level on paging pages

    List findByLevel(String level, Pageable pageable);

If methods in Repository have Pageable parameters, Spring automatically uses the parameter information in Pageable for paging and sorting. The method name supports and, OR, and like queries, as well as lt, GT, LTE, GTE, and eq comparisons. Refer to the official list Query Creation

KQL query

Although Spring JPA supports multiple conditional queries, method names can be very long if there are many criteria. KQL queries can be passed in using the @Query annotation

  1. Requirement description: Query specified log level data in pages and within a time range

    @Query(“{\n” + ” “bool”: {\n” + ” “must”: [\n” + ” {\n” + ” “term”: {\n” + ” “level”: ?0\n” + ” }\n” + ” },\n” + ” {\n” + ” “range”: {\n” + ” “recordTime.keyword”: {\n” + ” “gte”: ?1,\n” + ” “lte”: ?2\n” + ” }\n” + ” }\n” + ” }\n” + ” ]\n” + ” }\n” + ” }”) List exampleQuery(String level, LocalDateTime startDateTime, LocalDateTime endDateTime, Pageable pageable);

General scenario recommended JPA, complex queries or need ElasticsearchRestTemplate. The combination of the two is more efficient and concise.