Elasticsearch

Elasticsearch is a distributed, scalable, real-time search and data analysis engine. Elasticsearch is a Lucene-based search server. It provides a distributed multi – user – capable full – text search engine based on RESTful Web interface.

Wikipedia -Elasticsearch

1.1 Core Concepts of Elasticsearch

  • Index index: A structure for storing document data, similar to a database in a relational database
  • Type type: The logical structure used to store document, which is a descendant of index, similar to a table in a relational database
  • Document: JSON format, similar to row data in a relational database

Elasticsearch uses a structure called an inverted index, with Lucene inverted strings as the underlying layer. This structure is suitable for fast full-text searches, where an index consists of all non-repeating lists in a document and, for each word, a list of documents containing it. An Elasticsearch index is made up of multiple Lucene indexes.

< 2 > ElasticSearch: index, type, document

Create Elasticsearch environment

You need to configure the JDK environment (using JDK1.8 or later) and node environment (to start Elasticsearch Head) locally. In addition, you need to install and configure elasticSearch.

  • Elasticsearch
  • Elasticsearch Head
  • Ik
  • Kibana
  • Huawei Cloud Image download -Elasticsearch
  • Github download -Elasticsearch Head
  • Huawei Cloud Image Download -Logstash (This article is not required)
  • Making the download – Ik
  • Huawei Cloud Image download -Kibana

For all downloads, download version 7.6.1.

2.1 Configuring Elasticsearch Cross-domain Request

Configure a cross-domain request for Elasticsearch Head access in the \config\ ElasticSearch. yml file in the Elasticsearch installation directory.

http.cors.enabled: true 
http.cors.allow-origin: "*"
Copy the code

2.2 Ik word segmentation configuration

Create a new ik directory under \config\plugins of Elasticsearch installation directory and unzip the ik package.

2.3 Kibana localization

Set the default language to Chinese in the Kibana installation directory \config\kibana.yml file.

i18n.locale: "zh-CN"
Copy the code

2.4 start the Elasticsearch

Select * from Elasticsearch folder \bin\ Elasticsearch.

http://localhost:9200/

2.5 Starting Elasticsearch Head

Under the Elasticsearch Head directory, run the NPM run start command to start Elasticsearch Head.

http://localhost:9100/

2.6 start Kibana

Double-click the \bin\ Kibana. bat file in the Kibana installation directory to access:

http://localhost:5601/app/kibana#/dev_tools/console

Kibana operation ES

Kibana operation ES can refer to the following reference link.

  • The text type will split words, while the keyword type will not
  • Learn from Scratch Elasticsearch series – Use Kibana to implement ES basic operations
  • ES7 Study Notes (8) Data increase, delete and change

Elasticsearch is implemented in Spring Boot

Maven dependencies for Elasticsearch and HighLevelClient

<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId>  </dependency> <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-high-level-client</artifactId> <version>7.61.</version>
</dependency>
Copy the code

Specify version 7.6.1 for Elasticsearch.

<properties>
    <elasticsearch.version>7.61.</elasticsearch.version>
</properties>
Copy the code

RestHighLevelClient is configured as follows:

@Configuration
public class ESConfig {
    @Bean
    public RestHighLevelClient restHighLevelClient(a) {
        RestClientBuilder builder = RestClient.builder(new HttpHost("127.0.0.1".9200."http"));
        return newRestHighLevelClient(builder); }}Copy the code

Elasticsearch global configuration

Spring. Elasticsearch. Rest uris, = http://127.0.0.1:9200Copy the code

5. Shop search suggestions and GEO location search

This article implementation source code is as follows, welcome Star and Fork.

Github.com/just-right/…

The reference links are as follows:

  • ElasticSearch tutorial for ElasticSearch
  • Pit: ElasticSearch cannot resolve serialized GeoPoint fields
  • URL online transcoding
  • Reference link 1: ES7 Study Notes (xii) Highlighting and search suggestions
  • 3. GEO location search

5.1 Creating a Store Index

Use Kibana to create Shop index, including shopName (Shop name), address (Shop address), tags (Shop label), location address location field and suggest search suggestion field, this paper will be the tags array as the value of the search suggestion field.

PUT /shop
{
    "settings":{
        "analysis":{
            "analyzer":{
                "default":{
                    "type":"ik_max_word"
                }
            }
        }
    },
    "mappings":{
        "dynamic_date_formats": [
             "MM/dd/yyyy",
             "yyyy/MM/dd HH:mm:ss",
             "yyyy-MM-dd",
             "yyyy-MM-dd HH:mm:ss"
         ],
        "properties":{
			"shopName":{
				"type":"text"
			},
			"address":{
				"type":"text"
			},
			"tags":{
				"type":"text"
			},
            "suggest":{
                "type":"completion"
            },
			 "location":{
                "type":"geo_point"
            }
        }
    }
}
Copy the code

5.2 Added store documents

Use PostMan to test, call new document interface, new document:

http://127.0.0.1:8080/es/document/create/shop

The backend processing of new document request, using Gson to parse data, using Json parsing will fail serialization, the reason is agentd.cn/archives/es… .

The back-end implementation core code is as follows:

@PostMapping(value = "/es/document/create/shop")
public String createShopDocument(@RequestBody Shop shop) throws IOException {
    IndexRequest indexRequest = new IndexRequest("shop");
    indexRequest.timeout(TimeValue.timeValueSeconds(1));
    shop.setSuggest(new Completion(shop.getTags()));
    indexRequest.source(new Gson().toJson(shop), XContentType.JSON);
    IndexResponse indexResponse = highLevelClient.index(indexRequest, RequestOptions.DEFAULT);
    return indexResponse.toString();
}
Copy the code

5.3 Search stores based on search suggestions and GEO

PostMan was used for the test, the store document search interface was called, the store name search was used, and the latitude and longitude and distance len of the current geographical location were passed in for distance filtering.

http://127.0.0.1:8080/es/document/search/shop

The back-end implementation core code is as follows:

@PostMapping(value = "/es/document/search/shop")
public String shopSearch(@RequestBody ShopSearchInfo shopSearchInfo) throws IOException {
    SearchRequest searchRequest = new SearchRequest("shop");
    SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
    /** * search suggestions */
    CompletionSuggestionBuilder suggestionBuilder = SuggestBuilders
            .completionSuggestion(Shop.SUGGEST)
            .prefix(shopSearchInfo.getShopName());
    SuggestBuilder suggestBuilder = new SuggestBuilder();
    // Customize the search name
    suggestBuilder.addSuggestion("shopSearch", suggestionBuilder);
    /** * search */
    GeoPoint geoPoint = new GeoPoint(shopSearchInfo.getLatitude(), shopSearchInfo.getLongitude());
    // query the distance in geo
    QueryBuilder queryBuilder = QueryBuilders.geoDistanceQuery(Shop.LOCATION)
            .distance(shopSearchInfo.getLen(), DistanceUnit.KILOMETERS)
            .point(geoPoint);
    sourceBuilder.suggest(suggestBuilder);
    sourceBuilder.query(queryBuilder);
    searchRequest.source(sourceBuilder);
    SearchResponse response = highLevelClient.search(searchRequest, RequestOptions.DEFAULT);
    List<String> geoResList = new ArrayList<>();
    SearchHit[] searchHits = response.getHits().getHits();
    if (searchHits == null || searchHits.length <= 0) {
        return "No information found!";
    }
    for (SearchHit hit : searchHits) {
        geoResList.add(hit.getId());
    }
    CompletionSuggestion suggestion = response.getSuggest().getSuggestion("shopSearch");
    List<CompletionSuggestion.Entry.Option> optionList = suggestion.getOptions();
    if (optionList == null || optionList.size() <= 0) {
        return "No information found!";
    }
    List<String> suggestResList = new ArrayList<>();
    for (CompletionSuggestion.Entry.Option option : optionList) {
        suggestResList.add(option.getHit().getId());
    }
    /** * find the same element */
    geoResList.retainAll(suggestResList);
    if (geoResList == null || geoResList.size() <= 0) {
        return "No information found!";
    }
    List<SearchResultInfo> resList = new ArrayList<>();
    for (String shopID : geoResList) {
        GetRequest request = new GetRequest("shop", shopID);
        GetResponse searchResponse = highLevelClient.get(request, RequestOptions.DEFAULT);
        Map<String, Object> resultMap = searchResponse.getSourceAsMap();
        if (resultMap == null || resultMap.size() <= 0) {
            continue;
        }
        String shopName = (String) resultMap.get(Shop.SHOPNAME);
        Map<String, Double> geoInfo = (Map<String, Double>) resultMap.get(Shop.LOCATION);
        double _lon1 = geoInfo.get("lon");
        double _lat1 = geoInfo.get("lat");
        double _lon2 = shopSearchInfo.getLongitude();
        double _lat2 = shopSearchInfo.getLatitude();
        double len = this.getDistance(_lat1, _lon1, _lat2, _lon2);
        SearchResultInfo resultInfo = new SearchResultInfo();
        resultInfo.setShopID(shopID).setShopName(shopName).setLen(len);
        resList.add(resultInfo);
    }
    /** * sort by distance - in reverse order */
    resList = resList.stream().distinct().sorted(Comparator
            .comparing(SearchResultInfo::getLen)).collect(Collectors.toList());
    return JSONObject.toJSONString(resList);
}

public double getDistance(double _lat1, double _lon1, double _lat2, double _lon2) {
    double lat1 = (Math.PI / 180) * _lat1;
    double lat2 = (Math.PI / 180) * _lat2;
    double lon1 = (Math.PI / 180) * _lon1;
    double lon2 = (Math.PI / 180) * _lon2;
    // Radius of the earth
    double R = 6378.1;
    double d = Math.acos(Math.sin(lat1) * Math.sin(lat2) + Math.cos(lat1) * Math.cos(lat2) * Math.cos(lon2 - lon1)) * R;
    return new BigDecimal(d).setScale(4, BigDecimal.ROUND_HALF_UP).doubleValue();
}
Copy the code