preface
ElasticSearch6.8.4: poetry search ver 1.0
Suitable for students with a certain ES foundation, just say HighLevelClient function implementation.
Environment configuration: Docker (ElasticSearch6.8.4+Kibana6.8.4+IK descriptor) +SpringBoot 2.2.13.RELEASE + MySQL5.7 + MyBatis + Redis To control extended dictionaries and disabled dictionaries)
show
The source code
Gitee.com/ven1ce/elas…
How does it work?
Change the database account password in application.yml
create table poem
(
id bigint auto_increment
primary key,
no varchar(255) not null,
title varchar(100) not null,
author varchar(10) not null,
content varchar(500) not null,
type tinyint default 0 null comment 'Four quatrains of poetry types',
author_detail varchar(500) not null,
source varchar(50) default 'network' not null comment 'source',
create_time datetime null,
update_time datetime null.constraint no
unique (no)
)
comment 'poetry' charset = utf8mb4;
Copy the codeConfigure the elasticSearch address
Configuration index
PUT /es
{
"mappings": {
"poem":{
"properties":{
"id":{
"type":"keyword"
},
"no":{
"type":"keyword"
},
"title":{
"type":"text",
"analyzer":"ik_max_word"
},
"author":{
"type":"text",
"analyzer":"ik_max_word"
},
"content":{
"type":"text",
"analyzer":"ik_max_word"
},
"type":{
"type":"integer"
},
"authorDetail":{
"type":"text",
"analyzer":"ik_max_word"
},
"source":{
"type":"keyword"
},
"createTime":{
"type":"date",
"format":"yyyy-MM-dd HH:mm:ss"
},
"updateTime":{
"type":"date",
"format":"yyyy-MM-dd HH:mm:ss"
}
}
}
}
}
Copy the codeStart the front end, NPM Run serve
RestHighLevelClient operation
– ElasticSearch upgrade was pretty fast, I couldn’t find a good article online, I didn’t know what to do with RestHighLevelClient Fortunately, the official website document is still detailed, and indeed it is the best official document, although not comprehensive enough, but slowly grinding can come down. so
It is recommended to read the official English document directly.
It is recommended to read the official English document directly.
It is recommended to read the official English document directly.
Important things to say three times, no matter what version 6.x or 7.x you are, do not look for other articles on the web (only as a reference for the comprehensive approach)!
Afraid of some students can not find, I will directly help you to paste the following, pay attention to the version of the same.
ElasticSearch Rest Client
ElasticSearch Official Documentation
Generate ES documents for all data in the database
The first reaction of this feature is the batch processing method of ES, let’s look at the official documentation of ES.
Of course we can also add other requests to the batch request
The website also has additional configurable information and how to synchronize (asynchronous) requests, so check it out for your own needs.
A synchronous request is used here, first fetching all the data in the database (as a case, not a spritz), then generating a new index request from the database data and adding it to the batch. Then highLevelClient initiates the request, obtains the bulkResponse of the returned data of the batch, and processes it.
Code:
/** * Generate es document for all database data */
@Override
public void insertEsAllDate(a) throws IOException {
List<Poem> poems = poemMapper.queryAll();
BulkRequest bulkRequest = new BulkRequest();
for (Poem poem : poems) {
String s = JSON.toJSONString(poem);
bulkRequest.add(new IndexRequest("es"."poem",poem.getNo()).source(s, XContentType.JSON));
}
BulkResponse bulkResponse = highLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT);
if (bulkResponse.hasFailures()){
for(BulkItemResponse item : bulkResponse.getItems()) { System.out.println(item.getFailureMessage()); }}}Copy the codeClear all documents in ES
Looking at the documentation, we can see that there is a delete-by-query operation. Literally delete based on the query. So if we want to achieve the operation of clearing the document, then just need to query all, and then delete according to the query.
So how do we get query all documents, look at the official search document and you can see the demonstration already given.
This is slightly different from the documentation. Same idea.
Code:
/** * clear the table with 'es' index */
@Override
public void clearEsAllDate(a) throws IOException {
DeleteByQueryRequest deleteByQueryRequest = new DeleteByQueryRequest("es");
deleteByQueryRequest.setQuery(new MatchAllQueryBuilder());
deleteByQueryRequest.setConflicts("proceed");// Version conflicts do not abort the process
highLevelClient.deleteByQuery(deleteByQueryRequest, RequestOptions.DEFAULT);
}
Copy the codePaging document retrieval
Go straight to the search document
Set the sourceBuilder form and size attributes to page query. We can also set the timeout.
With the sourceBuilder set up, we can pass it on to the search request. Next, deal with the return value.
Code:
/** * get all es files */
@Override
public List<Poem> getAllEs(a) throws IOException {
List<Poem> result = new ArrayList<>();
SearchRequest searchRequest = new SearchRequest("es");
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(QueryBuilders.matchAllQuery());
sourceBuilder.from(0);
sourceBuilder.size(10);
searchRequest.source(sourceBuilder);
SearchResponse search = highLevelClient.search(searchRequest, RequestOptions.DEFAULT);
SearchHits searchHits = search.getHits();
SearchHit[] hits = searchHits.getHits();
if( hits ! =null&& hits.length ! =0) {for(SearchHit hit : hits) { String sourceAsString = hit.getSourceAsString(); Poem poem = JSON.parseObject(sourceAsString, Poem.class); result.add(poem); }}return result;
}
Copy the codeHighlighting the search
Query for Highlighting requests in official documents
We first set up SearchSourceBuilder and HighlightBuilder.
Filed is the highlighted field in HighlightBuilder, and preTags and postTags are the HTML for our highlighted field.
Set up multiple field searches using boolQuery. Configure the priority boost (the bigger the higher the priority) and the word segmentation.
// Set the search field
sourceBuilder.query(QueryBuilders.boolQuery()
.should(QueryBuilders.multiMatchQuery(searchString,"title").boost(4).analyzer("ik_smart"))
.should(QueryBuilders.multiMatchQuery(searchString,"author").boost(3).analyzer("ik_smart"))
.should(QueryBuilders.multiMatchQuery(searchString,"content").boost(2).analyzer("ik_smart"))
.should(QueryBuilders.multiMatchQuery(searchString,"authorDetail").boost(1).analyzer("ik_smart")));Copy the codeWe can get the highlighted fields separately from the results. Because the highlighted field String is distinguished from the attributes in the entity. Here you need to think about how to put highlighted strings into entity attributes. So since properties are constantly changing, we can’t write a dead-end program, so think of the set method that uses reflection to get the entity class. Use the set method to inject an attribute with a highlighted label.
// Get the highlighted field setting entity
Map<String, HighlightField> highlightFields = hit.getHighlightFields();
//s is the name of the entity field
for (String s : highlightFields.keySet()) {
HighlightField highlightField = highlightFields.get(s);
Text[] fragments = highlightField.getFragments();
// Gets the attributes of the highlighted field
StringBuilder sb = new StringBuilder();
for (Text fragment : fragments) {
sb.append(fragment.toString());
}
// We use reflection to get the set method because we don't know which attribute it is
try{
Class<? extends Poem> aClass = poem.getClass();
// Convert fields to hump format in accordance with the set method naming convention
String formatString = StringUtils.convertToCamelCase(s);
// Get the method by its name
Method declaredMethod = aClass.getDeclaredMethod("set" + formatString,String.class);
// Execute the method to inject the highlighted label
declaredMethod.invoke(poem,sb.toString());
} catch(NoSuchMethodException | InvocationTargetException | IllegalAccessException e) { e.printStackTrace(); }}Copy the codeCode:
/** ** highlight search **@paramSearchString searchString *@returnResults the * /
@Override
public List<Poem> search(String searchString) throws IOException {
// Todo adds redis hot words to the list
// Search for es first
SearchRequest searchRequest = new SearchRequest("es");
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
// Set the highlight
HighlightBuilder highlightBuilder = new HighlightBuilder();
highlightBuilder.field("title");
highlightBuilder.field("author");
highlightBuilder.field("content");
highlightBuilder.field("authorDetail");
highlightBuilder.preTags("<em style=\"color:red\" >");
highlightBuilder.postTags("</em>");
sourceBuilder.highlighter(highlightBuilder);
// Set the search field
sourceBuilder.query(QueryBuilders.boolQuery()
.should(QueryBuilders.multiMatchQuery(searchString,"title").boost(4).analyzer("ik_smart"))
.should(QueryBuilders.multiMatchQuery(searchString,"author").boost(3).analyzer("ik_smart"))
.should(QueryBuilders.multiMatchQuery(searchString,"content").boost(2).analyzer("ik_smart"))
.should(QueryBuilders.multiMatchQuery(searchString,"authorDetail").boost(1).analyzer("ik_smart"))); searchRequest.source(sourceBuilder);// Get the result
SearchResponse search = highLevelClient.search(searchRequest, RequestOptions.DEFAULT);
SearchHits searchHits = search.getHits();
SearchHit[] hits = searchHits.getHits();
List<Poem> result = new ArrayList<>();
if(hits ! =null&& hits.length ! =0) {for (SearchHit hit : hits) {
String sourceAsString = hit.getSourceAsString();
// Convert to entity class
Poem poem = JSON.parseObject(sourceAsString, Poem.class);
// Get the highlighted field setting entity
Map<String, HighlightField> highlightFields = hit.getHighlightFields();
for (String s : highlightFields.keySet()) {
HighlightField highlightField = highlightFields.get(s);
Text[] fragments = highlightField.getFragments();
StringBuilder sb = new StringBuilder();
for (Text fragment : fragments) {
sb.append(fragment.toString());
}
// We use reflection to get the set method because we don't know which attribute it is
try{
Class<? extends Poem> aClass = poem.getClass();
// Convert to hump format to capitalize the first letter
String formatString = StringUtils.convertToCamelCase(s);
Method declaredMethod = aClass.getDeclaredMethod("set" + formatString,String.class);
declaredMethod.invoke(poem,sb.toString());
} catch(NoSuchMethodException | InvocationTargetException | IllegalAccessException e) { e.printStackTrace(); } } result.add(poem); }}// Todo searches the database if it misses
// Add es to todo
return result;
}
Copy the codeconclusion
This is just a brief note of the general operations in learning ES, and most of them are also very simple. At present, there are many target functions that have not been realized. Recently, I have been busy, so I have to put down and do other things for the time being. Come back and try again sometime.
But having said that, I’m going to say it here
It is recommended to read the official English document directly!
Recommend to read the official English document directly!!
Recommend to read the official English document directly!!