This is the first day of my participation in the August More Text challenge
This section describes how to record an ES online service exception
The business scenario
Online business do video recommendation, using ES scoring function, dynamic sorting recommendation, specific implementation can learn big program _ art _ life article, very comprehensive: juejin.cn/post/698026…
The error log
- error message:{“@type”:”org.elasticsearch.ElasticsearchStatusException”,”detailedMessage”:”org.elasticsearch.ElasticsearchStat usException: Elasticsearch exception [type=search_phase_execution_exception, reason=all shards failed]”,”fragment”:true,”headerKeys”:[],”localizedMessage”:”Elasticsearch exception [type=search_phase_execution_exception, reason=all shards failed]”,”message”:”Elasticsearch exception [type=search_phase_execution_exception, reason=all shards failed]”,”metadataKeys”:[“es.phase”]
- stackTrace:[{“className”:”org.elasticsearch.rest.BytesRestResponse”,”fileName”:”BytesRestResponse.java”,”lineNumber”:177 ,”methodName”:”errorFromXContent”,”nativeMethod”:false},{“className”:”org.elasticsearch.client.RestHighLevelClient”,”fil eName”:”RestHighLevelClient.java”,”lineNumber”:1793,”methodName”:”parseEntity”,”nativeMethod”:false},{“className”:”org.e lasticsearch.client.RestHighLevelClient”,”fileName”:”RestHighLevelClient.java”,”lineNumber”:1770,”methodName”:”parseResp onseException”,”nativeMethod”:false},{“className”:”org.elasticsearch.client.RestHighLevelClient”,”fileName”:”RestHighLev elClient.java”,”lineNumber”:1527,”methodName”:”internalPerformRequest”,”nativeMethod”:false},{“className”:”org.elasticse arch.client.RestHighLevelClient”,”fileName”:”RestHighLevelClient.java”,”lineNumber”:1484,”methodName”:”performRequest”,” nativeMethod”:false},{“className”:”org.elasticsearch.client.RestHighLevelClient”,”fileName”:”RestHighLevelClient.java”,” lineNumber”:1454,”methodName”:”performRequestAndParseEntity”,”nativeMethod”:false},{“className”:”org.elasticsearch.clien t.RestHighLevelClient”,”fileName”:”RestHighLevelClient.java”,”lineNumber”:970,”methodName”:”search”,”nativeMethod”:false }
Problem orientation
1. Critical log error location:
- type=search_phase_execution_exception, reason=all shards failed
- “className”:”org.elasticsearch.rest.BytesRestResponse”,”fileName”:”BytesRestResponse.java”,”lineNumber”:177,”methodName” :”errorFromXContent”,”nativeMethod”:false
2. Baidu search solutions:
- And the answers that I got were pretty much the same:When term query is used, the type of the keyword in es must be keyword rather than text because it is an accurate match
- Through baidu answers, combined with the actual project analysis, our business tables used in the field may be the cause of only one field belongs to type text, but in the es version we use, as well as the pressure test we launch this time didn’t result in an error because the problem, so can be due to eliminate this problem, need to determine the problem in a different way
To solve the process
1. Simulate the real environment
- Obtain the request parameters from the log, local simulation online request, request the same ES data in the test environment, to achieve the simulation of the real environment
2. Get the query DSL
- SearchSourceBuilder, the final query result output
logger.info(searchSourceBuilder.toString());
Copy the code- The DSL query statement of ES is obtained through log output
{
"size":10,
"query":{
"function_score":{
"query":{
"bool":{
"filter":[
{
"term":{
"status":{
"value":1,
"boost":1
}
}
}
],
"adjust_pure_negative":true,
"boost":1
}
},
"functions":[
{
"filter":{
"match_all":{
"boost":1
}
},
"field_value_factor":{
"field":"new_test",
"factor":1,
"modifier":"none"
}
}
],
"score_mode":"sum",
"boost_mode":"sum",
"max_boost":340282350000000000000000000000000000000,
"min_score":70,
"boost":1
}
},
"_source":{
"includes":[
"xxxx"
],
"excludes":[
]
},
"sort":[
{
"_score":{
"order":"asc"
}
},
{
"xxxx":{
"order":"asc"
}
}
]
}
Copy the code3. Execute the DSL statement
- When executing a DSL statement in Kibana, you can get an error message from the query. The error message is very clear. The new_test field is used in the execution of functions, but because some data in the table lacks this field information, the function calculation cannot be performed, and this error exception is reported
- Identify the causes of problems through troubleshooting:
-
When the functions of ES are used, if the script script of script_score is used, it can judge whether the field is empty or not, but this will lead to poor execution efficiency of ES, so we use the function of ES in the actual project to achieve scoring processing. There is no way to determine whether a field is empty, so an error will be reported if the specified field does not exist during score calculation
-
Although filter can be used to filter out the data that does not contain the specified field, the business side will adjust the data in real time, and we will update the data in real time. It is inevitable that the invalid data will become valid data due to state adjustment, but the specified field is not included, which will lead to exceptions, and this is also the reason for our error
-
"root_cause": [
{
"type": "exception",
"reason": "Unable to find a field mapper for field [new_test]. No 'missing' value defined."
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true
Copy the code4. Problem repair
- Update data that does not contain the specified field with Kibana, increasing the default value
POST /test/_doc/_update_by_query { "query": { "bool" : { "must_not" : { "term" : { "field" : "Balance"}}}}, "script" : {" source ":" CTX. _source [' balance '] = 70.0 "}}Copy the code- Adjust the real-time streaming scheme to include insert and UPDATE processing and ensure that the data always has default values