In all applications, cache is a good thing. Because it’s in memory, it’s fast to access, and sometimes even used directly, which invisibly speeds up our search. Cache is enabled by default. You can also disable the cache by:
PUT /my-index-000001
{
"settings": {
"index.requests.cache.enable": false
}
}
Copy the codeIn today’s post, I’m going to share a few tips. Hope to help you with your study. In order to facilitate our explanation, we first use Kibana’s own index to show.
To prepare data
In today’s tutorial, we’ll use the index that comes with Kibana. Open the Kibana interface:
Click on the Add data:
Now our sample data is imported into Elasticsearch. In Elasticsearch we will generate an index called KiBANA_sample_datA_logs.
Use filter or general query?
In Elasticsearch, if you use filter, it will be cached, which will improve performance. When using filter, it is important to note that filter does not have any effect on our final score, that is, if you do not want this part of the query to affect the score, you filter first, and try to use filter.
Let’s start with our kibanA_SAMPLE_datA_logs:
GET kibana_sample_data_logs/_search { "query": { "range": { "@timestamp": { "gte": "2020-07-26", "lte": "2020-08-03"}}}}Copy the codeThe result of the query above is:
"Hits" : {" total ": {" value" : 2096, the "base" : "eq"}, "max_score" : 1.0, "hits" : [{" _index ": "Kibana_sample_data_logs ", "_type" : "_doc", "_id" : "Rm84uHMBZS6OOEix1ZMc", "_score" : 1.0, "_source" : {"agent" : "Mozilla/5.0 (X11; Linux X86_64; rv: 6.0A1) Gecko/20110421 Firefox/ 6.0A1 ", "bytes" : 6219, "clientip" : "223.87.60.27", "extension" : "deb," "geo" : {" srcdest ":" IN: the US ", "SRC" : "IN", "dest" : "US", "coordinates" : {" LAT ": 39.41042861," LON ": -88.8454325}}, "host" : "artifacts. Extice. co", "index" : "Kibana_sample_data_logs "," IP ": "223.87.60.27"," MACHINE ": {" RAM" : 8589934592, "OS" : "win 8"}, "memory" : null, "message" : "223.87.60.27 - [the 2018-07-22 T00:39:02. 912 z] \" GET/elasticsearch/elasticsearch - 6.3.2. HTTP / 1.1 deb_1 \ "200\6219" - \" \"Mozilla/5.0 (X11; Linux x86_64; rv:6.0a1) Gecko/20110421 Firefox/ 6.0A1 \", "PHPMemory" : null, "referer" : "Http://twitter.com/success/wendy-lawrence", "request" : "/ elasticsearch/elasticsearch - 6.3.2. Deb" and "response" : 200, "tags" : ["success", "info"], "TIMESTAMP" : "2020-07-26t00:39:02.912z "," URL ": "Https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.2.deb_1", "utc_time" : "The 2020-07-26 T00:39:02. 912 z", "the event" : {" dataset ":" sample_web_logs "}}},...Copy the codeIn the above query, it is not cached, and we can see that the score (_score) is 1.0.
If we use this query frequently, and we don’t care about the score of the search results, then we can use filter instead of the range query above:
GET kibana_sample_data_logs/_search
{
"query": {
"bool": {
"filter": [
{
"range": {
"@timestamp": {
"gte": "2020-07-26",
"lte": "2020-08-03"
}
}
}
]
}
}
}
Copy the codeThe result of the return result above is:
"Hits" : {" total ": {" value" : 2096, the "base" : "eq"}, "max_score" : 0.0, "hits" : [{" _index ": "Kibana_sample_data_logs ", "_type" : "_doc", "_id" : "Rm84uHMBZS6OOEix1ZMc", "_score" : 0.0, "_source" : {"agent" : "Mozilla/5.0 (X11; Linux X86_64; rv: 6.0A1) Gecko/20110421 Firefox/ 6.0A1 ", "bytes" : 6219, "clientip" : "223.87.60.27", "extension" : "deb," "geo" : {" srcdest ":" IN: the US ", "SRC" : "IN", "dest" : "US", "coordinates" : {" LAT ": 39.41042861," LON ": -88.8454325}}, "host" : "artifacts. Extice. co", "index" : "Kibana_sample_data_logs "," IP ": "223.87.60.27"," MACHINE ": {" RAM" : 8589934592, "OS" : "win 8"}, "memory" : null, "message" : "223.87.60.27 - [the 2018-07-22 T00:39:02. 912 z] \" GET/elasticsearch/elasticsearch - 6.3.2. HTTP / 1.1 deb_1 \ "200\6219" - \" \"Mozilla/5.0 (X11; Linux x86_64; rv:6.0a1) Gecko/20110421 Firefox/ 6.0A1 \", "PHPMemory" : null, "referer" : "Http://twitter.com/success/wendy-lawrence", "request" : "/ elasticsearch/elasticsearch - 6.3.2. Deb" and "response" : 200, "tags" : ["success", "info"], "TIMESTAMP" : "2020-07-26t00:39:02.912z "," URL ": "Https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.2.deb_1", "utc_time" : "The 2020-07-26 T00:39:02. 912 z", "the event" : {" dataset ":" sample_web_logs "}}},Copy the codeThe above score (_score) is shown as 0, which means it does not count toward the score. With this filter approach, the query uses the cache, which means that when you run the same query a second time or later, you will be much faster because the query results come from the cache. Because of our limited data, we can’t see the time difference. For a lot of data
Here’s another example:
GET kibana_sample_data_logs/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"geo.src": "CN"
}
},
{
"range": {
"@timestamp": {
"gte": "2020-07-26",
"lte": "2020-08-03"
}
}
}
]
}
}
}
Copy the codeHere, we send all documents from CN and between 2020-07-26 and 2020-08-03. We’d better change it to:
GET kibana_sample_data_logs/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"geo.src": "CN"
}
}
],
"filter": [
{
"range": {
"@timestamp": {
"gte": "2020-07-26",
"lte": "2020-08-03"
}
}
}
]
}
}
}
Copy the codeOf course, in any case, we can use request_cache=true to enforce cache usage:
GET kibana_sample_data_logs/_search? request_cache=true { "query": { "range": { "@timestamp": { "gte": "2020-07-26", "lte": "2020-08-03" } } } }Copy the code
Setting size=0 on aggregation will automatically cache
Even if request caching is enabled in index Settings, requests with size greater than 0 are not cached. If you want to use caching, one way to do that is to set size to 0.
We type the following command:
GET kibana_sample_data_logs/_search { "size": 0, "aggs": { "src_countries": { "terms": { "field": "geo.src", "size": 10}}}}Copy the codeWe can run the above command multiple times, in the return time:
We’ll see that the second execution takes less time than the first. If we remove size:0, then:
GET kibana_sample_data_logs/_search
{
"aggs": {
"src_countries": {
"terms": {
"field": "geo.src",
"size": 10
}
}
}
}
Copy the codeSo the search is not cached. This may not be obvious for small datasets, but for queries with large datasets, you will see big changes.
If you really don’t want size to be set to 0, then you must clearly indicate this when requesting:
GET kibana_sample_data_logs/_search? request_cache=true { "aggs": { "src_countries": { "terms": { "field": "geo.src", "size": 10 } } } }Copy the codeThe last point to emphasize is that when Lucene’s Segements are merged, these caches will become invalid.