If you have read my previous article “Elasticsearch: Aggregation Introduction”, you will see that in my previous example I used Script to generate a field and perform aggregation statistics on that field. Here’s how it was done:
GET sports/_search
{
"size": 0,
"aggs": {
"age_range": {
"range": {
"script": {
"source":
"""
def dob = doc['birthdate'].value;
return params.now - dob.getYear()
"""
,
"params": {
"now": 2019
}
},
"ranges": [
{
"from": 30,
"to": 31
}
]
}
}
}
}
Copy the codeSo can we do the same thing in Kibana?
The answer is yes. In today’s article, I’ll show you how to use Script in Kibana to produce a field and perform statistics on the field.
What is a Scripted Field?
Scripted Field is a script Field in Chinese. It is based on scripts that compute values in real time
- A value calculated at query time but not indexed
- It can take up a lot of resources and affect the Kibana effect
- No validation
- The Buggy script will generate an exception
- Scripts can be written using “Painless” or “Luncene expressions”
We can refer to the following link:
- www.elastic.co/guide/en/el…
- www.elastic.co/guide/en/el…
We’ll still use a concrete example to talk about the Scripted Fields.
To prepare data
To show you how to produce a Filed, we start with one basic piece of data. Enter the following data in Kibana:
POST _bulk { "index" : { "_index" : "twitter", "_id": 1}} {"user":" Zhang SAN ","message":" Nice weather today, Walk to, "" uid" : 2, "city", "Beijing", "province", "Beijing", "country" : "Chinese", "address" : "haidian district in Beijing, China", "location" : {" lat ":" 39.970718 ", "says lon" : "116. 325747 "}, "DOB" : "1980-12-01"} {" index ": {" _index" : "twitter", "_id" : 2}} {" user ":" liu ", "message" : "in yunnan, the next stop!" , "uid" : 3, "city" : "Beijing", "province", "Beijing", "country" : "Chinese", "address" : "China Beijing dongcheng district stylobate factory three 3", "location" : {" lat ":" 39.904313 ", "says lon" : "116 . 412754 "}, "DOB" : "1981-12-01"} {" index ": {" _index" : "twitter", "_id" : 3}} {" user ":" bill ", "message" : "happy birthday!" , "uid" : 4, "city" : "Beijing", "province", "Beijing", "country" : "Chinese", "address" : "China Beijing dongcheng district", "location" : {" lat ":" 39.893801 ", "says lon" : "116.408986 "}, "DOB":"1982-12-01"} { "index" : { "_index" : "twitter", "_id": 4}} {" user ", "old jia", "message" : "123, gogogo", "uid" : 5, "city" : "Beijing", "province", "Beijing", "country" : "Chinese", "address" : "China Beijing chaoyang district jianguomen", "location ": {" lat" : "39.718256", "says lon" : "116.367910"}, "DOB" : "1983-12-01"} {" index ": {" _index" : "twitter", "_id" : 5}} {"user":" Lao Wang ","message":"Happy BirthDay My Friend!" , "uid" : 6, "city" : "Beijing", "province", "Beijing", "country" : "Chinese", "address" : "chaoyang district in Beijing, China international trade", "location" : {" lat ":" 39.918256 ", "says lon" : "116.4679 10"}, "DOB":"1984-12-01"} { "index" : { "_index" : "twitter", "_id": 6}} {"user":" Lao Wu ","message":" today is my birthday, friends come, what birthday happy!" , "uid" : 7, "city", "Shanghai", "province", "Shanghai", "country" : "Chinese", "address" : "China Shanghai minhang district", "location" : {" lat ":" 31.175927 ", "says lon" : "121.383328 "}, "DOB":"1985-12-01"}Copy the codeThe Twitter schema looks like this:
{
"twitter" : {
"mappings" : {
"properties" : {
"DOB" : {
"type" : "date"
},
"address" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"city" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"country" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"location" : {
"properties" : {
"lat" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"lon" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"message" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"province" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"uid" : {
"type" : "long"
},
"user" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
}
Copy the codeWe found that the Schema contained a field called DOB (Date of Birth), but it did not have a field called age. We know that the age field actually varies depending on the time of query. For example, this year’s query and next year’s query should obviously show different ages. Our goal is to automatically produce an age for each search.
Let’s open Kibana first and click Management:
Let’s select Twitter Index Pattern. If you don’t know what Inxdex pattern is and how to produce Index pattern, please refer to my previous article “Kibana: How to Use Search Bar”. We click on the Scripted Field:
Then click On the Add Scripted Field:
As shown above, we want to create a Scripted field called Age. Its value is 2019-doc[‘DOB’].value.getYear(), which is our current year minus the year of the birthday. Since this script has no validation, it is possible to get an error, so how do we test it?
We can select Preview Results to see our Results. If there are no errors, then we can see the output value. If we changed the script to 2019-doc[‘DOB].value.getYear(), it would be obvious that this is an incorrect script, missing a quote in the DOB. Let’s repeat what we just did, so:
This time, apparently, there was misinformation. We can check our mistakes in this way. Finally, we select the Create Field button:
You’ll see a generated Scripted field called Age. So how do we use this field? We click Discover and search for the age we want:
Obviously, at this point, we can search our index by age. However, it is important to remember that this newly generated field is only for Kibana use and will not be written back to the Twitter index and will produce the corresponding token.
Depending on the age we just generated, we can even make a corresponding visualization:
