We know that Kibana is presented and analyzed as Elasticsearch data. In Kibana, search is all over almost every page. Search is critical to Elastic. Knowing how to search in Kibana is very important. It is not limited to our search for input words, or filtering for some words. It also includes:

  • Fuzzy query
  • Search by building regular expressions or wildcards
  • Weighted search for certain fields

 

Kibana search method

In Kibana search, there are three ways to search:

  • KQL (Kibana Query Language)

  • Lucene

 

When we turn off the KQL switch, we have the following Settings:

  • DSL: This works with Dev Tools in Kibana. I’ve used this in many of my previous tutorials. I’m not going to tell you all about it here. You can refer to my previous article “Getting Started with Elasticsearch (2)” for more details.

 

Below, we describe how to use each of these to do our search.

What is index pattern?

Index Pattern: This points to one or more Elasticsearch indexes and tells Kibana which indexes you want to operate on.

All indexes exist in Elasticsearch. We need the Index pattern for Kibana to operate on this Index.

 

The data type

For Elasticsearch, there are two types of data that can be analyzed:

  • Time series: Each event in this type of data is related to a specific moment in Time. Generally speaking, growth is rapid. There is usually a time related field.
{" Tweet ": "Wow Elasticsearch 7.0 seems awesome!" "Timestamp ": September 1st 2017, 07:15:40.035}Copy the code
  • Static data: This type of data usually grows very slowly and usually has no corresponding time stamp.
{
   "cuisine": "French",
   "ingredients": "Cheese, flour, butter, eggs, milk, nutmeg",
   "time_in_min": 50,
   "level": "easy"
}
Copy the code

When we create our Index pattern, we need to select our data type:

Above we need to enter the corresponding index pattern according to the name of our index. It can point to a single index, or it can point to multiple indexes through wildcards. If your index contains a Time related field, Kibana will automatically pop up an option to let us choose whether to require Time Filter:

If we select the Time Filter field, it will be treated as a Time series. Otherwise, we can choose not to use the Time Filter, so we can only search for the index, not the Time series related operations. In this case, the Time picker we describe below will no longer apply.

If we want to delete an Index pattern, we can also select delete from the above page:

We can click on the star icon in the upper right cornerMake this Index our default Index. In the figure above, we can also see that some fields can be searched and some can be aggregated. If you want to learn more about this, please read my previous post in detail.Inverted Index, DOC_values and Source of Elasticsearch”.

 

To prepare data

We can use the data that comes with Kibana to demonstrate this. We load the data as follows:

 

Select Add sample Data:

Select “Add Data” so we can load the sample data we need into Elasticsearch.

There are two important things to understand before conducting a search:

  • Select the Index Pattern that you want to operate
  • Time picker: This is very important for Time series data. If the corresponding time is not selected properly, there may be no data in Kibana.

Kibana search

Let’s start by looking at the contents of a document for our KibanA_SAMple_datA_FLIGHTS index.

"_source" : { "FlightNum" : "9HY9SWR", "DestCountry" : "AU", "OriginWeather" : "Sunny", "OriginCityName" : Frankfurt am Main", "AvgTicketPrice" : 841.2656419677076, "DistanceMiles" : 10247.856675613455, "FlightDelay" : false, "DestWeather" : "Rain", "Dest" : "Sydney Kingsford Smith International Airport", "FlightDelayType" : "No Delay", "OriginCountry" : "DE", "dayOfWeek" : 0, "DistanceKilometers" : 16492.32665375846, "timestamp" : 0 "2019-11-18t00:00:00 ", "DestLocation" : {"lat" : "-33.94609833", "lon" : "151.177002"}, "DestAirportID" : SYD, Carrier: Kibana Airlines, Cancelled: false, FlightTimeMin: 1030.7704158599038, Origin: "OriginLocation" : {" LAT ": "50.033333"," LON ": "8.570556"}, "DestRegion" : "SE-BD", "OriginAirportID" : "FRA", "OriginRegion" : "DE-HE", "DestCityName" : "Sydney", "FlightTimeHour" : 17.179506930998397, "FlightDelayMin" : 0}Copy the code

As I mentioned above, we first select our index and then set the time for our time picker.

KQL to search:

As you can see above, one of the big benefits when using KQL is that it automatically prompts you for the fields you want to search for, with auto-completion. For example, when we type in Day, Kibana automatically pops up options for us to select.

We can even type the desired string directly, as Baidu does, without having to specify a field:

We can also do fuzzy lookups with wildcards:

The quotation marks around the search term initiate a phrase (phrase) search. For example, messages: “Quick Brown Fox” will search for the phrase “Quick Brown Fox” in the message field. Without quotes, your query will decompose the parser configured through the message field into tokens, and will match the documents that contain these tokens regardless of the order in which they appear. This means that documents with “Quick Brown Fox” will match, but “Quick Fox Brown” will also match. If you search for phrases, remember to use quotation marks. When searching phrase, the order of each token is very important.

The query parser will no longer split in the margins. Multiple search terms must be separated by an explicit Boolean operator. Lucene combines search terms with or by default. These Boolean operators are OR, and, and not.

The above search will return all documents whose dayofWeek is 1 or whose OriginCountry is “DE”. If we want to search for documents that satisfy both criteria, we can use and

Obviously, the number of documents we see at this point is 23, which is a lot less than before. We can also use not to return non-operations. For example, if we want all documents whose OriginCountry is not DE, we can simply search not OriginCountry: “DE”.

We can also range fields, such as:

Lucene way to search:

To be able to search in Lucene mode, we must switch to Lucence mode. In this way, when we enter fields in the input field, there is no prompt to help us automatically complete the input.

We can search a range of documents:

Above, if we don’t want TO include 3, we should write: dayOfWeek:[0 TO 3}. You can also write any value from 3 up:

We can also search for all documents whose OriginCountry is US or DE as follows.

We can also weight a field, for example we can highlight countries whose OriginCountry is DE. In the following search, we give documents whose OriginCountry is DE a triple score so that they are as high up in the rankings as possible.

Or fuzzy query:

Or a fuzzy query with only an Edit (note the concept of Edit here, see the fuzzy query article) :

You can also use wildcards, right? To match any letter (note that this is not available in KQL) :

We can also use regular expressions to search. If we do not know the OriginCountry is US or something, and we know the following letter is S, then we can use the following method to query:

We can also use.? Re to query for 0 or 1 letters:

You can also match searches of zero or more letters with the.* re: