In today’s article I will show you how to search Elasticsearch using the Go language. First, I’m going to assume that you already know something about Elastic Stack. Elasticsearch has been hugely popular. Searching in relational databases always brings up scalability and performance issues. Elasticsearch is a NoSQL database and is very successful at addressing these issues. It provides excellent scalability and performance, and one of the most outstanding features is correlation analysis, which gives search results a great deal of flexibility.
Elastic Stack
Install Elasticsearch and Kibana
If you haven’t already installed your own Elasticsearch and Kibana, please check out my previous post:
- How do I install Elasticsearch on Linux, MacOS and Windows
- How to install Kibana in the Elastic stack on Linux, MacOS and Windows
Once you have Elasticsearch and Kibana installed, you can access them by typing http://localhost:9200 into your browser:
We can also go directly to Kibana:
Create your first index
In Elasticsearch, indexes are similar to databases. Previously, ElasticSearch had a table called Type. But because the type has been removed in the current version, there are only indexes now.
Confused now? No, in short, just think about indexing and then inserting data into Elasticsearch.
Now we will create an index named STUDENTS with the following query.
PUT students
Copy the codeThe response to the above command is:
{
"acknowledged" : true,
"shards_acknowledged" : true,
"index" : "students"
}
Copy the codeIt has created an index for us called Students. In the next step, we will write some data, i.e. some documents, to the index.
Write documents to the index
First, all we need to do now is populate our Elasticsearch index with documents. If you are not familiar with this definition, it is very similar to a row in a database. To import data into Elasticsearch, use the following command:
POST students/_doc
{
"name": "Alice",
"age": 17,
"average_score": 85
}
Copy the codeThe response to the above command is:
{
"_index" : "students",
"_type" : "_doc",
"_id" : "cMpCSnUBzapZdaLVYfwW",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1
}
Copy the codeYour Elasticsearch should now have a document. We will need to insert more data into the index students. Of course, we don’t insert our student data one by one – that’s a hassle!
Elasticsearch specifically prepares a Bulk API to send multiple requests at once. Let’s use it to insert multiple data at once.
POST students/_bulk { "index":{ } } { "name":"john doe","age":18, "Average_score" : 77.7} {" index ": {}} {" name" : "Bob", "age" : 16, "Average_score" : 65.5} {" index ": {}} {" name" : "Mary doe", "age" : 18. "Average_score" : 97.7} {" index ": {}} {" name" : "eve", "age" : 15, "average_score" : 98.9}Copy the codeWe can check the five documents we have entered by using the following command:
GET students/_count
Copy the codeSearch index documents
We finally filled Elasticsearch with more student data. Now let’s do what Elasticsearch is known for: we will try to search for the data just inserted in Elasticsearch.
Elasticsearch supports many types of search mechanisms, but in this example, we’ll use a simple matching query.
Let’s start our search by clicking on the following API:
GET students/_search
{
"query": {
"match": {
"name": "doe"
}
}
}
Copy the codeThe command above shows:
{ "took" : 0, "timed_out" : false, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 }, "hits" : {" total ": {" value" : 2, the "base" : "eq"}, "max_score" : 0.74487394, "hits" : [{" _index ":" students ", "_type" : "_doc", "_id" : "hstFSnUBzapZdaLVQQCN", "_score" : 0.74487394, "_source" : {"name" : 18, "average_score" : 77.7}}, {" _index ":" students ", "_type" : "_doc", "_id" : "iMtFSnUBzapZdaLVQQCN", "_score" : 0.74487394, "_source" : {"name" : "Mary doe", "age" : 18, "average_score" : 97.7}}]}}Copy the codeIt will return your query and the student data that matches your corresponding query. Now, congratulations you have officially become a search engineer!
Let’s use Go
If you’ve reached this point, you should already have a minimal understanding of how to use Elasticsearch. Now we will implement Elasticsearch in Go. A very primitive way to implement Elasticsearch is to continue sending HTTP requests to your Elasticsearch IP. But we don’t do that.
I find this very useful library for implementing Elasticsearch in Go. You should install the library before continuing with the Go module. We can use the following command to install:
go get github.com/olivere/elastic
Copy the codeCreate your own struct
First, you definitely need to make a structure for the model. In this example, we will use the same modeling as in the previous example, in this case the Student structure.
package main
type Student struct {
Name string `json:"name"`
Age int64 `json:"age"`
AverageScore float64 `json:"average_score"`
}
Copy the codeCreate a client connection
Now, let’s create a function that will allow us to initialize the ES Client connection. If you have a running instance of Elasticsearch outside your localhost, just change the part in SetURL.
func GetESClient() (*elastic.Client, error) { client, err := elastic.NewClient(elastic.SetURL("http://localhost:9200"), elastic.SetSniff(false), elastic.SetHealthcheck(false)) fmt.Println("ES initialized..." ) return client, err }Copy the codeInsert data into
After that, the first thing we’ll do is try to insert the data into Elasticsearch via Go. We will make a Student model and insert it into our Elasticsearch client.
insert.go
package main import ( "context" "encoding/json" "fmt" "github.com/olivere/elastic" ) type Student struct { Name string `json:"name"` Age int64 `json:"age"` AverageScore float64 `json:"average_score"` } func GetESClient() (*elastic.Client, error) { client, err := elastic.NewClient(elastic.SetURL("http://localhost:9200"), elastic.SetSniff(false), elastic.SetHealthcheck(false)) fmt.Println("ES initialized..." ) return client, err } func main() { ctx := context.Background() esclient, err := GetESClient() if err ! = nil { fmt.Println("Error initializing : ", err) panic("Client fail ") } //creating student object newStudent := Student{ Name: "Gopher doe", Age: 10, AverageScore: 99.9,} dataJSON, err := json.Marshal(newStudent) js := string(dataJSON) _, err = esclient.Index(). Index("students"). BodyJson(js). Do(ctx) if err ! = nil { panic(err) } fmt.Println("[Elastic][InsertProduct]Insertion Successful") }Copy the codeWe can run it with the following command:
go run insert.go
Copy the codeThe above application shows:
$ go run insert.go
ES initialized...
[Elastic][InsertProduct]Insertion Successful
Copy the codeWe can verify that the new document has been successfully written using Kibana:
GET students/_search
{
"query": {
"match": {
"name": "Gopher doe"
}
}
}
Copy the code
It shows that our document has been successfully written to Elasticsearch.
Query data
Finally, we can do some searching. The following code may seem complicated. But rest assured, it will make more sense to you after you’ve done it carefully. In the example below, I’ll use a basic matching query.
search.go
package main import ( "context" "encoding/json" "fmt" "github.com/olivere/elastic" ) type Student struct { Name string `json:"name"` Age int64 `json:"age"` AverageScore float64 `json:"average_score"` } func GetESClient() (*elastic.Client, error) { client, err := elastic.NewClient(elastic.SetURL("http://localhost:9200"), elastic.SetSniff(false), elastic.SetHealthcheck(false)) fmt.Println("ES initialized..." ) return client, err } func main() { ctx := context.Background() esclient, err := GetESClient() if err ! = nil { fmt.Println("Error initializing : ", err) panic("Client fail ") } var students []Student searchSource := elastic.NewSearchSource() searchSource.Query(elastic.NewMatchQuery("name", "Doe")) /* this block will basically print out the es query */ queryStr, err1 := searchSource.Source() queryJs, err2 := json.Marshal(queryStr) if err1 ! = nil || err2 ! = nil { fmt.Println("[esclient][GetResponse]err during query marshal=", err1, err2) } fmt.Println("[esclient]Final ESQuery=\n", string(queryJs)) /* until this block */ searchService := esclient.Search().Index("students").SearchSource(searchSource) searchResult, err := searchService.Do(ctx) if err ! = nil { fmt.Println("[ProductsES][GetPIds]Error=", err) return } for _, hit := range searchResult.Hits.Hits { var student Student err := json.Unmarshal(hit.Source, &student) if err ! = nil { fmt.Println("[Getting Students][Unmarshal] Err=", err) } students = append(students, student) } if err ! = nil { fmt.Println("Fetching student fail: ", err) } else { for _, s := range students { fmt.Printf("Student found Name: %s, Age: %d, Score: %f \n", s.Name, s.Age, s.AverageScore) } } }Copy the codeWe run it using the following command:
go run search.go
Copy the codeThe query should be printed like this:
ES initialized...
[esclient]Final ESQuery=
{"query":{"match":{"name":{"query":"Doe"}}}}
Copy the codeYes, this query is what will be published in Elasticsearch. If I had followed my example from the start, the query would have looked like this:
Student found Name: John Doe, Age: 18, Score: 77.700000 Student found Name: Mary Doe, Age: 18, Score: 77.700000 97.700000 Student found Name: Gopher Doe, Age: 10, Score: 99.900000Copy the codeHooray! We have finally finished the first process of importing data into Elasticsearch using Golang and searching it. The process is simple!
If you want to learn more about Elasticsearch and other more complex DSL queries, please read my other article “Elastic: A Beginner’s Guide”.