Install the Logstash

Download and unzip the Logstash

I downloaded the MACOS version from the binary installation package

Official Elastic website: www.elastic.co/cn/

Logstash introduction page: www.elastic.co/cn/logstash

Logstash download page: www.elastic.co/cn/download…

Prepare the logstash. Conf configuration file

Run Logstash

bin/logstash -f logstash.conf
Copy the code

Download the Movielens test dataset

Download the MovieLens minimum test data sets: grouplens.org/datasets/mo…

Write the logstash. Conf configuration file

Create the logstash. Conf configuration file and write (note the file path)

input {
  file {
    path = > "/Users/mankvis/Work/elasticsearchStudy/ml-latest-small/movies.csv"
    start_position = > "beginning"
    sincedb_path = > "/dev/null"}}filter {
  csv {
    separator = > ","
    columns = > ["id"."content"."genre"]}mutate {
    split = > { "genre" = > "|" }
    remove_field = > ["path"."host"."@timestamp"."message"]}mutate {

    split = > ["content"."("]
    add_field = > { "title" = > "%{[content][0]}"}
    add_field = > { "year" = > "%{[content][1]}"}}mutate {
    convert = > {
      "year" = > "integer"
    }
    strip = > ["title"]
    remove_field = > ["path"."host"."@timestamp"."message"."content"]}}output {
   elasticsearch {
     hosts = > "http://localhost:9200"
     index = > "movies"
     document_id = > "%{id}"
   }
  stdout{}}Copy the code

Try importing data through Logstash

Run the command in the logstash/bin directory

sudo ./logstash -f logstash.conf
Copy the code

The following is the output information

View the imported data

Open Kibana Dev Tools and type GET /movies/_search

You can see that all data from the downloaded test dataset has been dumped into Elasticsearch