Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”.

In the use of ES, full-text search is a common scenario, especially when ES is used as a log storage retrieval, how to query the corresponding log information according to the keyword?

1. Combine dynamic templates with copy_to

When creating the index, we add an allColumnValue column, copy all the other column values, and then search for this column

Dynamic_templtes implements the autocopy logic above, so we can create an index as follows

PUT search_all_demo 
{
  "mappings": {
    "dynamic_templates": [{"copy_to_allcolumnvalue" : {
            "match_mapping_type" : "*"."mapping" : {
              "copy_to" : "allColumnValue"."ignore_above" : 512."type" : "keyword"}}}]."properties": {
      "allColumnValue" : {
          "type" : "text"}}}}Copy the code

When creating the mapping table above, two points

  • AllColumnValue: field
  • Dynamic_templates: Implements field copying

Next, write a piece of data to test


POST search_all_demo/_doc
{
  "name": "A lump of ash"."site": "www.hhui.top"."title": "java developer"
}
Copy the code

It then retrieves to see if the desired result can be found

GET search_all_demo/_search
{
  "query": {
    "match": {
      "allColumnValue": "Gray"}}}Copy the code

The above query would normally hit our data and return

{
  "took" : 1."timed_out" : false."_shards" : {
    "total" : 1."successful" : 1."skipped" : 0."failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1."relation" : "eq"
    },
    "max_score" : 0.7911257."hits": [{"_index" : "search_all_demo"."_type" : "_doc"."_id" : "1FoBk3wB-kdeh8MF_IbL"."_score" : 0.7911257."_source" : {
          "name" : "A lump of ash"."site" : "www.hhui.top"."title" : "java developer"}}]}}Copy the code

Pay attention to

When using the above configuration, there are requirements for Field, which will fail when we make a Map type

POST search_all_demo/_doc
{
  "name": "A grey"."site": "blog.hhui.top"."ddd": {
    "user": "yihui"."pwd": "yihui"}}Copy the code

The above DDD prompts an exception

{
  "error" : {
    "root_cause": [{"type" : "mapper_parsing_exception"."reason" : "failed to parse field [ddd] of type [keyword] in document with id '11qek3wB-kdeh8MFm4bN'. Preview of field's value: '{pwd=yihui, user=yihui}'"}]."type" : "mapper_parsing_exception"."reason" : "failed to parse field [ddd] of type [keyword] in document with id '11qek3wB-kdeh8MFm4bN'. Preview of field's value: '{pwd=yihui, user=yihui}'"."caused_by" : {
      "type" : "illegal_state_exception"."reason" : "Can't get text on a START_OBJECT at 4:10"}},"status" : 400
}
Copy the code

2. Partial field combination search

AllColumnValue provides full text retrieval of allColumnValue data. Maybe in a real scenario, I just want to carry out joint retrieval for part of the field, based on which I can set up as follows

PUT search_union_demo 
{
  "mappings": {
    "properties": {
      "allColumnValue" : {
          "type" : "text"
        },
        "name": {
          "type" : "keyword"."ignore_above" : 512."copy_to" : [
            "allColumnValue"]},"site" : {
          "type" : "keyword"."ignore_above" : 512."copy_to" : [
            "allColumnValue"]}}}}Copy the code

Add two new data

POST search_union_demo/_doc
{
  "name": "test"."site": "spring.hhui.top"."ddd": {
    "user": "A grey"."pwd": "yihui"
  }
}

POST search_union_demo/_doc
{
  "name": "A grey"."site": "blog.hhui.top"."ddd": {
    "user": "yihui"."pwd": "yihui"}}Copy the code

Then when we retrieve a gray, we can find the second data

GET search_union_demo/_search
{
  "query": {
    "match": {
      "allColumnValue": "A grey"}}}Copy the code

The output

{
  "took" : 2."timed_out" : false."_shards" : {
    "total" : 1."successful" : 1."skipped" : 0."failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1."relation" : "eq"
    },
    "max_score" : 1.2814486."hits": [{"_index" : "search_union_demo"."_type" : "_doc"."_id" : "2Fqjk3wB-kdeh8MFy4aC"."_score" : 1.2814486."_source" : {
          "name" : "A grey"."site" : "blog.hhui.top"."ddd" : {
            "user" : "yihui"."pwd" : "yihui"}}}]}}Copy the code

3. Summary

This paper mainly introduces the use of COPY_to, to achieve es union/full text search function; Simple Settings to support more user-friendly query scenarios

II. The other

1. A gray Blog:liuyueyi.github.io/hexblog

A gray personal blog, recording all the study and work in the blog, welcome everyone to go to stroll

2. Statement

As far as the letter is not as good, the above content is purely one’s opinion, due to the limited personal ability, it is inevitable that there are omissions and mistakes, if you find bugs or have better suggestions, welcome criticism and correction, don’t hesitate to appreciate

  • Micro Blog address: Small Gray Blog
  • QQ: a gray /3302797840
  • Wechat official account: One Grey Blog