Background:

The search interface has slow performance and large data volume

Business Background: There are 1000 companies. The amount of data of different companies varies greatly, and the more recent the data is, the more likely it is to be used.

Plan 1: build index by company, one index for each company is a bit: technical implementation is simple.

Disadvantages: 1. The index reconstruction efficiency is slow when a fault occurs. 2. Poor scalability

Solution 2: Index hotspot data separately and index historical data by time

Time -based Indices (TIME-based Indices

1. How to choose the time range

According to the growth rate of different data can be divided by day (index name similar to 2019-09-24), by month (index name similar to 2019-09), by year (index name similar to 2019)

2. Design an index template

For our business, every index is the same except for the index name, so we can design a template for each index creation. Here we need the Index Templates mechanism for ES.

The basic principle of Index Templates is to first pre-define one or more “Index Templates” (including Settings and mappings). Then, when creating an index, once the index name matches an “index template”, ES will automatically apply the Settings and mappings contained in the “index template” to the newly created index.

For a business example, suppose our ES index requirements are as follows:

Index by index (index name like business-2019-09)

The data of each day is stored in the index of the corresponding month

When searching, you want to search for all indexes (with alias)

Based on the preceding index requirements, the corresponding index template can be designed as follows:

$ curl -XPUT http://localhost:9200/_template/business_template -d'{ "template": "business-*", "settings": { "number_of_shards": 1 }, "mappings": { "log": { "dynamic": false, "properties": { "content": { "type": "string" }, "created_at": { "type": "date", "format": "dateOptionalTime" } } } }, "aliases": { "search-business": {} } }'
Copy the code
Description:

  1. When creating an index, if the index name is in the format of “Business -*”, ES will automatically apply the Settings and mappings above to the index

  2. Aliases’ configuration tells ES to automatically add an alias named “search-business” to the index every time it is created.

Indexing and searching

Based on the above “index template” design, the indexing and search strategy is straightforward.

Index policy: The data of a day is indexed only to the corresponding index of the current month. For example, data from September 24, 2019 will only be indexed in business-2019-09.

Search strategy: Since the search requirement is to search in full, the index name should use the “search-business” alias.

Managing Elasticsearch Time -based Indices is an efficient and efficient way to manage a time-based index