This is the 8th day of my participation in the August More Text Challenge. For details, see:August is more challenging

Problem description:

Enterprise portal APP, with mobile organizational structure, employees can query company user information on the APP, support employees to search in the address book by multiple fields (name/employee ID/mobile phone number/email /……) , fuzzy search. We are using ElasticSearch to store employee information to facilitate this search.

Normal employee information into es, search will not have a problem. However, our products are enterprise-level products, which are specialized in serving different enterprise customers and solving various needs of customers. For example, the fields of normal employees include name, employee id, mobile phone number, email, gender, etc., which are also standard fields in our products, and cannot meet the personalized needs of customers. For example, customer needs:

  • Customer A: We have the rank field. What should we do
  • Customer B: We have a nickname field. What should we do
  • C: We have a hobby field. What should we do
  • .

As mentioned above, different enterprises have different fields, in order to satisfy customers can customize their own fields, we designed the function of personalized field.

Personalization field: Enterprises can customize this tenant employee field for their own tenants, increasing the flexibility of the code (this design will be written later)

In the personalization field, you can control whether certain fields can be searched in ES. To support search in ES, you need to initialize the field in the index of ES. When you add a field, it will be added to the corresponding index. To prevent accidents, we provide an interface for web-based operations where administrators can manually reset indexes and data for the entire tenant. During rebuild reinitialization, if the user happens to be using the search function, the index may not exist or the employee may not be able to search the data.

This article is to explain how to ensure that data can be normally accessed during the initialization of ES, scheme: space for time, the introduction of double index to process

In the book redis Design and Implementation, it is explained that when Redis performs rehash on the dictionary hash table, there are two hash table exchange processing, respectively ht[0] and HT [1].

Double index design

There are two default indexes index0 and Index1 in the system. If the index providing services is Index0, then index1 will be initialized in the next initialization and reconstruction. After initialization, Index1 will provide external services and index Index0 will be deleted. And so on.

1. Noun explanation

The following is a separate description of index design for tenant 0. In multi-tenant, separate redis key and ES indexes

  • There are two indexes in elasticsearch: index0 (employee index0, default) and index1(employee index1)
  • Ajisun: elastic: employee: init0 (single key expiration time, whether the tenants are initialized,)
  • Ajisun: elastic: employee, 0 (hash key, stored 0 tenants are using the index of the)
2. Index reconstruction steps:
  1. Through the redis keyajisun:elastic:employee:init0Check whether the current tenant is being initialized. If yes, ignore this operation. If no, see Step 2.
  2. Using the Redis hash keyAjisun: elastic: employee, {tenantId}Example Obtain the es index =oldIndex used by the current tenant
  3. HshPut (“ajisun:elastic:employee”, 0, “index0”), then the newIndex newIndex that will be initialized for reconstruction is “index1”
  4. If oldIndex is not empty, then the index that is currently serving is oldIndex, and the newIndex that you are going to initialize for reconstruction, newIndex, is non-oldindex (if oldIndex is)index0That’s not the oldIndexindex1Or vice versa)
  5. And then set the tenant index initializing key redis. StrSet (” ajisun: elastic: employee: init0 newIndex, 5 l, TimeUnit. MINUTES); The expiration time is added to prevent the application from crashing, resulting in the state being initialized
  6. Initialize the newIndex newIndex and add data.
  7. Delete the initialized state redis key, redis delKey (” ajisun: elastic: employee: init0 “);
  8. Change the default vstore index in Redis to newIndex, redis. HshPut (“ajisun:elastic: Employee “,0, newIndex);
  9. Delete oldIndex from ElasticSearch.

The flowchart is as follows:

3. Data search

HshGet (“ajisun: Elastic: Employee “,0) finds that tenant 0 is currently using the index. If it is empty, use the default index index0.

4. Data modification/storage

If the index is initialized, the data is modified/stored directly on the new index

  1. Redishelper. hshGet(“ajisun: Elastic: Employee “,0) obtains the index that tenant 0 is using.
  2. If empty, the default index is index0
  3. If the index is not empty, the access to redis cache redisHelper. StrGet (” ajisun: elastic: employee: init0 “) 0 if the tenant is empty
  4. If it is not empty, it indicates that the data is being initialized, and the index that holds the data is the index that is being initialized

The above is for the es index initialization during the normal use of the impact of the solution, in fact, no difficulty, just to provide a way of thinking, there are questions welcome to propose communication.