We are familiar with configuration files, which provide us with the ability to dynamically modify program execution. To quote someone else:

Dynamic adjustment of flight attitude during system runtime

I might call our job fixing parts on a fast-moving plane. We humans are always out of control and out of control. For our system, we always need to reserve some control lines, so that we can make adjustments when we need to control the direction of the system (such as gray control and current limiting adjustment), which is particularly important for the Internet industry to embrace changes. For the standalone version, we call this configuration (file), for the distributed cluster system, we call this configuration center (system); Let’s talk about our configuration center.

His shan zhishi

  • Configuration center Lion: yeming. Me /2017/11/27/…
  • Apollo Configuration Center: github.com/ctripcorp/a…
  • QConf build configuration center: segmentfault.com/a/119000000…
  • The basis of dynamic adjustment, configuration center: pingguohe.net/2016/03/18/…
  • An article about the configuration of length for TM center: jm.taobao.org/2016/09/28/…

Configuration in Evolution

When we are a stand-alone service, our configuration is usually written in a file, and when the code is released, the configuration file and application are pushed to the machine.

As the business grows, we typically deploy our services on multiple machines (clusters). At this point, the configuration is published as follows:

Ok. In this way, the deployment configuration can also accept the rapid expansion of services, resulting in the failure of the single-node service to meet the service requirements. At this time, the single large service needs to be cut open, the service to SOA(microservitization).

It’s a nightmare to deploy configurations like this, and you can’t adjust them quickly and dynamically. One of the main points of configuration is lost. This is where the unified configuration center is needed.

Simplified version of configuration center

Configuration center features

  • Add, delete, change and check the configuration
  • Isolation of different environment configurations (development, test, pre-release, grayscale/online)
  • High performance and availability
  • Large number of requests and high concurrency
  • Read more to write less

We can design the following simple configuration center

Design notes:

  • Add, delete, modify, and check each configuration (each configuration has a unique configuration ID) in the OA system.
  • Distinguish the configurations of different environments. Each environment corresponds to a different database record with the same configuration ID.
  • The configuration is eventually stored in the mysql database in JSON format, which is easy to edit and understand.
  • Introduce redis cluster, do the cache of configuration (for example, you can set the configuration to take effect 1 minute after modification)
  • Configure external services and deploy multiple machines to meet performance requirements.
  • If necessary, you can import configuration history changes.

Most of the time, this can basically meet the basic needs of our configuration system, to add, delete, change and check the configuration, can tolerate a period of data inconsistency.

This design has its own performance bottlenecks because all configuration is stored in a centralized cache. Also, each configured access requires an RPC request (network request), so consider introducing a localCache (localCache, such as Ehcache) on the client side.

Local cache can be referred to my previous article: [180409] Local cache selection and principle

Performance availability improvements for configuration centers

Considering the factors of reducing network requests, localCache is introduced in the client side to solve the high availability, high performance and scalability of the system.

One improvement over the first version is the introduction of localCache on the client side. Enable thread asynchronous invocation of configuration service to update local configuration. This reduces RPC calls.

Based on the DATA CAP principle, this method only achieves AP. There will be data inconsistency for a period of time, but the final consistency of configuration will be guaranteed. How to solve this data inconsistency problem?

Data consistency improvement in configuration center

Fortunately, there is usually only one entry change to the configuration, so consider notifying the application service to clean up local and distributed caches after a configuration change. This is where MQ or ZooKeeper can be introduced.

Finally, the data consistency is achieved through the combination of push and pull.

personal

Welcome to my technology blog, we grow together and progress together.

My neighbor totoro Lin Wan village: www.jianshu.com/u/5a327aab7…