How does the micro-service registry carry tens of millions of accesses for large-scale systems?

Directory:

First, the origin of the problem

Second, Eureka Server has a sophisticated registry storage structure

3. Excellent multi-level caching mechanism on Eureka Server

Four,

First, the origin of the problem

Eureka is a crucial component in the Spring Cloud microservice architecture. It acts as the microservice registry, and all service registration and service discovery depend on Eureka.

Many friends who are new to Spring Cloud often ask the following questions when deploying in the production environment of their company:

• How many machines will Eureka Server be deployed on?

• With so many services in our system, how much access pressure will it bring to Eureka Server?

• Can Eureka Server withstand the access pressure of a large system?

If you have any of these questions, don’t worry! Let’s take a look at the core rationale behind Eureka as a microservices registry

These questions below, we have a look first, have a general impression. With these questions in mind, look at the following content, the effect is better!

1. What method does the Eureka registry use to store the machine addresses and port numbers sent for each service registration?

2. What is the frequency with which each service pulls the Eureka Server registry?

3. How do services pull the registry?

4. How much access pressure would a large distributed system with hundreds of services deployed on thousands of machines put on Eureka Server?

5. How does Eureka Server resist daily tens of millions of page views from the technical level?

The Eureka Client component in each service, by default, sends a request to the Eureka Server every 30 seconds to retrieve the service information that has recently changed

Here’s an example:

• The inventory service was originally deployed on one machine, but now it has been expanded to three machines, all registered with Eureka Server.

• The Eureka Client of the order service then goes to the Eureka Server every 30 seconds to pull the most recent registry changes to see if the addresses of other services have changed.

In addition, Eureka also has a heartbeat mechanism. Each Eureka Client sends a heartbeat to the Eureka Server every 30 seconds to say, “Dude, my service instance is still alive!”

If a Eureka Client does not send a heartbeat to the Eureka Server for a long time, the service instance has been suspended.

Just by looking at the text, you may not be impressed. As usual! Let’s take a picture, just to get a sense of what’s going on.

Second, Eureka Server has a sophisticated registry storage structure

Now let’s assume we have a large distributed system with 100 services, each deployed on 20 machines with four cores and eight gigabytes as standard.

In other words, you have deployed 100 * 20 = 2000 service instances with 2000 machines.

There is a Eureka Client component inside the service instance on each machine, which requests the Eureka Server every 30 seconds to pull the changing registry.

In addition, the Eureka Client on each service instance sends heartbeat requests to the Eureka Server every 30 seconds.

So how many times does Eureka Server get requested per second as a microservice registry? How many times a day?

• According to the standard algorithm, each service instance requests to pull the registry twice per minute and sends heartbeat twice per minute

• Such a service instance would make four requests per minute, and 2,000 service instances would make 8,000 requests per minute

• When translated to the second, 8000/60 = 133 times, we estimate that Eureka Server will be requested 150 times per second

• On that day, 8,000 * 60 * 24 = 11.52 million, or 10 million visits per day

Good! After such a calculation, have you discovered the mystery here?

• First of all, for a component such as a microservice registry, the pull frequency and heartbeat sending frequency were designed to take into account the pressure of each service request on a large system, how many requests per second it would carry.

• So every 30 seconds each service instance makes a request to pull the changed registry, and every 30 seconds the heartbeat is sent to Eureka Server. This timing is intentional.

According to our calculation, a system with hundreds of services and thousands of machines requests Eureka Server at such a frequency, with daily requests of tens of millions and visits of about 150 times per second.

Let’s assume that Eureka Server is requested 200 to 300 times per second, even with some additional operations.

Therefore, by setting an appropriate pull registry and sending heartbeat frequency, you can ensure that the request pressure on Eureka Server is not too heavy in large-scale systems.

Now the key question is **, how can Eureka Server easily handle hundreds of requests per second and tens of millions of requests per day? **

What does Eureka Server use to store the registry? Three words, look at the source!

Let’s take a look at the Eureka source code:

• As shown in the figure above, the CocurrentHashMap named Registry is the core structure of the registry. After watching it, I couldn’t help admiring the exquisite design!

• As you can see from the code, Eureka Server’s registry is directly based on pure memory, that is, a data structure is maintained in memory.

• The registry is maintained and updated in memory for each service’s registration, service offline, and service failure.

• Eureka Server simply provides the changed registry data stored in memory to each service when it pulls the registry every 30 seconds.

Also, every 30 seconds that a heartbeat is initiated, the heartbeat time is updated in this pure memory Map data structure.

In a word: maintain the registry, pull the registry, update the heartbeat time, all happen in memory! This is a very core point of Eureka Server.

Once this is clear, let’s analyze the data structure of Registry again. Don’t be fooled by its complex appearance.

• First, the key of the ConcurrentHashMap is the service name, such as inventory-service, which is a service name.

• Value represents multiple service instances of a service.

• For example, the inventory-service can have three service instances, each deployed on one machine.

Map
,>

• The Map key is the id of the service instance

• Value is a class called Lease, and its generic type is something called InstanceInfo. What the hell are those, you might ask?

• Let’s start with InstanceInfo, which, by definition, represents the details of the service instance, such as the machine’s IP address, hostname, and port number.

• The Lease maintains the last heartbeat sent by each service

3. Excellent multi-level caching mechanism on Eureka Server

Assuming Eureka Server is deployed on a 4-core, 8-GIGAByte machine, what is the maximum number of requests per second that can be processed based on memory for each service?

• According to previous tests, a single 4-core, 8-GIGAByte machine that handles pure memory operations can handle a few hundred requests per second with ease and pleasure, even with some network overhead.

• In order to avoid the concurrency conflict caused by reading and writing data structure simultaneously, Eureka Server also adopts multi-level caching mechanism to further improve the response speed of service requests.

• When pulling the registry:

• First look up the cached registry from ReadOnlyCacheMap.

• If not, look for the registry cached in ReadWriteCacheMap.

• If not, get the actual registry data from memory.

• When the registry changes:

• Changes to registry data are updated in memory and ReadWriteCacheMap expires.

• This process does not affect the ReadOnlyCacheMap provider’s ability to query the registry.

• For a period of time (30 seconds by default), ReadOnlyCacheMap is read directly by each service pull registry

• After 30 seconds, the background thread of the Eureka Server finds that ReadWriteCacheMap has been cleared and will also clear the cache in ReadWriteCacheMap

• The next time a service pulls the registry, it will fetch the latest data from memory and populate the caches.

What are the advantages of multi-level caching?

• As far as possible, the memory registry data will not appear frequent read and write conflicts.

• And further ensure that a large number of requests to Eureka Server are quickly removed from pure memory, high performance.

In order to facilitate a better understanding, here is also a picture. Let’s review the whole process again with the picture:

Four,

• As can be seen from the above analysis, Eureka can ensure that a large-scale system can request The Eureka Server several hundred times per second by setting the appropriate request frequency ** (pull the registry at 30 seconds interval, send the heartbeat at 30 seconds interval) **.

• At the same time through the pure memory registry, to ensure that all requests can be processed in memory, to ensure high performance

• In addition, multi-level caching ensures that frequent concurrent read and write conflicts do not occur on memory data structures, further improving performance.

This is how Eureka, as a microservice registry in The Spring Cloud architecture, can carry tens of millions of daily visits to large-scale systems.

If there is any harvest, please help to forward, your encouragement is the biggest power of the author, thank you!

A large wave of micro services, distributed, high concurrency, high availability of original series of articles are on the way, welcome to scan the qr code below, continue to follow:

“Spring Cloud Parameter Optimization combat under tens of thousands of concurrent requests per second behind Double 11”, please look forward to it

How can micro-service architecture ensure 99.99% high availability under singles’ Day carnival

** Welcome to follow the wechat official account: Architecture Notes of Stone Spruce **

** ** 8:30am Monday to Friday! High-quality technical articles sent on time

More than ten years of _BAT architecture experience _ taught each other.