“This is the 13th day of my participation in the Gwen Challenge in November. Check out the details: The Last Gwen Challenge in 2021”

With the increasing scale of microservices, there may be complicated invocation relationships among microservices

In our actual work, there are indeed a lot of problems slowly, and the drawbacks of the whole system are slowly displayed

For example, this is the case:

  • Service A requests service B, and service B also requests service C. As A result, the whole link is faulty, or even the entire system is broken

In our work, in order to improve the robustness of the service, we usually set the retry mechanism after failure to avoid some temporary failures due to network jitter

However, if it is a chronic failure, the retry mechanism will only burden our service, constantly consuming connections and performance

At this point, service circuit breakers are needed

Service circuit breaker mechanism

What is the circuit breaker for services?

Actually fusing, is we used to study physics knowledge when heard the word, such as the circuit in the home, in the position of the main switch, there will be a fuse to protect the safety of our circuit, if the short circuit, or is under the condition of abnormal current is too large, the fuse will fuse because of overheating, and power outages, eventually reach meters and circuit protection role

In today’s microservices architecture, this fuse is also needed

The figure above is a very common invocation relationship between microservices

Request: Client – Gateway – Service A – Service B

Response: Service B – Service A – Gateway – Client

As long as one point in the whole link is faulty, the client cannot obtain the expected result

In microservices architecture, calls between services are generally divided into

  • Service caller
  • Service Provider

Why is a circuit breaker needed?

When the downstream service is unable to provide services due to overload or failure, we need to inform the upstream service in time and temporarily break the invocation chain between the caller and provider, in order to avoid the occurrence of service avalanche

Service avalanche

A service avalanche is when a link in the invocation chain becomes unavailable, specifically the provider of the service, causing the upstream service to become unavailable, and eventually the effect spreads like an avalanche through the entire system, resulting in the entire system becoming unavailable

Simple Fault Scenario

Once again A familiar scenario, the normal client-gateway-service A-service B request process is shown in the above three phases, which are also the three phases of the service avalanche

  • The service provider is unavailable

Where the system works, everything works well. Each service requests and responds normally. At a certain moment, service B is unavailable due to its own exception or network fault, and cannot timely respond to incoming requests

  • The service caller is not available

When service B is unavailable as A service provider, the client may repeatedly send the same request to the gateway due to an error message or A long period of blocking, and the request will be sent to the gateway again, to service A, and finally to service B until the timeout does not respond properly

Because service A initiates too many requests to service B, the waiting thread is generated and the resources in the thread pool are exhausted. As A result, service A cannot respond to external requests in A timely manner and service A is also unavailable

  • The entire system is unavailable

After the above process, service A also blocks the gateway for forwarding requests. Because the gateway waits for A large number of request responses, it will also generate A large number of blocked threads. Similarly, the gateway finally does not have enough resources to handle other requests, so the whole system cannot provide services externally

Plus services to ensure system availability

As shown in the figure above, A fuse, or circuit breaker, is inserted in the middle of service A’s access to service B.

  • When service A accesses service B, service B has A minor failure, resulting in A timeout return
  • When service A continues to access service B, service B is unavailable and fails
  • If the circuit breaker detects an anomaly, turn on the fuse and set abnormal return
  • When service A accesses service B again, the fuse itself immediately returns an error message to service A, preventing service A from running out of resources and becoming unusable, thus protecting the service caller

The circuit breaker

As shown in the figure above, the circuit breaker switches between three states, which can be interpreted as follows:

1 Off – On

  • If the number of failed functions exceeds the threshold within a period, the closed state changes to the open state

2 Open – Half open

  • At some point, the breaker will try to execute the request function and switch to half-open

3 Half-open – Off

  • If the number of successful attempts to execute the request exceeds the specified threshold, the state goes to closed

4 Half open – Open

  • If the number of successful attempts to execute the request function does not exceed the threshold, the state is turned to open

Today is here, learning, if there is a deviation, please correct

Welcome to like, follow and favorites

Friends, your support and encouragement, I insist on sharing, improve the quality of the power

All right, that’s it for this time

Technology is open, our mentality, should be more open. Embrace change, live in the sun, and strive to move forward.

I am Nezha, welcome to like, see you next time ~