This is the second day of my August challenge

This series is the reset version of the previous series. With the development of the project and the use of the project, many things in the previous series have changed, and there are some things that were not mentioned in the previous series, so restart this series and rearrange it. Welcome to leave a message and exchange, thank you! ~

A very simple microservice architecture is illustrated in the figure above:

  • Microservices are registered with the registry.
  • The microservice reads the list of service instances from the registry.
  • Microservices call each other based on the list of service instances they read.
  • External access is through a unified API gateway.
  • The API gateway reads the list of service instances from the registry and invokes the corresponding microservice for access based on the access path.

The functions that each process needs to implement in this microservice architecture are shown in the following figure:

Let’s take a look at each role in the architecture one by one, the functionality involved, the issues to consider, and the libraries we use in this series.

The basic functions of each microservice include:

  • Logs are displayed, and link tracing information is displayed in logs. In addition, with the increasing business pressure, each process may output more and more logs, which may become a performance bottleneck. We used log4J2 asynchronous logs and Spring-Cloud-sleUTH as the core dependency of link tracing.
  • Http container: Containers providing Http interfaces, divided into synchronous Spring-MVC and asynchronous Spring-webFlux:
    • For Spring-MVC, the default Http container is Tomcat. In a high-concurrency environment, there are many requests. Instead of using the default Tomcat, we used Undertow instead to optimize performance by minimizing GC application by using direct memory processing requests.
    • For Spring-WebFlux, we directly use WebFlux itself as Http container. In fact, the bottom layer is reactor-HTTP, and the bottom layer is netty server based on Http protocol. It is inherently asynchronous and responsive, and the request memory basically uses direct memory.
  • Microservice discovery and Registration: We used Eureka as the registry. Our cluster normally has a lot of releases and needs to be aware of instances going up and down quickly. At the same time, we have many sets of clusters, each cluster service instance node number is about 100, it would be a waste to use one Eureka cluster per cluster, and we want to have a management platform to directly manage all the cluster nodes. So we use the same set of Eureka for all clusters, but the framework configuration ensures that only instances within the same cluster are discovered and invoked by each other.
  • Health check: Because K8s requires processes to provide health check interfaces, we use Spring Boot’s actuator function as health check interfaces. At the same time, we also expose other actuator interfaces using Http, such as dynamic log level changes and hot reboots.
  • Indicators collection: We use Prometheus to collect internal indicators of processes, and expose the interface of the actuator for grafana and K8s calls.
  • The Http client: Internal microservice calls are Http calls. Every microservice requires an Http client. In our case the Http client has:
    • For synchronous Spring-MVC, we generally useOpen-feignAnd each microservice itself maintains its own microservice provided by the Open-Feign client. We don’t use it@LoadBalancedannotationsRestTemplate
    • For synchronous Spring-Flux, WebClient is generally used to invoke it.
  • Load balancing: obviously, most of the load balancing in Spring Cloud is client-side load balancing, and we use spring-cloud-loadbalancer as our loadbalancer.
  • Gracefully shut down: We want the microserver process to mark itself offline in the registry when it receives a shut down signal; All requests received at the same time are not processed, and the status code similar to 503 is returned. And exit after all the threads have finished processing the work at hand, which is graceful closure. This feature was introduced after Spring Boot 2.3.x and will be used throughout this series.

There will also be retry mechanisms, flow limiting mechanisms, and disconnection mechanisms. Here we will focus on these mechanisms and the issues to be considered in Http clients that call other microservices.

Take a look at a few scenarios:

1. When the service is published online or a service is offline due to problems, the old service instance has been offline in the registry and the instance has been closed, but other microservices have local service instance cache or are using this service instance to call. A java.io.IOException is thrown because a TCP connection cannot be established. Different frameworks use different subexceptions of this exception, but the message is usually connect time out or no route to host. If you try again and retry a normal instance instead of the same instance, the call succeeds. As shown below:

2. When calling a microservice returns a non-2xx response code:

A) 4XX: When publishing interface updates, both the caller and the called may need to publish them. If the new interface parameter changes and is not compatible with the old call, there will be an exception, usually a parameter error, that is, return 4XX response code. For example, the new caller calls the old called. In this case, retry can solve the problem. But to be on the safe side, we only retry GET methods (that is, query methods, or non-GET methods that are explicitly marked to retry) for requests that have already been issued. We do not retry non-GET requests. As shown below:

B) 5XX: a 5XX exception occurs when an instance fails to connect to the database, JVM stop-the-world, etc. In this case, retry is also possible. Again, to be on the safe side, we retry only GET methods (that is, query methods, or non-GET methods that are explicitly marked to retry) for such requests that have already been issued, and we do not retry non-GET requests. As shown below:

3. Circuit breaker open exception: As we will see later, our circuit breaker is specific to an instance of a microservice at a method level. If a circuit breaker open exception is thrown and the request is not actually sent, we can retry directly.

These scenarios are still common when updates are published online, and when traffic suddenly arrives causing problems in some instances. If no retry is performed, users often see abnormal pages, which affects user experience. So retries are necessary in these scenarios. For retries, we used Resilience4J as the core of our entire framework to implement the retry mechanism.

Here’s another scenario:

Microservice A invokes all instances of microservice B from the same thread pool. If an instance has a problem, the request is blocked, or the response is very slow. Over time, the thread pool will be filled with requests to the exception instance, but microservice B actually has a working instance.

To prevent this, and to limit the concurrency (i.e., flow limiting) of calling each microservice instance, we use different thread pools to call different instances of different microservices. This is also done through Resilience4J.

If an instance for a period of stress lead to request slow, or instance is shut down, and there is something wrong with the instance to request response is mostly 500, so even if we have a retry mechanism, if there is something wrong with the many requests are in accordance with the request to the instance of the – > – > try again other instance failure, so efficiency is very low. This requires the use of circuit breakers.

In practice, we find that in most cases, some interfaces of some instances of a microservice have exceptions, while other interfaces on these instances are often available. So our circuit breaker cannot directly disconnect the entire instance, let alone the entire microservice. Therefore, what we use Resilience4J to achieve is the method level circuit breaker of microservice instance (i.e. different microservices, different methods of different instances are different circuit breakers).

In this section, we proposed a simple micro service architecture, and carefully analyses the micro service instances involving common components use library as well as the need to consider problems, and aimed at the core of the micro service invocation retry mechanism, from the Http client thread isolation mechanism and the circuit breaker mechanism need to consider the problems and how to design made a more detailed instructions. Next, let’s look at the mechanisms that need to be considered for Eureka registry and API gateway design.

Wechat search “my programming meow” public account, a daily brush, easy to improve skills, won a variety of offers