RestTemplate and load balancer

This article mainly introduces SpringCloud Ribbon load balancer and RestTemplate network request framework to build microservice system. In addition, it will analyze the source code of load balancer and prove how to achieve load balancing through the combination of Ribbon and RestTemplate through examples. Now imagine a distributed system consisting of many services running on different computers. When the number of users is large, it is common to create multiple copies of the service. Each copy is running on another computer, which at this point helps evenly distribute incoming traffic between servers.

In a system, services often need to invoke other services. In a singleton application, services invoke other services through language-level methods or procedures. In a traditional distributed deployment, services run on fixed, known address hosts and ports and can therefore be requested via HTTP/REST or other RPC mechanisms. However, a modern microservice application typically runs in a virtual or container environment, where the number of service instances and their addresses change dynamically. Therefore, there is a need to implement a mechanism that allows a client of a service to make requests to a dynamically changing set of ephemeral service instances. This is service registration and discovery, which is the most important fundamental component of a microservice architecture.

The next thing we need to figure out is what is client discovery and what is server discovery?

The registry here is actually equivalent to the brothel madam, A is the whoremonger, B is the miss. I believe that all the old drivers know the interactive logic between the three. Client discovery is that when A needs to call B service, it requests the registry (B service registers information with the registry when it starts), and the registry returns A complete list of available services to A service, which decides which B service to use. Client discovery features:

• Simple and direct, without the intervention of an agent
• The client (A) knows all the actual service addresses available
• The client (A) needs to implement its own load balancing logic

Example of discovery using a client: Eureka

Compared with client discovery, server discovery has an additional agent. The agent helps A select B from A large number of B. Server discovery features:

• Service (B) and registry are not visible to A due to the broker’s involvement

Examples of server discovery: Nginx, ZooKeeper, Kubernetes

By understanding the difference between client-side discovery and server-side discovery, we can see that which service is called depends on the client or the server. That is, it depends on whether the service registration and discovery uses client-side discovery or server-side discovery.

Server load balancing

Server-side load balancers, common ones such as Nginx and F5, are components placed on the server side. When requests come from clients, they go to the load balancer, which specifies the server for the request. The simplest algorithm used by a load balancer is randomly specified. In this case, most load balancers are hardware-integrated software for controlling load balancing.



Features of server load balancing:

• It is not transparent to the client. The client does not know the list of services on the server, or even that the destination address to which the request is sent has a load balancer.
• Server Maintains load balancing servers and controls load balancing policies and algorithms.

Client load balancing

1. Directly fill in the service address

If the Order application wants to directly access the Shop application interface, enter the service address for direct access.

2. Use LoadBalancerClient

3. Inject the RestTemplate Bean

Inject the RestTemplate Bean and access it using the service name:

In the example above we use the RestTemplate and enable client load balancing. Enabling load balancing is as simple as adding an @loadBalanced annotation to the RestTemplate bean. We can start with this annotation:

/**
 * Annotation to mark a RestTemplate or WebClient bean to be configured to use a
 * LoadBalancerClient.
 * @author Spencer Gibb
 */
@Target({ ElementType.FIELD, ElementType.PARAMETER, ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
@Qualifier
public @interface LoadBalanced {

}
Copy the code

LoadBalancerClient: LoadBalancerClient: LoadBalancerClient: LoadBalancerClient: LoadBalancerClient: LoadBalancerClient: LoadBalancerClient And inherits from ServiceInstanceChooser:

Public Interface LoadBalancerClient extends ServiceInstanceChooser {// Use a service instance selected from the load balancer to execute the request <T> T execute(String serviceId, LoadBalancerRequest<T> request) throws IOException; <T> T execute(String serviceId, ServiceInstance ServiceInstance, LoadBalancerRequest<T> request) throws IOException; / / for the system to build an appropriate URI / / such as http://SHOP-CLIENT/shop/show - > http://localhost:8080/shop/show URI reconstructURI (ServiceInstance  instance, URI original); }Copy the code

ServiceInstanceChooser Implement the choose method: select an instance from the client load balancer based on the service name serviceId: select an instance from the client load balancer.

ServiceInstance choose(String serviceId);
Copy the code

As for the specific configuration and we need to see LoadBalancerAutoConfiguration source code of a class, the class is a client side load balancing server automation configuration of class, the class of the source code is as follows:

/** * Auto-configuration for Ribbon (client-side load balancing). */ @Configuration(proxyBeanMethods = false) @ConditionalOnClass(RestTemplate.class) @ConditionalOnBean(LoadBalancerClient.class) @EnableConfigurationProperties(LoadBalancerRetryProperties.class) public class LoadBalancerAutoConfiguration { @LoadBalanced @Autowired(required = false) private List<RestTemplate> restTemplates = Collections.emptyList(); @Autowired(required = false) private List<LoadBalancerRequestTransformer> transformers = Collections.emptyList(); @Bean public SmartInitializingSingleton loadBalancedRestTemplateInitializerDeprecated( final ObjectProvider<List<RestTemplateCustomizer>> restTemplateCustomizers) { return () -> restTemplateCustomizers.ifAvailable(customizers -> { for (RestTemplate restTemplate : LoadBalancerAutoConfiguration.this.restTemplates) { for (RestTemplateCustomizer customizer : customizers) { customizer.customize(restTemplate); }}}); } @Bean @ConditionalOnMissingBean public LoadBalancerRequestFactory loadBalancerRequestFactory( LoadBalancerClient loadBalancerClient) { return new LoadBalancerRequestFactory(loadBalancerClient, this.transformers); } @Configuration(proxyBeanMethods = false) @ConditionalOnMissingClass("org.springframework.retry.support.RetryTemplate")  static class LoadBalancerInterceptorConfig { @Bean public LoadBalancerInterceptor ribbonInterceptor( LoadBalancerClient  loadBalancerClient, LoadBalancerRequestFactory requestFactory) { return new LoadBalancerInterceptor(loadBalancerClient, requestFactory); } @Bean @ConditionalOnMissingBean public RestTemplateCustomizer restTemplateCustomizer( final LoadBalancerInterceptor loadBalancerInterceptor) { return restTemplate -> { List<ClientHttpRequestInterceptor> list = new ArrayList<>( restTemplate.getInterceptors()); list.add(loadBalancerInterceptor); restTemplate.setInterceptors(list); }; } } @Configuration(proxyBeanMethods = false) @ConditionalOnClass(RetryTemplate.class) public static class RetryAutoConfiguration { @Bean @ConditionalOnMissingBean public LoadBalancedRetryFactory loadBalancedRetryFactory() { return new LoadBalancedRetryFactory() { }; } } @Configuration(proxyBeanMethods = false) @ConditionalOnClass(RetryTemplate.class) public static class RetryInterceptorAutoConfiguration { @Bean @ConditionalOnMissingBean public RetryLoadBalancerInterceptor ribbonInterceptor( LoadBalancerClient loadBalancerClient, LoadBalancerRetryProperties properties, LoadBalancerRequestFactory requestFactory, LoadBalancedRetryFactory loadBalancedRetryFactory) { return new RetryLoadBalancerInterceptor(loadBalancerClient, properties, requestFactory, loadBalancedRetryFactory); } @Bean @ConditionalOnMissingBean public RestTemplateCustomizer restTemplateCustomizer( final RetryLoadBalancerInterceptor loadBalancerInterceptor) { return restTemplate -> { List<ClientHttpRequestInterceptor> list  = new ArrayList<>( restTemplate.getInterceptors()); list.add(loadBalancerInterceptor); restTemplate.setInterceptors(list); }; }}}Copy the code

LoadBalancerAutoConfiguration class has two key notes, ConditionalOnClass(restTemplate.class) and ConditionalOnBean(loadBalancerClient.class), The RestTemplate class must exist in the current project environment. Second, there must Bean implementation Bean for LoadBalancerClient in the Spring container.

RetryInterceptorAutoConfiguration class ribbonInterceptor method returns an interceptor called LoadBalancerInterceptor, the function of the interceptor is mainly in the client request to intercept, The restTemplateCustomizer method returns a restTemplateCustomizer, This method is mainly used to add the LoadBalancerInterceptor interceptor to the RestTemplate. LoadBalancerAutoConfiguration restTemplates is a @ LoadBalanced annotation of modified RestTemplate object list, Add a LoadBalancerInterceptor interceptor to each RestTemplate object using the restTemplateCustomizer method.

These interceptors provide load balancing for a normal RestTemplate object. LoadBalancerInterceptor provides load balancing for a normal RestTemplate object.

public class LoadBalancerInterceptor implements ClientHttpRequestInterceptor { private LoadBalancerClient loadBalancer; private LoadBalancerRequestFactory requestFactory; public LoadBalancerInterceptor(LoadBalancerClient loadBalancer, LoadBalancerRequestFactory requestFactory) { this.loadBalancer = loadBalancer; this.requestFactory = requestFactory; } public LoadBalancerInterceptor(LoadBalancerClient loadBalancer) { // for backwards compatibility this(loadBalancer, new LoadBalancerRequestFactory(loadBalancer)); } @Override public ClientHttpResponse intercept(final HttpRequest request, final byte[] body, final ClientHttpRequestExecution execution) throws IOException { final URI originalUri = request.getURI(); String serviceName = originalUri.getHost(); Assert.state(serviceName ! = null, "Request URI does not contain a valid hostname: " + originalUri); return this.loadBalancer.execute(serviceName, this.requestFactory.createRequest(request, body, execution)); } } @FunctionalInterface public interface ClientHttpRequestInterceptor { ClientHttpResponse intercept(HttpRequest var1, byte[] var2, ClientHttpRequestExecution var3) throws IOException; }Copy the code

When a RestTemplate object decorated with the @LoadBalanced annotation makes an HTTP request, it is intercepted by the LoadBalancerInterceptor class’s Intercept method, In this method, we get the service name directly from the getHost method (since we call the RestTemplate service using the service name instead of the domain name, we can get the service name directly from the getHost method and call the execute method to initiate the request).

RibbonLoadBalancerClient: Execute method: ILoadBalancer

This is an interface to add service instances, select service instances, get all service instances, etc.

Void addServers(List<Server> var1); Server chooseServer(Object var1); // Select a specific service instance from the load balancing Server through some policy. Void markServerDown(Server var1); // Indicates that a specific instance of the load balancer has stopped serving. List<Server> getReachableServers(); <Server> getAllServers(); // Get a List of all service instances, both normal and stopped. }Copy the code

Let’s look at the basic BaseLoadBalancer:

It is not hard to find that the default load balancing policy is polling. As for the load balancing strategy, there are many implementations:

To sum up, the RestTemplate initiates a request, which is intercepted by the LoadBalancerInterceptor, converts the logical service name in the requested address to the specific service address, and proceeds with the requested process.