Examples of this series with glue code address:
Github.com/HashZhang/s…

Load balancing Ribbon replaced with Spring Cloud Load Balancer

Spring Cloud Load Balancer is not a standalone project, but a module of Spring-Cloud-Commons. With Eureka and related starter, it is almost impossible to completely remove Ribbon dependencies. People in the Spring community have also seen this, and have configured to close the Ribbon and enable Spring-cloud-loadbalancer.

spring.cloud.loadbalancer.ribbon.enabled=falseCopy the code

When the ribbon is closed, Spring Cloud LoadBalancer loads as the default LoadBalancer.

The Spring Cloud LoadBalancer structure is as follows:



Among them:

  1. There’s only one globalBlockingLoadBalancerClientIs responsible for executing all load balancing requests.
  2. BlockingLoadBalancerClientfromLoadBalancerClientFactoryLoad the load balancing configuration of the corresponding microservice.
  3. Each microservice has its ownLoadBalancer.LoadBalancerIt contains load balancing algorithms, for exampleRoundRobin. According to the algorithm, fromServiceInstanceListSupplierSelect an instance from the list of returned instances.

1. Implement zone isolation

To realize zone isolation, should from ServiceInstanceListSupplier inside them. The default implementation inside about zone isolation ServiceInstanceListSupplier – > org.springframework.cloud.loadbalancer.core.ZonePreferenceServiceInstanceListSupplier:

private List<ServiceInstance> filteredByZone(List<ServiceInstance> serviceInstances) {
	if(zone == null) { zone = zoneConfig.getZone(); } // If the zone is not null and there are living instances in the zone, return the list of instances // otherwise, return all instancesif(zone ! = null) { List<ServiceInstance> filteredInstances = new ArrayList<>();for (ServiceInstance serviceInstance : serviceInstances) {
			String instanceZone = getZone(serviceInstance);
			if(zone.equalsIgnoreCase(instanceZone)) { filteredInstances.add(serviceInstance); }}if (filteredInstances.size() > 0) {
			return filteredInstances;
		}
	}
	// If the zone is not set or there are no zone-specific instances available,
	// we return all instances retrieved for given service id.
	return serviceInstances;
}Copy the code

If no zone is specified or there is no active instance in the zone, all queried instances are returned regardless of zones. This doesn’t meet our requirements, so we modify and implement our own com. Making. Hashjang. The hoxton. Service. Consumer. Config. SameZoneOnlyServiceInstanceListSupplier:

private List<ServiceInstance> filteredByZone(List<ServiceInstance> serviceInstances) {
    if (zone == null) {
        zone = zoneConfig.getZone();
    }
    if(zone ! = null) { List<ServiceInstance> filteredInstances = new ArrayList<>();for (ServiceInstance serviceInstance : serviceInstances) {
            String instanceZone = getZone(serviceInstance);
            if(zone.equalsIgnoreCase(instanceZone)) { filteredInstances.add(serviceInstance); }}if (filteredInstances.size() > 0) {
            returnfilteredInstances; }} // Return an empty list if not found, and never return another cluster instancereturn List.of();
}Copy the code

Then let’s look at the LoadBalancer provided by the default Spring Cloud LoadBalancer, which is cached:

org.springframework.cloud.loadbalancer.annotation.LoadBalancerClientConfiguration

@Bean
@ConditionalOnBean(ReactiveDiscoveryClient.class)
@ConditionalOnMissingBean
public ServiceInstanceListSupplier discoveryClientServiceInstanceListSupplier(
		ReactiveDiscoveryClient discoveryClient, Environment env,
		ApplicationContext context) {
	DiscoveryClientServiceInstanceListSupplier delegate = new DiscoveryClientServiceInstanceListSupplier(
			discoveryClient, env);
	ObjectProvider<LoadBalancerCacheManager> cacheManagerProvider = context
			.getBeanProvider(LoadBalancerCacheManager.class);
	if(cacheManagerProvider.getIfAvailable() ! = null) {return new CachingServiceInstanceListSupplier(delegate,
				cacheManagerProvider.getIfAvailable());
	}
	return delegate;
}Copy the code

DiscoveryClientServiceInstanceListSupplier every time from Eureka pull instance list above, CachingServiceInstanceListSupplier provides a cache, so every time don’t have to pull over the top of Eureka. Can see CachingServiceInstanceListSupplier is a proxy mode of implementation, and SameZoneOnlyServiceInstanceListSupplier pattern is the same.

We to assemble our ServiceInstanceListSupplier, due to the environment we are synchronous, only to realize synchronous ServiceInstanceListSupplier.

Public class CommonLoadBalancerConfig {/ ServiceInstanceListSupplier * * * * synchronous environment Instances of SameZoneOnlyServiceInstanceListSupplier restricted only to return to the same zone (note) * CachingServiceInstanceListSupplier enable cache, Do not access eureka request instance list every time * * @param discoveryClient * @param env * @param zoneConfig * @param context * @return
     */
    @Bean
    @Order(Integer.MIN_VALUE)
    public ServiceInstanceListSupplier discoveryClientServiceInstanceListSupplier(
            DiscoveryClient discoveryClient, Environment env,
            LoadBalancerZoneConfig zoneConfig,
            ApplicationContext context) {
        ServiceInstanceListSupplier delegate = new SameZoneOnlyServiceInstanceListSupplier(
                new DiscoveryClientServiceInstanceListSupplier(discoveryClient, env),
                zoneConfig
        );
        ObjectProvider<LoadBalancerCacheManager> cacheManagerProvider = context
                .getBeanProvider(LoadBalancerCacheManager.class);
        if(cacheManagerProvider.getIfAvailable() ! = null) {return new CachingServiceInstanceListSupplier(
                    delegate,
                    cacheManagerProvider.getIfAvailable()
            );
        }
        returndelegate; }}Copy the code

2. During the next retry, if other instances exist, the system will retry other instances

The default RoundRobinLoadBalancer is a RoundRobinLoadBalancer, whose polling position is of Atomic type, shared by all threads and requests under a microservice invocation request (each other microservice invocation creates a RoundRobinLoadBalancer). In use, there is a problem like this:

  • Suppose A microservice has two instances, instance A and instance B
  • Position = position + 1 for request X to instance A
  • When the request does not return, request Y arrives and is sent to instance B, position = position + 1
  • Request A fails. Retry. The retry instance is still instance A

Thus, in a retry case, a retry of a request might be sent to the last instance for retries, which is not what we want. For this, I have an Issue: Enhance RoundRoubinLoadBalancer position. The idea of my modification is that we need a position isolated by a single request, and this position can obtain the instance to which the request is to be sent by mod the number of instances. So how do you isolate requests?

The first thing that comes to mind is thread isolation, but that doesn’t work. Spring Cloud LoadBalancer uses the reactor framework to actually host the selected instance, not the business thread, but the thread pool in the REACTOR, as shown in the figure below:

Therefore, you cannot implement position using ThreadLocal.

Since we used sleuth, the context of the normal request will pass the traceId in it. We can use this traceId to distinguish between different requests and implement our LoadBalancer:

RoundRobinBaseOnTraceIdLoadBalancer

// This timeout needs to be set higher than your request's connectTimeout +readTimeout Long Private final LoadingCache<Long, AtomicInteger> positionCache = Caffeine. NewBuilder (). TimeUnit.SECONDS).build(k -> new AtomicInteger(ThreadLocalRandom.current().nextInt(0, 1000))); private Response<ServiceInstance> getInstanceResponse(List<ServiceInstance> serviceInstances) {if (serviceInstances.isEmpty()) {
        log.warn("No servers available for service: " + this.serviceId);
        returnnew EmptyResponse(); } // If there is no traceId, a new traceId is generated, but it is better to check why there is no traceId. Span currentSpan = tracer.currentSpan();if (currentSpan == null) {
        currentSpan = tracer.newTrace();
    }
    long l = currentSpan.context().traceId();
    int seed = positionCache.get(l).getAndIncrement();
    return new DefaultResponse(serviceInstances.get(seed % serviceInstances.size()));
}Copy the code

3. Replace the default load balancing related Bean implementation

To replace the default implementation with the above two classes, write a configuration class first:

public class CommonLoadBalancerConfig {

    private volatile boolean isValid = false; ServiceInstanceListSupplier * / * * * synchronous environment SameZoneOnlyServiceInstanceListSupplier limit only returns the same instance under the zone (note) * CachingServiceInstanceListSupplier enable cache, Do not access eureka request instance list every time * * @param discoveryClient * @param env * @param zoneConfig * @param context * @return
     */
    @Bean
    @Order(Integer.MIN_VALUE)
    public ServiceInstanceListSupplier discoveryClientServiceInstanceListSupplier(
            DiscoveryClient discoveryClient, Environment env,
            LoadBalancerZoneConfig zoneConfig,
            ApplicationContext context) {
        isValid = true;
        ServiceInstanceListSupplier delegate = new SameZoneOnlyServiceInstanceListSupplier(
                new DiscoveryClientServiceInstanceListSupplier(discoveryClient, env),
                zoneConfig
        );
        ObjectProvider<LoadBalancerCacheManager> cacheManagerProvider = context
                .getBeanProvider(LoadBalancerCacheManager.class);
        if(cacheManagerProvider.getIfAvailable() ! = null) {return new CachingServiceInstanceListSupplier(
                    delegate,
                    cacheManagerProvider.getIfAvailable()
            );
        }
        return delegate;
    }

    @Bean
    public ReactorLoadBalancer<ServiceInstance> reactorServiceInstanceLoadBalancer(
            Environment environment,
            ServiceInstanceListSupplier serviceInstanceListSupplier,
            Tracer tracer) {
        if(! isValid) { throw new IllegalStateException("should use the ServiceInstanceListSupplier in this configuration, please check config");
        }
        String name = environment.getProperty(LoadBalancerClientFactory.PROPERTY_NAME);
        returnnew RoundRobinBaseOnTraceIdLoadBalancer( name, serviceInstanceListSupplier, tracer ); }}Copy the code

Then, specify the default load balancing configuration to take this configuration, via comments:

@LoadBalancerClients(defaultConfiguration = {CommonLoadBalancerConfig.class})Copy the code