Series of articles:

SpringCloud source series (1) – registry initialization for Eureka

SpringCloud source code series (2) – Registry Eureka service registration, renewal

SpringCloud source code series (3) – Registry Eureka crawl registry

SpringCloud source code series (4) – Registry Eureka service offline, failure, self-protection mechanism

SpringCloud source series (5) – Registry EurekaServer cluster for Eureka

SpringCloud source Code Series (6) – Summary of the Registry Eureka

SpringCloud source series (7) – load balancing Ribbon RestTemplate

SpringCloud source series (8) – load balancing Ribbon core principles

SpringCloud source series (9) – load balancing Ribbon core components and configuration

SpringCloud source Series (10) – HTTP client component of load balancing Ribbon

Ribbon retry

From the analysis of the previous articles, we can know that there are two components with retry function, one is LoadBalancerCommand of the Ribbon and the other is RetryTemplate of Spring-Retry. RetryableRibbonLoadBalancingHttpClient and RetryableOkHttpLoadBalancingClient relies on RetryTemplate, so must first introduced spring – retry dependence, They all end up using the RetryTemplate to implement the ability to request retry. Besides RetryTemplate, other client want to retry functionality, with AbstractLoadBalancerAwareClient related components in the ribbon, and call the executeWithLoadBalancer method.

Load balancing client

1, AbstractLoadBalancerAwareClient

Again see AbstractLoadBalancerAwareClient system, through the source code can learn:

  • RetryableFeignLoadBalancer, RetryableRibbonLoadBalancingHttpClient, RetryableOkHttpLoadBalancingClient are usedRetryTemplateSpring-retry retries are implemented.
  • RestClient, FeignLoadBalancer, RibbonLoadBalancingHttpClient, OkHttpLoadBalancingClient is in AbstractLoadBalancerAwareClient The use ofLoadBalancerCommandRetries are implemented by the Ribbon.

2. Load balancing calls

Specific AbstractLoadBalancerAwareClient can call the client want to load balance and retry, need to call AbstractLoadBalancerAwareClient executeWithLoadBalancer method.

In this method, it first builds LoadBalancerCommand, and then commits a ServerOperation with command that reconstructs the URI, Go to the specific LoadBalancerContext to execute the request.

public T executeWithLoadBalancer(final S request, final IClientConfig requestConfig) throws ClientException {
    // Load balancing command
    LoadBalancerCommand<T> command = buildLoadBalancerCommand(request, requestConfig);

    try {
        return command.submit(
            new ServerOperation<T>() {
                @Override
                public Observable<T> call(Server server) {
                    / / refactoring URI
                    URI finalUri = reconstructURIWithServer(server, request.getUri());
                    S requestForServer = (S) request.replaceUri(finalUri);
                    try {
                        / / the use of specific AbstractLoadBalancerAwareClient client request execution
                        return Observable.just(AbstractLoadBalancerAwareClient.this.execute(requestForServer, requestConfig));
                    }
                    catch (Exception e) {
                        returnObservable.error(e); } } }) .toBlocking() .single(); }}Copy the code

Build LoadBalancerCommand

Look at buildLoadBalancerCommand method, it will first through getRequestSpecificRetryHandler RequestSpecificRetryHandler request retry () method to obtain the processor, While getRequestSpecificRetryHandler () is an abstract method. This is where you have to pay attention.

// Abstract method to get the request retry handler
public abstract RequestSpecificRetryHandler getRequestSpecificRetryHandler(S request, IClientConfig requestConfig);

protected LoadBalancerCommand<T> buildLoadBalancerCommand(final S request, final IClientConfig config) {
    // Get the request retry handler
    RequestSpecificRetryHandler handler = getRequestSpecificRetryHandler(request, config);
    LoadBalancerCommand.Builder<T> builder = LoadBalancerCommand.<T>builder()
            .withLoadBalancerContext(this)
            .withRetryHandler(handler)
            .withLoadBalancerURI(request.getUri());
    customizeLoadBalancerCommandBuilder(request, config, builder);
    return builder.build();
}
Copy the code

Request retry handler

RequestSpecificRetryHandler

To understand the RequestSpecificRetryHandler:

  • First look at its construction method, pay attention to the first and second parameters, because different getRequestSpecificRetryHandler () method, the main difference is that these two parameters.
  • Then watchisRetriableExceptionThis method is used by LoadBalancerCommand to determine whether an exception needs to be retriedokToRetryOnAllErrors=trueCan be retried, otherwiseokToRetryOnConnectErrors=trueMay retry. Note that this method may not retry even if it returns true, depending on the number of retries.
public RequestSpecificRetryHandler(boolean okToRetryOnConnectErrors, boolean okToRetryOnAllErrors, RetryHandler baseRetryHandler, @Nullable IClientConfig requestConfig) {
    Preconditions.checkNotNull(baseRetryHandler);
    this.okToRetryOnConnectErrors = okToRetryOnConnectErrors;
    this.okToRetryOnAllErrors = okToRetryOnAllErrors;
    this.fallback = baseRetryHandler;
    if(requestConfig ! =null) {
        // Number of retries on the same Server
        if (requestConfig.containsProperty(CommonClientConfigKey.MaxAutoRetries)) {
            retrySameServer = requestConfig.get(CommonClientConfigKey.MaxAutoRetries);
        }
        // The number of times to retry the next Server
        if(requestConfig.containsProperty(CommonClientConfigKey.MaxAutoRetriesNextServer)) { retryNextServer = requestConfig.get(CommonClientConfigKey.MaxAutoRetriesNextServer); }}}@Override
public boolean isRetriableException(Throwable e, boolean sameServer) {
    // All errors are retried
    if (okToRetryOnAllErrors) {
        return true;
    }
    // ClientException may be retried
    else if (e instanceof ClientException) {
        ClientException ce = (ClientException) e;
        if (ce.getErrorType() == ClientException.ErrorType.SERVER_THROTTLED) {
            return! sameServer; }else {
            return false; }}else  {
        // Retry only when SocketException is thrown
        returnokToRetryOnConnectErrors && isConnectionException(e); }}Copy the code

Retry handlers for different HTTP clients

1, the RestClient

The default configuration, the RestClient getRequestSpecificRetryHandler will go to the last step, okToRetryOnConnectErrors, okToRetryOnAllErrors to true, That is, isRetriableException always returns true, which means that an exception thrown is always retried. For non-GET requests, okToRetryOnAllErrors is false and will be retried only if the connection is abnormal.

@Override
public RequestSpecificRetryHandler getRequestSpecificRetryHandler( HttpRequest request, IClientConfig requestConfig) {
    if(! request.isRetriable()) {return new RequestSpecificRetryHandler(false.false.this.getRetryHandler(), requestConfig);
    }
    if (this.ncc.get(CommonClientConfigKey.OkToRetryOnAllOperations, false)) {
        return new RequestSpecificRetryHandler(true.true.this.getRetryHandler(), requestConfig);
    }
    if(request.getVerb() ! = HttpRequest.Verb.GET) {return new RequestSpecificRetryHandler(true.false.this.getRetryHandler(), requestConfig);
    } else {
        // okToRetryOnConnectErrors and okToRetryOnAllErrors are true
        return new RequestSpecificRetryHandler(true.true.this.getRetryHandler(), requestConfig); }}Copy the code

2, AbstractLoadBalancingClient

AbstractLoadBalancingClient getRequestSpecificRetryHandler is equivalent to a default implementation of default okToRetryOnAllOperations to false, And then the last step, okToRetryOnConnectErrors, okToRetryOnAllErrors are both true, and isRetriableException always returns true. For non-GET requests, okToRetryOnAllErrors is false and will be retried only if the connection is abnormal.

@Override
public RequestSpecificRetryHandler getRequestSpecificRetryHandler(final S request, final IClientConfig requestConfig) {
    // okToRetryOnAllOperations: Whether all operations are retried. The default is false
    if (this.okToRetryOnAllOperations) {
        return new RequestSpecificRetryHandler(true.true.this.getRetryHandler(), requestConfig);
    }
    if(! request.getContext().getMethod().equals("GET")) {
        return new RequestSpecificRetryHandler(true.false.this.getRetryHandler(), requestConfig);
    }
    else {
        return new RequestSpecificRetryHandler(true.true.this.getRetryHandler(), requestConfig); }}Copy the code

3, RibbonLoadBalancingHttpClient

RibbonLoadBalancingHttpClient also overloaded getRequestSpecificRetryHandler, But it sets okToRetryOnConnectErrors, okToRetryOnAllErrors to false, and isRetriableException always returns false.

At this point we should know why call RibbonLoadBalancingHttpClient executeWithLoadBalancer does not have the function that try again. So to enable the apache httpclient, RibbonLoadBalancingHttpClient calls are not support to retry.

@Override
public RequestSpecificRetryHandler getRequestSpecificRetryHandler(RibbonApacheHttpRequest request, IClientConfig requestConfig) {
    // okToRetryOnConnectErrors and okToRetryOnAllErrors are both false
    return new RequestSpecificRetryHandler(false.false, RetryHandler.DEFAULT, requestConfig);
}
Copy the code

Also rewrite the getRequestSpecificRetryHandler RetryableRibbonLoadBalancingHttpClient, Also set okToRetryOnConnectErrors and okToRetryOnAllErrors to false. But after spring-Retry is introduced, it uses RetryTemplate for retries.

@Override
public RequestSpecificRetryHandler getRequestSpecificRetryHandler(RibbonApacheHttpRequest request, IClientConfig requestConfig) {
    // okToRetryOnConnectErrors and okToRetryOnAllErrors are both false
    return new RequestSpecificRetryHandler(false.false, RetryHandler.DEFAULT, null);
}
Copy the code

4, OkHttpLoadBalancingClient

OkHttpLoadBalancingClient didn’t rewrite getRequestSpecificRetryHandler, so it is use the method in the superclass AbstractLoadBalancingClient, OkToRetryOnConnectErrors and okToRetryOnAllErrors are both true.

So, enable okhttp, OkHttpLoadBalancingClient is support all GET retry, not GET requests are thrown support for connection to the abnormal (SocketException) try again.

While RetryableOkHttpLoadBalancingClient like RetryableRibbonLoadBalancingHttpClient way of rewriting, use RetryTemplate implement retry.

@Override
public RequestSpecificRetryHandler getRequestSpecificRetryHandler(RibbonApacheHttpRequest request, IClientConfig requestConfig) {
    // okToRetryOnConnectErrors and okToRetryOnAllErrors are both false
    return new RequestSpecificRetryHandler(false.false, RetryHandler.DEFAULT, null);
}
Copy the code

LoadBalancerCommand retry

Look at the Submit method of LoadBalancerCommand, which is the core code for retry.

  • The number of retries of the same Server is obtainedmaxRetrysSameAnd retry the next ServermaxRetrysNextIn fact, this is the previous configurationribbon.MaxAutoRetriesribbon.MaxAutoRetriesNextServerI set it to 1.
  • It then creates an Observable whose first layer passes throughloadBalancerContextAccess to the Server. When the next Server is retried, the next Server is fetched here.
  • On the second layer, we create an Observable, which is calledServerOperationIs a refactoring a URI, call specific AbstractLoadBalancerAwareClient perform the request.
  • In the second layer, it will be based onmaxRetrysSameRetry the same Server fromretryPolicy()When the number of retries is greater thanmaxRetrysSameAfter the same Server retry ends, otherwise useretryHandler.isRetriableException()Determine whether to retry, which was analyzed earlier.
  • In the outer layer, according tomaxRetrysNextRetry different servers from retryPolicy()When the number of retries of different servers is greater thanmaxRetrysNextThen the retry is complete and the whole retry is complete. If it still fails, the next checkup is onErrorResumeNext.
public Observable<T> submit(final ServerOperation<T> operation) {
    final ExecutionInfoContext context = new ExecutionInfoContext();

    // Number of retries for the same Server
    final int maxRetrysSame = retryHandler.getMaxRetriesOnSameServer();
    // The number of times to retry the next Server
    final int maxRetrysNext = retryHandler.getMaxRetriesOnNextServer();

    // Create an Observable
    Observable<T> o =
            // Use loadBalancerContext to get the Server
            (server == null ? selectServer() : Observable.just(server))
            .concatMap(new Func1<Server, Observable<T>>() {
                @Override
                public Observable<T> call(Server server) {
                    / / set the Server
                    context.setServer(server);
                    final ServerStats stats = loadBalancerContext.getServerStats(server);

                    / / create observables
                    Observable<T> o = Observable
                            .just(server)
                            .concatMap(new Func1<Server, Observable<T>>() {
                                @Override
                                public Observable<T> call(final Server server) {
                                    // Increase the number of attempts
                                    context.incAttemptCount();
                                    // ...
                                    / / call ServerOperation
                                    return operation.call(server).doOnEach(new Observer<T>() {
                                        // Some callback methods}); }});// Retry the same Server
                    if (maxRetrysSame > 0)
                        o = o.retry(retryPolicy(maxRetrysSame, true));
                    returno; }});if (maxRetrysNext > 0 && server == null)
        // Retry different servers
        o = o.retry(retryPolicy(maxRetrysNext, false));

    return o.onErrorResumeNext(new Func1<Throwable, Observable<T>>() {
        @Override
        public Observable<T> call(Throwable e) {
            // Exception handling
            returnObservable.error(e); }}); }// retryPolicy returns an assertion about whether to retry
private Func2<Integer, Throwable, Boolean> retryPolicy(final int maxRetrys, final boolean same) {
    return new Func2<Integer, Throwable, Boolean>() {
        @Override
        public Boolean call(Integer tryCount, Throwable e) {
            // Request rejection exception is not allowed to retry
            if (e instanceof AbortExecutionException) {
                return false;
            }
            // Whether the number of attempts exceeds the maximum number of retries
            if (tryCount > maxRetrys) {
                return false;
            }
            / / use RequestSpecificRetryHandler judge whether to retry
            returnretryHandler.isRetriableException(e, same); }}; }Copy the code

To conclude, LoadBalancerCommand retry:

  • Retries are classified into retries on the same Server and retries on the next Server. When the number of retries exceeds the value of retries, the system stops retries. Otherwise, byretryHandler.isRetriableException()Determine whether to retry.
  • So how many times have we asked for this? The following formula can be summarized:Number of requests = (maxRetrysSame + 1) * (maxRetrysNext + 1), so the ribbon. MaxAutoRetries = 1, ribbon. MaxAutoRetriesNextServer = 1 configuration, if every request timeout, a total of four will initiate the request.

RetryTemplate retry

spring-retry

To enable RetryTemplate, spring-retry is introduced:

<dependency>
    <groupId>org.springframework.retry</groupId>
    <artifactId>spring-retry</artifactId>
</dependency>
Copy the code

RetryableRibbonLoadBalancingHttpClient, for example, first take a look at it the execute method, it created the first load balancing LoadBalancedRetryPolicy retry strategy class, The RetryCallback logic is then wrapped into the RetryCallback, and the RetryTemplate is used to execute the RetryCallback, meaning that the retry logic is in the RetryTemplate.

public RibbonApacheHttpResponse execute(final RibbonApacheHttpRequest request, final IClientConfig configOverride) throws Exception {
    / /...
    / / load balancing RibbonLoadBalancedRetryPolicy retry strategy
    final LoadBalancedRetryPolicy retryPolicy = loadBalancedRetryFactory.createRetryPolicy(this.getClientName(), this);

    RetryCallback<RibbonApacheHttpResponse, Exception> retryCallback = context -> {
        // ...
        // delegate => CloseableHttpClient
        final HttpResponse httpResponse = RetryableRibbonLoadBalancingHttpClient.this.delegate.execute(httpUriRequest);
        // ...
        // Success returns the result
        return new RibbonApacheHttpResponse(httpResponse, httpUriRequest.getURI());
    };

    return this.executeWithRetry(request, retryPolicy, retryCallback, recoveryCallback);
}

private RibbonApacheHttpResponse executeWithRetry(RibbonApacheHttpRequest request, LoadBalancedRetryPolicy retryPolicy, RetryCallback
       
         callback, RecoveryCallback
        
          recoveryCallback)
        
       ,> throws Exception {
    RetryTemplate retryTemplate = new RetryTemplate();

    // Retryable => The Retryable parameter is obtained from the RibbonCommandContext setting
    boolean retryable = isRequestRetryable(request);
    // Set the retry policy
    retryTemplate.setRetryPolicy(retryPolicy == null| |! retryable ?new NeverRetryPolicy() : new RetryPolicy(request, retryPolicy, this.this.getClientName()));

    BackOffPolicy backOffPolicy = loadBalancedRetryFactory.createBackOffPolicy(this.getClientName());
    retryTemplate.setBackOffPolicy(backOffPolicy == null ? new NoBackOffPolicy() : backOffPolicy);

    // Perform the request callback using retryTemplate
    return retryTemplate.execute(callback, recoveryCallback);
}
Copy the code

It is important to note that in executeWithRetry, it determines whether to retry, The getRetryable parameter is the retronCommandContext parameter created by the executeInternal method of ApacheClientHttpRequest. This connects with the logic of the previous customization.

private boolean isRequestRetryable(ContextAwareRequest request) {
    if (request.getContext() == null || request.getContext().getRetryable() == null) {
        return true;
    }
    return request.getContext().getRetryable();
}
Copy the code

RetryTemplate

Enter the RetryTemplate execute method, and the core logic is reduced to the following code, which is basically a while loop to determine whether it can retry, and then call retryCallback to execute the request. When a request fails, such as a timeout, and an exception is thrown, registerThrowable() is used to register the exception.

protected <T, E extends Throwable> T doExecute(RetryCallback
       
         retryCallback, RecoveryCallback
        
          recoveryCallback, RetryState state)
        
       ,> throws E, ExhaustedRetryException {

    // retryPolicy => InterceptorRetryPolicy
    RetryPolicy retryPolicy = this.retryPolicy;
    BackOffPolicy backOffPolicy = this.backOffPolicy;
    //....
    try {
        // ...
        // canRetry Determines whether to retry
        while(canRetry(retryPolicy, context) && ! context.isExhaustedOnly()) {try {
                / / retryCallback calls
                return retryCallback.doWithRetry(context);
            }
            catch (Throwable e) {
                // ...
                // The registration is abnormal
                registerThrowable(retryPolicy, state, context, e);
                // ...
            }
        }
        exhausted = true;
        return handleRetryExhausted(recoveryCallback, context, state);
    }
    / /...
}
Copy the code

The canRetry method actually calls the canRetry of the InterceptorRetryPolicy. The first time it’s called, it gets the Server; RibbonLoadBalancedRetryPolicy or you use to judge whether to retry the next Server, pay attention to it, the logic of judgment is a GET request or allow retry all operating operations, And the number of Server retries nextServerCount is less than or equal to the configured MaxAutoRetriesNextServer. That is, the canRetry judged by the while loop is to retry the next Server.

protected boolean canRetry(RetryPolicy retryPolicy, RetryContext context) {
    return retryPolicy.canRetry(context);
}

//////////// InterceptorRetryPolicy
public boolean canRetry(RetryContext context) {
    LoadBalancedRetryContext lbContext = (LoadBalancedRetryContext) context;
    if (lbContext.getRetryCount() == 0 && lbContext.getServiceInstance() == null) {
        / / access to the Server
        lbContext.setServiceInstance(this.serviceInstanceChooser.choose(this.serviceName));
        return true;
    }
    / / RibbonLoadBalancedRetryPolicy = > try again next Server
    return this.policy.canRetryNextServer(lbContext);
}

///////// RibbonLoadBalancedRetryPolicy
public boolean canRetryNextServer(LoadBalancedRetryContext context) {
    // Decide to retry the next Server
    return nextServerCount <= lbContext.getRetryHandler().getMaxRetriesOnNextServer()
            && canRetry(context);
}

public boolean canRetry(LoadBalancedRetryContext context) {
    // Retries are allowed when a GET request or all operations are allowed to retry
    HttpMethod method = context.getRequest().getMethod();
    return HttpMethod.GET == method || lbContext.isOkToRetryOnAllOperations();
}
Copy the code

Then request failed to register abnormal registerThrowable (), it will last to the abnormal RibbonLoadBalancedRetryPolicy registration. In RibbonLoadBalancedRetryPolicy registerThrowable () method, if can not to retry the same Server and retry the next Server, will be under polling for a Server. If it can be retried on the sameServer, the sameServerCount counter is +1, otherwise reset sameServerCount, then nextServerCount +1.

protected void registerThrowable(RetryPolicy retryPolicy, RetryState state, RetryContext context, Throwable e) {
    retryPolicy.registerThrowable(context, e);
    registerContext(context, state);
}

///////// InterceptorRetryPolicy /////////
public void registerThrowable(RetryContext context, Throwable throwable) {
    LoadBalancedRetryContext lbContext = (LoadBalancedRetryContext) context;
    lbContext.registerThrowable(throwable);
    // RibbonLoadBalancedRetryPolicy
    this.policy.registerThrowable(lbContext, throwable);
}

///////// RibbonLoadBalancedRetryPolicy /////////
public void registerThrowable(LoadBalancedRetryContext context, Throwable throwable) {
    / /...
    // If the Server is not on the same Server and the next Server can be retried, select a new Server
    if(! canRetrySameServer(context) && canRetryNextServer(context)) { context.setServiceInstance(loadBalanceChooser.choose(serviceId)); }// If the sameServer retries beyond the set value, the sameServerCount is reset
    if (sameServerCount >= lbContext.getRetryHandler().getMaxRetriesOnSameServer()
            && canRetry(context)) {
        / / reset nextServerCount
        sameServerCount = 0;
        // Next Server retries +1
        nextServerCount++;
        if(! canRetryNextServer(context)) {// Cannot retry the next Servercontext.setExhaustedOnly(); }}else {
        // Number of retries for the same Server +1sameServerCount++; }}// Determine whether to retry the same Server
public boolean canRetrySameServer(LoadBalancedRetryContext context) {
    return sameServerCount < lbContext.getRetryHandler().getMaxRetriesOnSameServer()
            && canRetry(context);
}

public boolean canRetry(LoadBalancedRetryContext context) {
    // Retries are allowed when a GET request or all operations are allowed to retry
    HttpMethod method = context.getRequest().getMethod();
    return HttpMethod.GET == method || lbContext.isOkToRetryOnAllOperations();
}
Copy the code

Ribbon and RetryTemplate retry summary

1. First, the Ribbon configuration parameters for timeouts and retries are as follows. These parameters can also be configured for a client

ribbon:
  Client read timeout
  ReadTimeout: 1000
  Client connection timeout
  ConnectTimeout: 1000
  If set to true, retry all types, such as POST, PUT, and DELETE
  OkToRetryOnAllOperations: false
  Number of retries for the same Server
  MaxAutoRetries: 1
  Retry a maximum of several servers
  MaxAutoRetriesNextServer: 1
Copy the code

The RetryTemplate component is spring-Retry. LoadBalancerCommand is the retry component of the Ribbon. They retry the same number of requests and the retry logic is similar. They retry the current Server and then the next Server. The total number of requests = (MaxAutoRetries + 1) * (MaxAutoRetriesNextServer + 1).

RetryTemplate allows retries only when the request method is GET or OkToRetryOnAllOperations=true. LoadBalancerCommand allows retries only when the request method is GET or OkToRetryOnAllOperations=true. Non-get methods can also retry when they throw a connection exception. Generally, only GET is allowed to retry. GET is a query operation and the interface is idempotent. POST, PUT, and DELETE are non-idempotent. Therefore, it is recommended to use RetryTemplate and set OkToRetryOnAllOperations=false.

4. To improve inter-service communication performance, apache HttpClient or OkHttp can be enabled. To enable the retry function, you need to introduce the Spring-Retry dependency. When retries occur, the current Server tries the next Server instead of trying again (MaxAutoRetries=0).

Ribbon: # ReadTimeout:1000ConnectTimeout:1000# retries only GET by defaulttrueWhen will retry all types, such as POST, PUT, DELETE OkToRetryOnAllOperations:falseMaxAutoRetries:0# again up several Server MaxAutoRetriesNextServer:1Httpclient: enabled:falseRestClient: enabled:falseOkhttp: enabledtrue
Copy the code

Summary of Ribbon Architecture

Finally, the Ribbon core component architecture is summarized in two class diagrams.

ILoadBalancer load balancer

Load balancing client