First, we recommend you to read dubbo official website source code interpretation of service invocation process section, portal: dubbo.apache.org/zh-cn/docs/…

Call mode

Dubbo calls ResponseFuture’s get method. Dubbo calls ResponseFuture’s get method.

  1. In synchronous invocation mode, the framework itself calls the ResponseFuture get method.
  2. In asynchronous invocation mode, the ResponseFuture get method is called by the user.

ResponseFuture is an interface. The default implementation of ResponseFuture is DefaultFuture. The user thread calling the GET method is blocked when the service consumer has not received the call result.

  1. In synchronous invocation mode, as soon as the framework obtains the DefaultFuture object, it calls the GET method and waits.
  2. In asynchronous mode, the object is encapsulated in the FutureAdapter instance and the FutureAdapter instance is set in the RpcContext for users to use. FutureAdapter is an adapter that ADAPTS ResponseFuture in Dubbo to Future in JDK. So when the user thread calls the Future’s get method, after FutureAdapter adaptation, it ends up calling the ResponseFuture implementation’s get method, which is DefaultFuture’s GET method. Dubbo2.7.0 uses CompletableFuture and sets it in an asynchronous context.

Typically, a service caller invokes multiple services concurrently, and after each user thread sends a request, the get method of a different DefaultFuture object is called to wait. Over time, the service caller’s thread pool receives multiple response objects. One problem to consider at this point is how to pass each response object to the corresponding DefaultFuture object without error. The answer is by calling the number. When DefaultFuture is created, a Request object is passed in. DefaultFuture can then get the call number from the Request object and store the < call number, DefaultFuture object > mapping into the static Map, FUTURES. After receiving the Response object, the thread in the thread pool will fetch the corresponding DefaultFuture object from the FUTURES collection according to the call number in the Response object, and then set the Response object into the DefaultFuture object. Finally, the user thread is woken up so that it can get the result of the call from the DefaultFuture object.

2. Timeout exception

The sent variable in DefaultFuture is written after the client sends a successful request to the server to indicate that the message has been sent.

-->NettyChannel#send
  -->io.netty.channel.ChannelOutboundInvoker#writeAndFlush
    -->NettyClientHandler#write
      -->.DefaultFuture#sent
Copy the code

1. The client times out

If the client fails to send a message, the server does not return a Response, the SENT variable in DefaultFuture is not written, and DefaultFuture#getTimeoutMessage will output a client timeout exception based on whether sent is greater than 0.

2. The server times out

If the client successfully sends a message, the server returns a Response, and the sent variable in DefaultFuture is written. In DefaultFuture#getTimeoutMessage, the server times out based on whether sent is greater than 0.

private String getTimeoutMessage(boolean scan) {
       long nowTimestamp = System.currentTimeMillis();
       return (sent > 0 ? "Waiting server-side response timeout" : "Sending request timeout in client-side")
               + (scan ? " by scan timer" : "") + ". start time: "
               + (new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS").format(new Date(start))) + ", end time: "
               + (new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS").format(new Date())) + ","
               + (sent > 0 ? " client elapsed: " + (sent - start)
               + " ms, server elapsed: " + (nowTimestamp - sent)
               : " elapsed: " + (nowTimestamp - start)) + " ms, timeout: "
               + timeout + " ms, request: " + request + ", channel: " + channel.getLocalAddress()
               + "- >" + channel.getRemoteAddress();
   }
Copy the code

3. Server timeout or client timeout Dubbo how to construct Response

When a timeout exception occurs, no Response is returned. The dubbo client creates a TimeoutCheckTask delay when creating DefaultFuture, which will be executed when the timeout expires. This code is not hard to understand.

public static DefaultFuture newFuture(Channel channel, Request request, int timeout) {
        final DefaultFuture future = new DefaultFuture(channel, request, timeout);
        // timeout check
        timeoutCheck(future);
        return future;
    }
    
 private static class TimeoutCheckTask implements TimerTask {

        private DefaultFuture future;

        TimeoutCheckTask(DefaultFuture future) {
            this.future = future;
        }

        @Override
        public void run(Timeout timeout) {
            if (future == null || future.isDone()) {
                return;
            }
            // create exception response.
            Response timeoutResponse = new Response(future.getId());
            // set timeout status.
            timeoutResponse.setStatus(future.isSent() ? Response.SERVER_TIMEOUT : Response.CLIENT_TIMEOUT);
            timeoutResponse.setErrorMessage(future.getTimeoutMessage(true));
            // handle response.
            DefaultFuture.received(future.getChannel(), timeoutResponse);

        }
    }
    
    private Object returnFromResponse() throws RemotingException {
        Response res = response;
        if (res == null) {
            throw new IllegalStateException("response cannot be null"); } // Return the result of the call if the response is normalif (res.getStatus() == Response.OK) {
            returnres.getResult(); } // throw a timeout exceptionif(res.getStatus() == Response.CLIENT_TIMEOUT || res.getStatus() == Response.SERVER_TIMEOUT) { throw new TimeoutException(res.getStatus() == Response.SERVER_TIMEOUT, channel, res.getErrorMessage()); Throw new RemotingException(channel, res.geTerrorMessage ()); }Copy the code

The use of a delayed task will make the RPC call complete even when the call times out, rather than stay in! IsDone (), which is probably better.

Three, some understanding of the timeout problem.

  • First, set a proper timeout period based on the service. All services should set the same timeout period.
  • Not all timeout exceptions require retries. In some service scenarios, users should initiate requests manually.
  • The timeout exception does not mean that the server failed (so the appropriate timeout period is important). In this case, you can actively pull information by querying the interface.
  • The front end of timeout should be set up a mask layer (some hands base did not return to the point), otherwise it may cause an avalanche.