This is the fifth day of my participation in the August Wen Challenge.More challenges in August

We’ve already looked at Okhttp interceptors, so I’m going to take a look at the role of those interceptors in the Okhttp system. In chain of responsibility mode, what happens?

Source code analysis of five interceptors

RetryAndFollowInterceptor redirect interceptors

The name suggests that the interceptor’s purpose is to fail to reconnect; For example, if we want to fail to reconnect, we can configure it in OKhttpClient, but it is important to note that not all network requests can be reconnected after a failure, there are certain limits. Therefore, Okhttp checks for network request exceptions and behavior codes internally, and only performs reconnection if appropriate conditions are met. This is why redirection interceptors exist

Let’s take a look at the source code, first of allinterceptmethods

  1. Let’s start with the initialization part
  Request request = chain.request();
    RealInterceptorChain realChain = (RealInterceptorChain) chain;
    Call call = realChain.call();
    EventListener eventListener = realChain.eventListener();

    StreamAllocation streamAllocation = new StreamAllocation(client.connectionPool(),
        createAddress(request.url()), call, eventListener, callStackTrace);
    this.streamAllocation = streamAllocation;
Copy the code
  • This creates a streamAllocation object, which is actually used to establish the components of the network needed to perform the HTTP request

Allocate a stream

  • Note that although the streamAllocation object is already created, it is not used. Instead, it is used by the ConnectInterceptor(more on that later). The Connection to the server Connection and the I/O stream to the server for data transmission are passed to the ConnectInterceptor interceptor through the interceptor chain

It is also worth considering that the entire OkHttp interceptor chain is understood as a recursion and is internally connected through the **RealInterceptorChain()** class, so it is returned to the Repsons only when all interceptors have been executed e; But in the normal development process, the network is not always stable, there will certainly be different degrees of problems, Possible network interruption or failure, so the response returned by the code, it is not normal, 200, has appeared abnormal, then use RetryAndFollowInterceptor interceptors, it is how to intercept?

  1. Take a look at the while loop in the Intercept method
 private static final int MAX_FOLLOW_UPS = 20;
while(true) {...if (++followUpCount > MAX_FOLLOW_UPS) {
        streamAllocation.release();
        throw new ProtocolException("Too many follow-up requests: "+ followUpCount); }}Copy the code
  • There is too much code here and only part of it is shown, all the logic is in this while loop, students can go to see it in detail, I will not say more
  • Let’s see, In other words, if okHTTP releases its straeamAllocation object after 20 retries, it will not request any more. If okHTTP releases straeamAllocation after 20 retries, it will not request any more This is the core flow of the interceptor, and it has a logic to jump out of

RetryAndFollowInterceptor summary

  1. Create a StreamAllocation object that establishes all the network components required for the Okhttp request to allocate the Stream
  2. Call RealInterceptorChain. Proceed () method for the actual network requests
  3. Determine whether to request again based on the exception result or response result
  4. Calls the next interceptor, processes the response, and returns it to the previous interceptor

BridgeInterceptor BridgeInterceptor

For the BridgeInterceptor BridgeInterceptor, it is responsible for setting the content length, encoding, compression, etc., and adding headers

Check out the Intercept method

  1. First, the initialization work
 @Override public Response intercept(Chain chain) throws IOException {
    Request userRequest = chain.request();
    Request.Builder requestBuilder = userRequest.newBuilder();

    RequestBody body = userRequest.body();
    if(body ! =null) {
      MediaType contentType = body.contentType();
      if(contentType ! =null) {
        requestBuilder.header("Content-Type", contentType.toString());
      }

      long contentLength = body.contentLength();
      if(contentLength ! = -1) {
        requestBuilder.header("Content-Length", Long.toString(contentLength));
        requestBuilder.removeHeader("Transfer-Encoding");
      } else {
        requestBuilder.header("Transfer-Encoding"."chunked");
        requestBuilder.removeHeader("Content-Length"); }}if (userRequest.header("Host") = =null) {
      requestBuilder.header("Host", hostHeader(userRequest.url(), false));
    }

    if (userRequest.header("Connection") = =null) {
      requestBuilder.header("Connection"."Keep-Alive");
    }

    // If we add an "Accept-Encoding: gzip" header field we're responsible for also decompressing
    // the transfer stream.
    boolean transparentGzip = false;
    if (userRequest.header("Accept-Encoding") = =null && userRequest.header("Range") = =null) {
      transparentGzip = true;
      requestBuilder.header("Accept-Encoding"."gzip");
    }

    List<Cookie> cookies = cookieJar.loadForRequest(userRequest.url());
    if(! cookies.isEmpty()) { requestBuilder.header("Cookie", cookieHeader(cookies));
    }

    if (userRequest.header("User-Agent") = =null) {
      requestBuilder.header("User-Agent", Version.userAgent()); }... }Copy the code
  • The main purpose of these lines of code is to add a lot of header information to an ordinary request and make it a request that can send network requests
  • ContentType: content-Type,Content-Length,Transfer-Encoding,Host,Connection, etc Keep-alive (to Keep the connection Alive for a certain amount of time), all this code does is initialize the work and add the header information
  1. Transform the available Repsonse
 Response networkResponse = chain.proceed(requestBuilder.build());
 HttpHeaders.receiveHeaders(cookieJar, userRequest.url(), networkResponse.headers());
Copy the code
  • The Proceed method of RealInteceptor is called to send a request to the server, which retrieves the requested action and returns a client response
  • We then call the static method receiveHeaders of HttpHeaders to convert our network requests and the Repsonse returned to us by the server into a Respsonse that the user can use
  1. Take a look at the receiveHeaders implementation
  public static void receiveHeaders(CookieJar cookieJar, HttpUrl url, Headers headers) {
    if (cookieJar == CookieJar.NO_COOKIES) return;

    List<Cookie> cookies = Cookie.parseAll(url, headers);
    if (cookies.isEmpty()) return;

    cookieJar.saveFromResponse(url, cookies);
  }
Copy the code

In fact, the CookieJar is mainly used to store the response message information, among which the HTTP Cookie is used to store some user information

  1. The Content – support gzip Encoding
Response.Builder responseBuilder = networkResponse.newBuilder()
        .request(userRequest);

    if (transparentGzip
        && "gzip".equalsIgnoreCase(networkResponse.header("Content-Encoding"))
        && HttpHeaders.hasBody(networkResponse)) {
      GzipSource responseBody = new GzipSource(networkResponse.body().source());
      Headers strippedHeaders = networkResponse.headers().newBuilder()
          .removeAll("Content-Encoding")
          .removeAll("Content-Length")
          .build();
      responseBuilder.headers(strippedHeaders);
      String contentType = networkResponse.header("Content-Type");
      responseBuilder.body(new RealResponseBody(contentType, -1L, Okio.buffer(responseBody)));
    }
     return responseBuilder.build();
Copy the code
  • It’s time to rebuild the response Repsonse
  • Then, with transparentGzip, the response header Content-Encoding supports the Gzip format and is decompressed into the user-usable response Repsonse, where the GzipSource is decompressed to read the stream data
  • Finally, the response is returned to proceed to the next interceptor

BridgeInterceptor summary

  1. Is responsible for converting a user-constructed Request into a Request that can be accessed on the network
  2. Make a network Request to the Request that matches the network Request
  3. Convert the Response returned from the network request into a Response available to the user (Gzip compressed, Gzip decompressed)

CacheInterceptor a CacheInterceptor

The interceptor’s main job is to retrieve the response from the cache, and if not, to retrieve the response through the network request

The following look at the Cache class source implementation

Before parsing the CacheInterceptor, let’s take a look at how Okhttp uses the cache. It’s actually quite simple. Call the cache class in the cache method, pass in the file directory and size parameters, and then set the desired cache path

 private fun cacheRequest(a) {
        val client = OkHttpClient.Builder()
            .cache(Cache(File("cache"), 24 * 1024 * 1024)).build()
        val request = Request.Builder().url("https://www.baidu.com")
            .get().build()
        
        val call = client.newCall(request)
       kotlin.runCatching {
           val response = call.execute()
           response.close()
       }.onFailure {
           println(it)
       }

    }
Copy the code
  1. The first thing we can see is that there’s an internalCache interface
final InternalCache internalCache = new InternalCache() {
    @Override public Response get(Request request) throws IOException {
      return Cache.this.get(request);
    }

      @Override public CacheRequest put(Response response) throws IOException {
      return Cache.this.put(response); }... };Copy the code

As you can see, all of its methods, whether get or PUT, are implemented through the Cache class

  1. Let’s look at the implementation of the PUT method
  @Nullable CacheRequest put(Response response) {...if (HttpHeaders.hasVaryAll(response)) {
      return null;
    }

    Entry entry = new Entry(response);
    DiskLruCache.Editor editor = null;
    try {
      editor = cache.edit(key(response.request().url()));
      if (editor == null) {
        return null;
      }
      entry.writeTo(editor);
      return new CacheRequestImpl(editor);
    } catch (IOException e) {
      abortQuietly(editor);
      return null; }}Copy the code
  • An Entry instance is created, and the part of the cache (which contains many urls, headers, protocol methods, and so on) is written, already wrapped
  • There is also a very interesting DiskLruCache, see here, the whole okHttpD cache does so much encapsulation, finally handed over to the DiskLruCache cache algorithm to implement. I won’t go into detail here, but it is important to understand that Okhttp maintains a clean thread pool internally to automatically clean and manage the cache
  • After the edit object is defined, a key value is passed through the cache.edit method. This value is to convert the URL of our network into the corresponding key, which has done md5 encryption processing, and then get the md5 hexadecimal representation
  • After all the work is done, we call the entry.writeto () method, which writes our cache to disk

The writeTo method is passed in to the editor, and the key in the editor only holds the URL. If we save the request header and the response header, where do we save the body of the response?

  1. Take a look at the CacheRequestImpl class for the last return
private final class CacheRequestImpl implements CacheRequest {
    private final DiskLruCache.Editor editor;
    private Sink cacheOut;
    private Sink body;
    boolean done;

    CacheRequestImpl(final DiskLruCache.Editor editor) {
      this.editor = editor;
      this.cacheOut = editor.newSink(ENTRY_BODY);
      this.body = newForwardingSink(cacheOut) { .... }; }...Copy the code
  • You can see inside there’s a body, which is our response body, and there’s an editor, which is our body written by DiskLruCache.Editor
  • Here CaheRequestImpl implements a CacheRequest interface, which is exposed to the CacheInterceptor CacheInterceptor. The CacheRqeustImpl implementation class updates and writes cached data directly to the cache

The return value of the passed client.internalCache() method can be seen in internalCache

 InternalCache internalCache(a) {
    returncache ! =null ? cache.internalCache : internalCache;
  }
Copy the code

If the okHttpClient. Builder is created with a Cache or InternalCache parameter, InternalCache and InternalCache will be set to null. If neither parameter is passed, they will be set to null Null, as shown in the following code

  void setInternalCache(@Nullable InternalCache internalCache) {
      this.internalCache = internalCache;
      this.cache = null;
    }

    /** Sets the response cache to be used to read and write cached responses. */
    public Builder cache(@Nullable Cache cache) {
      this.cache = cache;
      this.internalCache = null;
      return this;
    }
Copy the code

Take a look at the CacheInterceptor Intercept method

  1. Getting cache policy
Response cacheCandidate = cache ! =null
        ? cache.get(chain.request())
        : null;

    long now = System.currentTimeMillis();

    CacheStrategy strategy = new CacheStrategy.Factory(now, chain.request(), cacheCandidate).get();
    Request networkRequest = strategy.networkRequest;
    Response cacheResponse = strategy.cacheResponse;

    if(cache ! =null) {
      cache.trackResponse(strategy);
    }
Copy the code
  • If none is returned, null is used to set the CacheStrategy cache. NetworkRequest and cacheResponse objects are the networkRequest and the cacheResponse returned, respectively. The next step is to filter the relevant criteria, which is the job of the cache interceptor
  1. A set of conditional judgment logic
if (networkRequest == null && cacheResponse == null) {
      return new Response.Builder()
          .request(chain.request())
          .protocol(Protocol.HTTP_1_1)
          .code(504)
          .message("Unsatisfiable Request (only-if-cached)")
          .body(Util.EMPTY_RESPONSE)
          .sentRequestAtMillis(-1L)
          .receivedResponseAtMillis(System.currentTimeMillis())
          .build();
    }
Copy the code
  • If networkRequest is null and cacheResponse is null, if neither networkRequest nor cache is available, 504 error Response is returned
 if (networkRequest == null) {
      return cacheResponse.newBuilder()
          .cacheResponse(stripBody(cacheResponse))
          .build();
    }
Copy the code
  • If the network request is null, no network request is allowed, but there is a cache, here the cached response is returned directly
 if(cacheResponse ! =null) {
      if (networkResponse.code() == HTTP_NOT_MODIFIED) {
        Response response = cacheResponse.newBuilder()
            .headers(combine(cacheResponse.headers(), networkResponse.headers()))
            .sentRequestAtMillis(networkResponse.sentRequestAtMillis())
            .receivedResponseAtMillis(networkResponse.receivedResponseAtMillis())
            .cacheResponse(stripBody(cacheResponse))
            .networkResponse(stripBody(networkResponse))
            .build();
        networkResponse.body().close();

        // Update the cache after combining headers but before stripping the
        // Content-Encoding header (as performed by initContentStream()).
        cache.trackConditionalCacheHit();
        cache.update(cacheResponse, response);
        return response;
      } else{ closeQuietly(cacheResponse.body()); }}if(cache ! =null) {
      if (HttpHeaders.hasBody(response) && CacheStrategy.isCacheable(response, networkRequest)) {
        // Offer this request to the cache.
        CacheRequest cacheRequest = cache.put(response);
        return cacheWritingResponse(cacheRequest, response);
      }

      if (HttpMethod.invalidatesCache(networkRequest.method())) {
        try {
          cache.remove(networkRequest);
        } catch (IOException ignored) {
          // The cache cannot be written.}}}Copy the code
  • If networkRequest is not empty, it indicates that network needs to be used, then the remaining interceptors of the interceptor chain will be called to continue processing networkResponse response
  • If the cached response exists and is valid, the cache needs to be updated, whereas if there is no cached response, it can be written directly to the cache

ConnectInterceptor Connection interceptor

The interceptor’s main job is to get a connection to a target request, formally opening the network request

Look directly at its Intercept method

  @Override public Response intercept(Chain chain) throws IOException {
    RealInterceptorChain realChain = (RealInterceptorChain) chain;
    Request request = realChain.request();
    StreamAllocation streamAllocation = realChain.streamAllocation();

    // We need the network to satisfy this request. Possibly for validating a conditional GET.
    booleandoExtensiveHealthChecks = ! request.method().equals("GET");
    HttpCodec httpCodec = streamAllocation.newStream(client, chain, doExtensiveHealthChecks);
    RealConnection connection = streamAllocation.connection();

    return realChain.proceed(request, streamAllocation, httpCodec, connection);
  }
Copy the code
  • As mentioned earlier, the streamAllocation object obtained here is passed in from the redirection interceptor, and then the HTTPCodec and RealConnection objects are created
  • Finally we call the proceed method of the interceptor chain, which we are familiar with before, and at this point the entire network connection interceptor is set up

So the question is what exactly is this HTTPCodec thing? Let’s take a look at the code

HttpCodec

As you can see, it is an abstract implementation of the HTTP protocol. Http1Codec and Http2Codec implement this interface, corresponding to the two versions of HTTP1.1/HTTP2

public interface HttpCodec {
  int DISCARD_STREAM_TIMEOUT_MILLIS = 100; . }Copy the code
  • Java. IO and java.nio are implemented in HttP1Codec code, which encapsulates sockets
  • Mainly used to encode our request and decode our Repsonse

Take a look at the newStream method

The following code creates the RealConnection and HttpCodec objects, and then returns the HttpCodec object with a synchronized block of code

 public HttpCodec newStream(
      OkHttpClient client, Interceptor.Chain chain, boolean doExtensiveHealthChecks) {
    int connectTimeout = chain.connectTimeoutMillis();
    int readTimeout = chain.readTimeoutMillis();
    int writeTimeout = chain.writeTimeoutMillis();
    int pingIntervalMillis = client.pingIntervalMillis();
    boolean connectionRetryEnabled = client.retryOnConnectionFailure();

    try {
      RealConnection resultConnection = findHealthyConnection(connectTimeout, readTimeout,
          writeTimeout, pingIntervalMillis, connectionRetryEnabled, doExtensiveHealthChecks);
      HttpCodec resultCodec = resultConnection.newCodec(client, chain, this);

      synchronized (connectionPool) {
        codec = resultCodec;
        returnresultCodec; }}catch (IOException e) {
      throw newRouteException(e); }}Copy the code

There are basically two things that we’re doing here

  • The findHelathyConnection method is called to generate a RealConnection object for the actual network connection
  • The HttpCodec object is then generated via realConnection and returned in a synchronized code block
  1. Take a look at the findHealthyConnection method
 private RealConnection findHealthyConnection(int connectTimeout, int readTimeout,
      int writeTimeout, int pingIntervalMillis, boolean connectionRetryEnabled,
      boolean doExtensiveHealthChecks) throws IOException {
    while (true) {
      RealConnection candidate = findConnection(connectTimeout, readTimeout, writeTimeout,
          pingIntervalMillis, connectionRetryEnabled);
      synchronized (connectionPool) {
        if (candidate.successCount == 0) {
          returncandidate; }}if(! candidate.isHealthy(doExtensiveHealthChecks)) { noNewStreams();continue;
      }

Copy the code
  • You can see that another findConnection method is called to get the RealConnection
  • As you can see in the synchronization block, if the successCount for this RealConnection is 0, the entire network connection has ended
  • If it is an unhealthy connection (for example, the Socket connection is not closed, the input/output streams are not closed), the noNewStreams() method is called and the findConnection() method is looped through
  1. Take a look at the findConnection() method

Because the findConnection() method is too verbose, only part of the core code is shown here

      releasedConnection = this.connection;
      toClose = releaseIfNoNewStreams();
      if (this.connection ! =null) {
        result = this.connection;
        releasedConnection = null;
      }
Copy the code
  • Try to reuse the connection variable to releaseconnection
  • To determine if the reusable connection is empty, if not, assign the connection to our result

Result is empty if it cannot be reused

 if (result == null) {
        Internal.instance.get(connectionPool, address, this.null);
        if(connection ! =null) {
          foundPooledConnection = true;
          result = connection;
        } else{ selectedRoute = route; }}Copy the code
  • At this point we get an actual RealConnection from the pool and assign it to our result
    result.connect(connectTimeout, readTimeout, writeTimeout, pingIntervalMillis,
        connectionRetryEnabled, call, eventListener);
     Internal.instance.put(connectionPool, result);
Copy the code
  • Once you get the realConnection, you call Connect to make the actual network connection, and then you need to pool it

In simple terms, findConnection() attempts to retrieve a connection, demultiplexing it if it can, or retrieving a new connection from the pool if it can’t, and then adding the new connection to the pool

ConnectInterceptor summary

  1. Get a RealConnection
  2. Select different connection modes
  3. ConnectInterceptor access pass Inteceptor StreamAllocation, StreamAllocation newStream ()
  4. Pass to the interceptor the RealConnection object you just created for network IO and, most critical for interacting with the server, HttpCodec and so on
  5. Finally, call the CallServerInterceptor to complete the okHTTP network request operation

The CallServerInterceptor calls the service interceptor

This interceptor is mainly responsible for making a real network request to our server, receiving a read response from the server, and finally returning it

Intercept () method

  1. Five objects are defined
    RealInterceptorChain realChain = (RealInterceptorChain) chain;
    HttpCodec httpCodec = realChain.httpStream();
    StreamAllocation streamAllocation = realChain.streamAllocation();
    RealConnection connection = (RealConnection) realChain.connection();
    Request request = realChain.request();
Copy the code
  • RealInterceptorChain: a chain of interceptors through which all interceptors are linked together to complete the function of responding
  • HttpCodec: includes Htpp1.1 and Http1.2 protocols, encoding request and decoding response
  • StreamAllocation: Establishes all network components required for HTTP requests and allocates streams
  • RealConnection: The connection between client and server has been abstracted into connection, this is the concrete implementation of abstract connection
  • Request: indicates a network Request
  1. The core part of the
    realChain.eventListener().requestHeadersStart(realChain.call());
    httpCodec.writeRequestHeaders(request);
    realChain.eventListener().requestHeadersEnd(realChain.call(), request);
Copy the code
  • Here httpCodec calls the writeRequestHeaders() method and passes in our network request to write the request header to the socket
  1. Handling special cases
if ("100-continue".equalsIgnoreCase(request.header("Expect"))) {
        httpCodec.flushRequest();
        realChain.eventListener().responseHeadersStart(realChain.call());
        responseBuilder = httpCodec.readResponseHeaders(true);
      }
Copy the code
  • If we can add Expect and 100-continue to the request header, we will be returned with a response code of 100. The client will continue to send the request, skip writing the body, and directly get the response information
  1. I’ll write the body

In normal cases, the following procedure is performed if the preceding conditions are not met

request.body().writeTo(bufferedRequestBody); . httpCodec.finishRequest();Copy the code
  • Call the body method writeTo() to write the body information into the socket
  • The httpCodec finishRequest() method call indicates that the HTTP request has been written
  1. Request to read the corresponding work
if (responseBuilder == null) {
      realChain.eventListener().responseHeadersStart(realChain.call());
      responseBuilder = httpCodec.readResponseHeaders(false); }...if (forWebSocket && code == 101) {
      response = response.newBuilder()
          .body(Util.EMPTY_RESPONSE)
          .build();
    } else {
      // The builder pattern creates a response
      response = response.newBuilder()
          .body(httpCodec.openResponseBody(response))
          .build();
    }  

      streamAllocation.noNewStreams();
Copy the code
  • Call the readResponseHeaders method to read the HTTP request header information
  • Calling noNewStreams() disables the creation of new streams, closes IO streams, and closes Connection

Finally check if the code response code is 204 or 205, and if it is, an exception will be thrown

if ((code == 204 || code == 205) && response.body().contentLength() > 0) {
      throw new ProtocolException(
          "HTTP " + code + " had non-zero Content-Length: " + response.body().contentLength());
    }
Copy the code

At this point, we are done getting the entire Repsonse

Stern said

Okhttp source code parsing is now complete, a brief overview of the entire OKHTTP network request process

  1. The Call object encapsulates the request
  2. Dispatcher distribution of requests
  3. GetResponseWithInterceptors () method, the use of the interceptor chain, five of the interceptor chain calls, progressive transformation
  • RetryAndFollowInterceptor
  • BridgeInterceptor
  • CacheInterceptor
  • ConnectInterceptor
  • CallServerInterceptor

There’s a lot more to okHTTP than that, but I’m going to start looking at Retrofit, which is a web framework, and how does it differ from OkHTTP? Keep learning ing….