Content of this article:

  1. OkHttp Process Guide
  2. The role of each interceptor
  3. Https Dns implementation
  4. Http1.1 and HTTP2 implementation
  5. The function and implementation of connection pool

OkHttp process

The class diagram

The class diagram here only shows the class diagram in the chain of Responsibility pattern process; the more important network UML diagram is provided in the ConnectInterceptor below

The flow chart

This flow chart only involves a simple process and does not involve specific details. What stands out is the chain of responsibility mode. This mode is more suitable for stratification, but there is a dependency between the top and the bottom. Object initialization (recommended here to see C++), Input event transmission and processing, five layer network protocol, basically are passed from the outside in and processed from the inside out

Initialization of OkHttp

Let’s take a look at the use of OkHttp

URL url = new URL("http://www.baidu.com/");
OkHttpClient client = new OkHttpClient.Builder()
            .build();
Request request = new Request.Builder()
                .url(url.toString())
                .build();
okhttp3.Call call = client.newCall(request);
okhttp3.Response response = call.execute();
Copy the code
  1. First, in the constructor of okHttpClient.Builder, a number of defaults are initialized, and if none can be found, most can be retrieved from here
  2. In OkHttpClient, SSLSocketFactory is also created to support TLS or SSL

RealCall initialization

Using okHttpClient.newCall () creates RealCall and creates another important Transmitter class, which, according to the official note, connects OKHTTP to the network layer (actually the HTTP layer)

RealCall.execute()

  1. callDispatcher.executed()The call is added to the call queue for execution. The main purpose of this step is to control the start, end, and cancellation of the call, but since it is a synchronous request, it is already started
  2. callgetResponseWithInterceptorChain()Start sending a request to the server and get a response

RealCall.getResponseWithInterceptorChain()

  1. Add a custom interceptor, an interceptor that does something extra before sending and after receiving, such as Logging, but I think it’s a good idea to use EventListener, which can also be added to each request. But then the code repeats a lot
  2. In turn, add RetryAndFollowUpInterceptor BridgeInterceptor, CacheInterceptor ConnectInterceptor, Custom networkInterceptor,
  3. Create a RealInterceptorChain object and call it laterRealInterceptorChain.proceed()Method, calling each interceptor processing logic in turn

RealInterceptorChain.proceed()

  1. Check whether the custom interceptor is valid
  2. Create a new RealInterceptorChain, note index+1
  3. Gets the current index interceptor and calls itinterceptor.intercept()Method to the Interceptor, get the return value, why not use a loop
  4. All custom interceptors are required to be calledRealInterceptorChain.proceed()Otherwise, the request cannot be sent
  5. Check whether Response or response.body() is empty

The role of interceptors

HTTP is in the application layer, which means the HTTP protocol is implemented by the application, and finally transmitted through the socket to the TCP transport layer. Let’s first look at how to send HTTP requests without using the OKHTTP framework

String path = "http://www.baidu.com/";
String host = "www.baidu.com";
Socket socket = null;
OutputStreamWriter streamWriter = null;
BufferedWriter bufferedWriter = null;
try {
  socket = new Socket(host,80);
  streamWriter = new OutputStreamWriter(socket.getOutputStream());
  bufferedWriter = new BufferedWriter(streamWriter);
  bufferedWriter.write("GET " + path + "HTTP / 1.1 \ r \ n");
  bufferedWriter.write("Host: www.baidu.com\r\n");
  bufferedWriter.write("\r\n");
  bufferedWriter.flush();
  BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream(),"UTF-8"));
  int d = -1 ;
  while((d = in.read()) ! = -1){
    System.out.print((char)d); }}catch (IOException e) {
  e.printStackTrace();
}
Copy the code
  1. The first step is to establish a socket connection, which is equivalent to establishing a three-way handshake connection
  2. Sending an HTTP request
  3. Obtain the HTTP result

Okhttp, on the other hand, implements THE HTTP protocol in the interceptor. Each part of the interceptor implements the corresponding part of the HTTP, so as to achieve full decoupling

RetryAndFollowUpInterceptor

From the name, it’s easy to see that this is a retry and continue interceptor. Why resend the request

  1. Routing failed, just to explain a little bit about routing is that a Web service may have a proxy or multiple IP addresses, and these can form a so-called routing, as long as we can connect to any one of these routes to communicate with the server, so if the routing fails, we can choose other routes to connect
  2. After obtaining the redirection request of 3XX, the server or proxy requests authentication information

Here’s a look at the Intercept () process:

  1. Set up an endless loop of repeated resend requests for the above two possible scenarios
  2. callTransmitter.prepareToConnect()Create prepared objects such as ExchangeFinder, Address, RouteSelector
  3. Next callRealInterceptorChain.proceed()The next Interceptor continues processing and gets its returned response
  4. Handle routing and I/O exceptions. The exceptions listed here cannot be retried. For details, seerecover()methods
  5. callfollowUpRequest()Follow-up processing, which according to the server return code for the corresponding processing, specific to look at the HTTP protocol return code corresponding meaning
  6. Determine whether to resend the request or return a response based on followUp’s results

BridgeInterceptor

This interceptor is relatively simple, it is to repackage the Request set by the user and turn it into a real Http Request. Frankly speaking, it is to supplement the header in the Http Request. If the user has set it, there is no need to set it. Save the cookie and extract the ResponseBody

  1. Adds headers in Http requests
  2. callRealInterceptorChain.proceed()The next Interceptor continues processing and gets the response returned
  3. Parse the Response returned, save the cookie, and extract the ResponseBody

CacheInterceptor

This Interceptor implements if-modified-since requests. For details, see HTTP

Determines whether the data cached by the client has expired and retrieves new data if it has

This field is typically used when the server needs to return large files to speed up HTTP transfers, so take a look at the intercept() process

  1. Gets the current Cache policy, there are also some things can not understand, should be the cache-control to the understanding of the field
  2. Get the cached Request and Response, where the Request is usually modified by userRequest
  3. If networkRequest is null, return the cached Response. When Request is null, the cache-control field is only-if-cached, or userRequest is null. Can you cache server information in this way when there is no network?
  4. callRealInterceptorChain.proceed()The next Interceptor continues processing and gets the response returned
  5. If the code returned is HTTP_NOT_MODIFIED, the cached Response is returned and updated, with the main expiration date updated, corresponding to the date of the if-Modified-since field
  6. Otherwise, the cache is invalid. Use the Response returned by the server and update the cache. Before updating the cache, check whether the corresponding cache policy supports the cache

ConnectInterceptor

This is the core code of OKHTTP, but some of the concepts are not clear and the logic is complicated. But there are three core constants:

  1. DNS request to convert hostname to IP address
  2. Set up the socket and shake hands three times
  3. TLS handshake

But there are two more theoretical questions,

  1. How do I confirm whether TO use HTTP1.1 or http2
  2. How do I confirm whether TO use HTTP or HTTPS

Here’s an explanation:

  1. Okhttp currently supports http1.1 and http2, which are two protocols. Http2 requires the server to support TLS1.2. If the scheme part of the URL is not an HTTPS string, http2 is not supported. See protocol.java for details. If TLS1.2 is supported, check whether the server and client support HTTP2 using TLS1.2
  2. It depends on whether the string in Scheme is HTTP or HTTPS. If it is HTTPS, the TLS handshake will be attempted with the server

Next take a look at the intercept() process

  1. Create a new Exchange object that acts as a bridge between I/O and socket connection interaction
  2. callRealInterceptorChain.proceed()The next Interceptor continues processing and returns the corresponding Response directly

From the above, the final step is to create the Exchange object

UML diagrams

Explain a little bit about what classes mean and what they do

  1. Transmitter: Bridging the OKHTTP application layer and the network layer, which in my view is more like the interface that controls socket startup and cancellation
  2. Exchange: A storage class for objects that interact with the I/O layer
  3. ExchangeFinder: objects used to find and create a RealConnection
  4. ExchangeCodec: Interface for sending I/O requests
  5. RouteSelctor: Class that creates Rout and Proxy and is responsible for DNS resolution
  6. Selection: A subclass of RouteSelector that stores a wrapper class for a Route
  7. Http1ExchangeCodec: The class Http1.1 uses to interact with I/O. Codec stands for Code and Decode
  8. Http2ExchangeCodec: Class that Http2 uses to interact with I/O
  9. Connection: indicates the interface used for socket and TLS connections
  10. Address: stores the properties needed for the connection
  11. Route: Route, which is the wrapper class for the proxy and Address
  12. RealConnection: Used for socket connection and TLS handshake

Sequence diagram

Transmitter.newExchange()

The method logic is simple, just call ExchangeFinder.find() to create an ExchangeCodec object, which is then used as an argument to create an Exchange object

ExchangeFinder.find()

Did two things:

  1. callfindHealthyConnection()Get RealConnection
  2. callRealConnection.newCodec()Create an ExchangeCodec object Http2ExchangeCodec or Http1ExchangeCodec

ExchangeFinder.findHealthyConnection()

  1. Create an infinite loop. Here the infinite loop is for health. Why not create an infinite loop?
  2. callfindConnection()Find or create a RealConnection
  3. If it is a new RealConnection, return it directly
  4. Healthy is defined as whether the socket is closed or released if it is not a HealthyConnection

This is because if not a HealthConnection, the next Connection will be found. If all connections are not Healthy, a new Connection will be created. This Connection must be Healthy

ExchangeFinder.findConnection()

  1. Attempts to obtain the connection directly may fail if the connection is null or the Exchange cannot be created
  2. callRealConnectionPool.transmitterAcquirePooledConnection()Try to get a matching case Connection from RealConnectionPoolisEligible():
    1. If the Connection has been equipped with Transmitter or cannot create Exchange, it does not match
    2. Is it the same Address? If yes, a match is found
  3. Select a Route based on the conditions to prepare for the creation of a new Connection
  4. Second callRealConnectionPool.transmitterAcquirePooledConnection()Try to get a matching case Connection from RealConnectionPool, this time on Routes obtained after DNS
  5. If not, create a new RealConnection
  6. callRealConnection.connect()Method for socket and TLS connection
  7. The third timeRealConnectionPool.transmitterAcquirePooledConnection()Try to get match from RealConnectionPool Connection, matching conditions are requireMultiplexed = true
  8. If not, call to update the connectionPool

The first Connection is an http1.1 or http2 Connection, the second and third Connection is an http2 multiplexed Connection, the second Connection is an http2 speed. Select * from http2; select * from http2; select * from http2

RealConnection.connect()

  1. Check the validity of the Connection
    1. If HTTPS is not supported, the ConnectionSpec type must contain CLEARTEXT(plain text request, for example: HTTP, FTP, etc.), which is included by default
    2. Determine whether CLEARTEXT type requests are supported based on the platform,
  2. Determines whether an HTTP proxy exists and invokes it if soconnectTunnel()The connection,The principle of the tunnel is not understood, do not understand
  3. Otherwise the callconnectSocket()Create a rawSocket and do a three-way handshake
  4. callestablishProtocol()Identify the agreement

RealConnection.connectSocket()

  1. If the proxy is HTTP or has no proxy, create a Socket from the SocketFactory and call the platform’sconnectSocket()AndroidPlatform has no special handling and calls directlysocket.connect()To connect
  2. Create Source and Sink for InputStream(get HTTP response) and OutputStream(send request), respectively

RealConnection.establishProtocol()

  1. SslSocketFactory: SslSocketFactory: SslSocketFactory: SslSocketFactory: SslSocketFactory: SslSocketFactory: SslSocketFactory: SslSocketFactory: SslSocketFactory: SslSocketFactory: SslSocketFactory: SslSocketFactory: SslSocketFactorystartHttp2()
  2. callconnectTls()TLS handshake
  3. If the HTTP2 protocol is supportedstartHttp2()What is the purpose of establishing HTTP2 PREFACE connection? Sync with streamId? Do not know much about

RealConnection.connectTls()

  1. Creating SSLSocket, which is implemented in the JDK used here, is not particularly clear, perhaps using the OpenSSL library or some other library
  2. Get ConnectionSpec, which seems to be primarily supported by encryption algorithms. The default is connectionSpec.modern_TLS
  3. If the TLS extension is supported, configure the TLS extension information
  4. SSLSession is the third step in the TLS handshake, and the client validates the certificate
  5. After the handshake is successful, try to obtain the corresponding HTTP protocol from TLS. ALPN is based on TLS to obtain whether the client and server support HTTP2

CallServerInterceptor

The Interceptor’s function is to output fields to the IO stream

  1. For Http1.1, input to the string stream
  2. For Http2, input to binary frames, HEADER frames and DATA frames

Next, take a look at the interceptor() logic

  1. This step is mainly to create the request header into the corresponding flow, HTTP1.1 and HTTP2 have different implementations
  2. Whether it is a GET request or a POST request (PUT and DELETE requests are not involved for the moment) is mainly judged by whether there is a RequestBody. If there is a RequestBody, write it to sink, where a RealConnection is created
  3. Send HTTP requests for HTTP1.1 callssink.flush()For HTTP2, the final callFramingSink.close()Determine if there are DATA frames in close and send DATA and HEADER frames
  4. Read request headerexchange.readResponseHeaders()And the request bodyexchange.readResponseHeaders

Okhttp connection pool

The core function of connection pooling is the implementation of the KEEP-alive field in the HTTP1.1 protocol, in order to reduce the creation of sockets to reduce HTTP latency. Above analysis ExchangeFinder. FindConnection () has been done for the storage and reuse of the connection pool was explained, the main complement it

  1. Creation of a connection pool
  2. The relationship between Call and Connection
  3. The connection pool expires

Creation of a connection pool

The default is to create a connection pool with a maximum of five connections, each for a maximum of five minutes, using delegate mode, and finally creating a RealConnectionPool object

The relationship between Call and Connection

A Request corresponds to a Call, and a Call corresponds to a Connection in the Transmitter. Therefore, if the Connection in the Transmitter can be reused, However, a Connection can correspond to multiple transmitters and Calls, indicating that the same request calls Call.execute() for multiple times

The connection pool expires

Let’s take a look at the logic of updating realConnection.put () for the connection pool

  1. Check to see if cleanup is in progress. If not, start a cleanup thread
  2. Add Connection to the ArrayDeque

Next, look at the logic for cleaning up Runnable

Runnable.run()

  1. Create an endless loop
  2. callcleanup()Clean up Connection and get the return value,Only clean the one that’s been out of date the longest
  3. There are three return values, each with different meanings
    1. If -1 is returned, the ConnectionPool is empty and the cleanup thread is terminated
    2. A return of 0 indicates that a Connection has been cleared, and the thread continues to be cleared
    3. If the value is greater than 0, a Connection has been suspended, but the time has not reached the corresponding time. In this case, sleep the returned time and try to clear the Connection again

RealConnectionPool.cleanup()

  1. All connections in the queue are traversed and called one by onepruneAndGetAllocationCount()To obtain the Idle Connection
  2. Through to thelongestIdleDurationNsFind the one with the longest Idle time
  3. If the longest Idle time is greater than the lifetime of the ConnectionkeepAliveDurationNsOr the number of Idle connections in the queue is greater than the maximum value, the Connection with the maximum Idle time is released and 0 is returned
  4. If there are Idle connections, the hibernation time is returned. During this time, no Idle Connection is likely to exceed the lifetime of the Connection
  5. If no Idle Connection exists and a Connection is in use, the Connection lifetime is returned
  6. If ConnectionPool is empty, -1 is returned, terminating the cleanup thread

RealConnectionPool.pruneAndGetAllocationCount()

  1. There is a hidden judgment here, which is whenConnection.transmittersIf the number is 0, 0 is returned. Therefore, 0 is returned for normal Idle connections
  2. The main function of the TransmitterReference is to find the memory leak if there is any TransmitterReference.
    1. If there is a Transmitter in use, skip it
    2. If there is no Transmitter Null, there is no TransmitterReferenceresponse.body.close()To print Log information and remove Reference
    3. If you end up in a loopConnection.transmittersIf empty, the expiration time is set to the maximum life time and 0 is returned

The connection pool clean expired or interesting, especially pruneAndGetAllocationCount () me up half a day

Problems to be solved

  1. It looks like OKHTTP will always reuse Connection, even if the Connection field of the Header in the Request is not keep-alive
  2. Principle of tunnel
  3. The implementation of Http2, especially OKHTTP, merges multiple requests and does not see how it is implemented or used