“This article has participated in the call for good writing activities, click to view: the back end, the big front end double track submission, 20,000 yuan prize pool waiting for you to challenge!”
As a network request framework, OkHttp is self-evident. The advantage of studying OkHttp is to instantiate and abstract the basic network knowledge such as TCP, HTTP and HTTPS into images.
After reading this article, you will know:
- OkHttp’s overall request structure;
- Implementation details and responsibilities of each interceptor in responsibility chain mode;
- How do I find a usable and healthy connection? Namely connection pool reuse;
- How do I find codecs for Http1 and Http2?
- The difference between NetworkInterceptor and ApplicationInterceptor?
- How to establish a TCP/TLS connection?
This article source for OKHTTP :4.9.1 version, the article does not post a large number of source code, combined with the source code to read the best.
OkHttp overall structure
The use of OkHttp is not the main content of this article; it is simply an entry point for interpreting the source code.
val okHttpClient = OkHttpClient()
val request: Request = Request.Builder()
.url("https://cn.bing.com/")
.build()
okHttpClient.newCall(request).enqueue(object :Callback{
override fun onFailure(call: Call, e: IOException) {
}
override fun onResponse(call: Call, response: Response){}})Copy the codeOkHttp is very simple to use. First create OkHttpClient and Request object, then create a RealCall object from Request, and use it to perform asynchronous enqueue or synchronous execute to send requests. And listen for feedback Callback if the request fails or succeeds.
There are three main classes that need to be explained: OkHttpClient, Request, and RealCall.
- OkHttpClient: equivalent to a configuration center that can be used to send HTTP requests and read their responses. There are many configurations, such as connectTimeout, readTimeout, Dispatcher, and so on. There are other configurations available to view the source code.
- Request: a set of network Request Url, Request method (GET, POST……) , request header, request body request class.
- RealCall: RealCall is returned by the newCall(Request) method. RealCall is one of the core classes for OkHttp to execute requests. It is used as a link between OkHttp’s application layer and network layer, that is, combining OkHttpClient and Request to initiate asynchronous and synchronous requests.
As you can see from the above steps, OkHttp finally executes okHttpClient.newCall(request). Enqueue, also known as the enqueue method of RealCall, which is an asynchronous request. A synchronous request realcall.execute () can also be performed.
RealCall synchronous request finally actually invokes RealCall. GetResponseWithInterceptorChain (), and RealCall asynchronous request is to use a thread pool to request placed onto a background processing, But in the end, or would you call RealCall. GetResponseWithInterceptorChain () to obtain the return value of a network request Response. Basic can smell from here at the core of the network request and actually getResponseWithInterceptorChain () method, the exactly how to connect to the server network request? I’ll leave that for now and get into more details later.
Let’s start with the asynchronous request EnQueue and look at the main structure of the asynchronous request.
Dispatcher private fun promoteAndExecute():Boolean{... val executableCalls = mutableListOf<AsyncCall>()synchronized(this) {
val i = readyAsyncCalls.iterator()
while (i.hasNext()) {
val asyncCall = i.next()
if (runningAsyncCalls.size >= this.maxRequests) break // Max capacity.
if (asyncCall.callsPerHost.get() >= this.maxRequestsPerHost) continue // Host max capacity.
i.remove()
asyncCall.callsPerHost.incrementAndGet()
executableCalls.add(asyncCall)
runningAsyncCalls.add(asyncCall)
}
isRunning = runningCallsCount() > 0
}
for (i in 0 until executableCalls.size) {
val asyncCall = executableCalls[i]
asyncCall.executeOn(executorService)
}
return isRunning
}
Copy the codeAn asynchronous request first adds AsyncCall to a two-way queue called readyAsyncCalls (a queue ready to execute but not yet executed) to prepare the request. ExecutableCalls and runningAsyncCalls are the executableCalls and runningAsyncCalls queue. There are two main filtering criteria: readyAsyncCalls
-
if (runningAsyncCalls.size >= this.maxRequests) break: The number of concurrent requests must be smaller than the maximum number of requests 64. -
if (asyncCall.callsPerHost.get() >= this.maxRequestsPerHost) continue: The maximum number of concurrent requests for a host cannot exceed 5
In other words, when our number of concurrent requests exceeds 64 or the number of requests from a host exceeds 5, the exceeded requests will not be executed and will have to wait to be added to the execution queue.
The valid requests are screened and saved, and the requests are immediately traversed, performing Runnable tasks with the ExecutorService in the Dispatcher, and then added to the thread pool to execute the valid network requests.
Class: realCall. AsyncCall Override funrun() {
threadName("OkHttp ${redactedUrl()}"){...try {
val response = getResponseWithInterceptorChain()
signalledCallback = true
responseCallback.onResponse(this@RealCall, response)
} catch (e: IOException) {
...
responseCallback.onFailure(this@RealCall, e)
}
}
}
Copy the codeThe code above is the request of the tasks in the thread pool, you can see in the val a try-catch block, the response = getResponseWithInterceptorChain () to get results resonse network request, Return a response or error to the user via callback. This callback is the same callback that was registered to listen on when OkHttp was first used.
By the way, is this method familiar? Because in the above mentioned three main core classes, RealCall synchronous or asynchronous request, finally will all come to getResponseWithInterceptorChain () this step.
Results the response is through the network request getResponseWithInterceptorChain () method returns, the results of how to get the network request? How do you interact with the server? Let’s take a look at the internal structure of this method.
Interceptor internal implementation
OkHttp structure analysis to know from the above that all details are encapsulated in network request getResponseWithInterceptorChain () is the core method. So let’s look at the implementation.
Class: RealCall internal fun getResponseWithInterceptorChain () : the Response {// Build a full stack of interceptors.
val interceptors = mutableListOf<Interceptor>()
interceptors += client.interceptors
interceptors += RetryAndFollowUpInterceptor(client)
interceptors += BridgeInterceptor(client.cookieJar)
interceptors += CacheInterceptor(client.cache)
interceptors += ConnectInterceptor
if(! forWebSocket) { interceptors += client.networkInterceptors } interceptors += CallServerInterceptor(forWebSocket) val chain = RealInterceptorChain( call =this,
interceptors = interceptors,
index = 0,
exchange = null,
request = originalRequest,
connectTimeoutMillis = client.connectTimeoutMillis,
readTimeoutMillis = client.readTimeoutMillis,
writeTimeoutMillis = client.writeTimeoutMillis
)
...
try {
val response = chain.proceed(originalRequest)
...
return response
}
...
}
Copy the codeGetResponseWithInterceptorChain () the internal implementation is done through a chain of responsibility pattern, will be in each stage of the packaging to all network request chain (i.e., each blocker Interceptor), configured each Interceptor and put it in a List, Then, as an argument, create a RealInterceptorChain object and call chain.proceed(request) to initiate the request and get the response.
In each interceptor, some preparatory action is done, such as determining whether the request is available, converting the request to a format that the server parses, etc., and then chain.proceed(request) is performed on the request. Mentioned above getResponseWithInterceptorChain () the internal implementation is of a chain of responsibility pattern, and chain. Proceed (request) role is the key of the chain of responsibility pattern, transferred the request to the next interceptor.
There are seven types of interceptors in OkHttp, including custom interceptors. Here, the details of the network request are encapsulated in each interceptor, and each interceptor has its own responsibility. Once each interceptor is studied, the entire network request is understood. Here’s a look at the responsibilities of each interceptor.
Seven interceptor responsibilities
1. User-defined interceptors
User-defined interceptors are before all other interceptors, developers can customize network interceptors based on business needs, for example we often customize Token processing interceptors, log printing interceptors, etc.
2, RetryAndFollowUpInterceptor
RetryAndFollowUpInterceptor request failure and redirect is a retry interceptors. Its internal opened a request cycle, each loop will make a ready for action (call. EnterNetworkInterceptorExchange (request, newExchangeFinder)), The primary purpose of this preparation action is to create an ExchangeFinder, find available Tcl or Tsl connections for the request and set parameters associated with the connection, such as the connection codec. ExchangeFinder will explain this later when you connect to the network.
With that done, a network request (Response = realchain.proceed (request)) is initiated, and the purpose of this line of code is to pass the request to the next interceptor. At the same time, it determines whether the current request is faulty and needs to be redirected. If there is an error or a redirect is required, the cycle starts again until there are no errors or redirects required.
Here is a brief description of the criteria for error and redirection:
- Error detection criteria: The try-catch block is used to catch exceptions for the request, which will catch RouteException and IOException, and determine whether the current request can be retried after an error.
- Redirection criteria: Check the status Code of Response to determine whether redirection is required. If the status Code is 3XX, it indicates that redirection is required. Then create a new Request and retry.
3, BridgeInterceptor
BridgeInterceptor is an interceptor that connects application code to network code. That is, the interceptor prepares the user for some configuration required by the server request. If the definition is too abstract, let’s look at what the server header of a request Url looks like.
URL: wanandroid.com/wxarticle/c… Methods: the GET
The corresponding request header is as follows:
GET /wxarticle/ Chapters /json HTTP/1.1 Host: wanandroid.com Accept: Application /json, text/plain, / accept-encoding: gzip, deflate, br Accept-Language: zh-CN,zh; Q =0.9 Connection: keep-alive user-agent: Mozilla/5.0 XXX……
You may ask, what does the BridgeInterceptor interceptor have to do with this? BridgeInterceptor is used to process network requests for users. It helps users fill in configuration information required for server requests, such as user-Agent, Connection, Host, and Accept-Encoding, as shown above. The result of the request is also processed accordingly.
The internal implementation of BridgeInterceptor consists of three main steps:
-
Set content-Type, Content-Length, Host, Connection, Cookie and other parameters for user network requests, that is, convert general requests into formats suitable for server parsing to adapt to the server side;
-
With the chain.proceed (RequestBuilder.build ()) method, the converted request is handed over to the next CacheInterceptor and a Response is returned;
-
Gzip and Content-Type transformation is also performed on the Response result to adapt to the application side.
So the BridgeInterceptor is a bridge between the application and the server.
4, CacheInterceptor
CacheInterceptor is an interceptor that handles the network request cache. Its internal handling is similar to the logic of some image caches in that it first determines if a cache is available, if so, it returns to the cache, otherwise, the chain.proceed(networkRequest) method is called to hand over the request to the next interceptor, and when it has the result, it is put to the cache.
5, ConnectInterceptor
ConnectInterceptor is an interceptor that establishes a connection to request.
internal fun initExchange(chain: RealInterceptorChain): Exchange {
...
val exchangeFinder = this.exchangeFinder!!
val codec = exchangeFinder.find(client, chain)
val result = Exchange(this, eventListener, exchangeFinder, codec)
this.interceptorScopedExchange = result
this.exchange = result
...
if (canceled) throw IOException("Canceled")
return result
}
Copy the codeAs you can see from its source code, it first queries to codec through ExchangeFinder. Is ExchangeFinder familiar? In RetryAndFollowUpInterceptor analysis above, each loop will do create ExchangeFinder preparation first.
And what is this codec? It is a codec that determines whether to request Http1 or Http2.
After a suitable CODEC is found, an Exchange is created as a parameter. There are a lot of network connectivity implementations involved in Exchange. More on this later, but how do you find the appropriate CODEC?
How do I find available connections?
To find a suitable coDEC, you must first find an available network connection, and then use that available connection to create a new coDEC. In order to find available connections, there are about five ways to filter internally.
The first is to look it up from the connection pool
if (connectionPool.callAcquirePooledConnection(address, call, null.false)) {
val result = call.connection!!
return result
}
Copy the codeAn attempt is made to find available connections in the connection pool. When traversing connections in the connection pool, each connection is judged to be available under the following conditions:
- The number of requests should be less than the maximum number of requests the connection can handle, up to Http2, the maximum number of requests is 1, and new exchanges can be created on the connection;
- The connected host is the same as the requested host;
If a qualified connection is retrieved from the pool, it is returned directly.
If not, use the second method to get the available connection.
The second way is to pass in a Route and look it up from the connection pool
if (connectionPool.callAcquirePooledConnection(address, call, routes, false)) {
val result = call.connection!!
return result
}
Copy the codeThe second option is again taken from the connection pool, but this time it passes routes, which is a List containing the Route, which actually refers to the IP address of the connection, the TCP port, and the proxy mode.
This time from the connection pool, primarily for Http2, routes must share an IP address, the server certificate for this connection must contain the new host and the certificate must match the host.
Third: create your own connection
If you don’t get a connection available from the pool the first two times, create your own connection.
val newConnection = RealConnection(connectionPool, route)
call.connectionToCancel = newConnection
try {
newConnection.connect(
connectTimeout,
readTimeout,
writeTimeout,
pingIntervalMillis,
connectionRetryEnabled,
call,
eventListener
)
}
Copy the codeHow to create a connection between TCP/TLS connections?
Once you have created your own connection, you do another lookup from the connection pool.
Fourth: multiplexing set to true, still look from the connection pool
if (connectionPool.callAcquirePooledConnection(address, call, routes, true)) {
val result = call.connection!!
newConnection.socket().closeQuietly()
return result
}
Copy the codeRequireMultiplexed is set to true to only find connections that support multiplexing. And after the connection is established, the new connection is saved to the connection pool.
How do I find codecs for Http1 and Http2?
Having analyzed several ways to find available and healthy connections, the creation of coDEC requires a distinction between Http1 and Http2 based on these connections. If http2Connection is not null, Http2ExchangeCodec is created, and Http1ExchangeCodec is created instead.
Once we find the codec, we go back to the beginning of the ConnectInterceptor and create an Exchange using the codec. Inside the Exchange is either an Http1 decoder or an Http2 decoder. Write the Request header writeRequestHeaders or create the Request Body and send it to the server.
After Exchange successfully initializes, the request is handed over to the next interceptor, CallServerInterceptor.
6, CallServerInterceptor
The CallServerInterceptor is the last interceptor in the chain. It is mainly used to send content to the server, mainly transmitting HTTP header and body information.
Internally, it uses the Exchange created above to write the Request header, create the Request body, send the Request, get the result, parse the result and send back.
7, NetworkInterceptor
NetworkInterceptor is also a user-defined interceptor. It comes after the ConnectInterceptor and before the CallServerInterceptor. We know that the first interceptor was user-defined, how is that different from this one?
NetworkInterceptor already exist for the use of multiple interceptors, in the request arrived at the interceptor, the request information is quite complicated, including RetryAndFollowUpInterceptor retry interceptors, through analysis to know, every time try again once, All subsequent interceptors will also be called once, resulting in networkInterceptor being called multiple times, while the first custom interceptor will only be called once. When we need to customize interceptors, such as token and log, we usually use the first one for resource consumption.
So far, all seven interceptors have been analyzed. An analysis of the ConnectInterceptor raises a question: How is a TCP/TLS connection implemented?
How to establish a TCP/TLS connection?
A TCP connection
fun connect(
connectTimeout: Int,
readTimeout: Int,
writeTimeout: Int,
pingIntervalMillis: Int,
connectionRetryEnabled: Boolean,
call: Call,
eventListener: EventListener
){...while (true) {
try {
if (route.requiresTunnel()) {
connectTunnel(connectTimeout, readTimeout, writeTimeout, call, eventListener)
if (rawSocket == null) {
// We were unable to connect the tunnel but properly closed down our resources.
break}}else {
connectSocket(connectTimeout, readTimeout, call, eventListener)
}
establishProtocol(connectionSpecSelector, pingIntervalMillis, call, eventListener)
eventListener.connectEnd(call, route.socketAddress, route.proxy, protocol)
break
} catch (e: IOException) {
...
}
Copy the code- Open a while loop inside connect, and you can see that the first step is a route.requirestunnel () determination. The requiresTunnel() method indicates whether the request uses a proxy.type. HTTP Proxy and targets an Https connection.
- If so, create a proxy Tunnel (connectTunnel). The purpose of this tunnel is to use Http to proxy requests for Https;
- If not, a TCP connection is directly established.
- Establish a request protocol.
How are proxy tunnels created? The Http Proxy creates a TLS request internally, that is, adding Host, proxy-Connection, and user-agent headers to the address URL. Then up to 21 attempts are made to open a TCP connection using connectSocket and to create a proxy tunnel using TLS requests.
If a proxy tunnel is not required, a TCP connection will be established.
private fun connectSocket(connectTimeout: Int, readTimeout: Int, call: Call, eventListener: EventListener) {
val proxy = route.proxy
val address = route.address
val rawSocket = when (proxy.type()) {
Proxy.Type.DIRECT, Proxy.Type.HTTP -> address.socketFactory.createSocket()!!
else -> Socket(proxy)
}
this.rawSocket = rawSocket
eventListener.connectStart(call, route.socketAddress, proxy)
rawSocket.soTimeout = readTimeout
try {
Platform.get().connectSocket(rawSocket, route.socketAddress, connectTimeout)
} catch (e: ConnectException) {
throw ConnectException("Failed to connect to ${route.socketAddress}").apply {
initCause(e)
}
}
...
}
Copy the codeFrom the source, if the proxy type is direct or HTTP/FTP proxy, create a socket directly, otherwise, specify the proxy type to create. We see that a rawSocket is returned after creation, which represents a TCP connection. Finally, platform.get ().connectsocket is called, which actually calls the socket’s connect method to open a TCP connection.
The TLS connection
Establishing a Connection Protocol (establishProtocol) begins after a TCP connection is established or an Http proxy tunnel is created.
private fun establishProtocol(connectionSpecSelector: ConnectionSpecSelector, pingIntervalMillis: Int, call: Call, eventListener: EventListener) {
if (route.address.sslSocketFactory == null) {
if (Protocol.H2_PRIOR_KNOWLEDGE in route.address.protocols) {
socket = rawSocket
protocol = Protocol.H2_PRIOR_KNOWLEDGE
startHttp2(pingIntervalMillis)
return
}
socket = rawSocket
protocol = Protocol.HTTP_1_1
return
}
eventListener.secureConnectStart(call)
connectTls(connectionSpecSelector)
eventListener.secureConnectEnd(call, handshake)
if (protocol === Protocol.HTTP_2) {
startHttp2(pingIntervalMillis)
}
}
Copy the code- Check whether the current address is HTTPS.
- If not HTTPS, check whether the current protocol is plaintext HTTP2. If yes, call startHttp2 to start the HTTP2 handshake. If Http/1.1, return.
- If HTTPS is used, the TLS protocol is connectTls.
- If HTTPS and HTTP2 is used, in addition to establishing the TLS connection, startHttp2 is called to start the HTTP2 handshake.
In step 3 above, we mentioned TLS connectTls, so let’s look at its internal implementation:
private fun connectTls(connectionSpecSelector: ConnectionSpecSelector) {
val address = route.address
val sslSocketFactory = address.sslSocketFactory
var success = false
var sslSocket: SSLSocket? = null
try {
// Create the wrapper over the connected socket.sslSocket = sslSocketFactory!! .createSocket( rawSocket, address.url.host, address.url.port,true /* autoClose */) as SSLSocket
// Configure the socket's ciphers, TLS versions, and extensions.
val connectionSpec = connectionSpecSelector.configureSecureSocket(sslSocket)
if (connectionSpec.supportsTlsExtensions) {
Platform.get().configureTlsExtensions(sslSocket, address.url.host, address.protocols)
}
// Force handshake. This can throw!
sslSocket.startHandshake()
// block for session establishment
val sslSocketSession = sslSocket.session
val unverifiedHandshake = sslSocketSession.handshake()
// Verify that the socket's certificates are acceptable for the target host.
if(! address.hostnameVerifier!! .verify(address.url.host, sslSocketSession)) { val peerCertificates = unverifiedHandshake.peerCertificatesif (peerCertificates.isNotEmpty()) {
val cert = peerCertificates[0] as X509Certificate
throw SSLPeerUnverifiedException(""" |Hostname ${address.url.host} not verified: | certificate: ${CertificatePinner.pin(cert)} | DN: ${cert.subjectDN.name} | subjectAltNames: ${OkHostnameVerifier.allSubjectAltNames(cert)} """.trimMargin())
} else {
throw SSLPeerUnverifiedException(
"Hostname ${address.url.host} not verified (no certificates)")
}
}
val certificatePinner = address.certificatePinner!!
handshake = Handshake(unverifiedHandshake.tlsVersion, unverifiedHandshake.cipherSuite, unverifiedHandshake.localCertificates){ certificatePinner.certificateChainCleaner!! .clean(unverifiedHandshake.peerCertificates, address.url.host) }// Check that the certificate pinner is satisfied by the certificates presented.certificatePinner.check(address.url.host) { handshake!! .peerCertificates.map { itas X509Certificate }
}
// Success! Save the handshake and the ALPN protocol.
val maybeProtocol = if (connectionSpec.supportsTlsExtensions) {
Platform.get().getSelectedProtocol(sslSocket)
} else {
null
}
socket = sslSocket
source = sslSocket.source().buffer()
sink = sslSocket.sink().buffer()
protocol = if(maybeProtocol ! =null) Protocol.get(maybeProtocol) else Protocol.HTTP_1_1
success = true
} finally{... }}Copy the codeThis code is very long, specific logic I summarized the following points on the source code:
- SslSocket is created using the request address host, port, and TCP socket.
- Configure the Socket encryption algorithm, TLS version, etc.
- Call startHandshake() to enforce a handshake;
- Verify the validity of the server certificate.
- Using handshake record for certificate lock check (Pinner);
- If the connection succeeds, the handshake record and the ALPN protocol are saved.
The Tsl source code for encrypted connections is actually consistent with the client-server communication rules defined by HTTPS. After sslSocket is created, communication between the client and server begins.
conclusion
OkHttp generally implements the request as described above, following the source code through a request to process and return the whole process, OkHttp does a lot of details during the encapsulation, also uses a lot of design patterns, such as the core responsibility chain pattern, builder pattern, factory pattern and strategy pattern, etc., are worth learning.
That’s OkHttp. I hope this article can help you. Thanks for reading.
The resources
OkHttp source code deep analysis -OPPO Internet technology
Recommended reading
UDP/TCP protocol