The original link: cloud.tencent.com/developer/a…

1, the preface

Network communication has always been an important module in the Android project, Android open source project appeared a lot of excellent network framework, from the beginning just some of the HttpClient and HttpUrlConnection simple package used tool class, Later, Volley, which is relatively complete and rich and open source by Google, and then Okhttp and Retrofit, which are relatively popular today.

To understand the similarities and differences between them (or more specifically, to have a deeper understanding of network communication technologies in Android development), you must have a good knowledge of network basics, the basic principles of the Android network framework, and so on, so that you can find the best network communication practices for your APP at the critical moment.

It turns out that in Android’s daily development and source code reading will often encounter relevant knowledge, grasp the basic knowledge of the network, but also Android programmers really advanced to the process of some basic technical qualities necessary.

In view of this, this article will mainly introduce some computer network foundation, as well as in the Android development of some use and encountered problems and solutions.

This paper is mainly divided into the following parts:

1) Computer network architecture;

2) Http correlation;

3) Tcp related;

4) Socket.

2. About the author

ShuDaFei: ctrip Android development engineer, author’s blog: www.jianshu.com/u/ae3793361… .

Note: In order to make this article easier to understand, the content has been revised more carefully.

3. Computer network architecture

It is necessary to clarify this in computer network architecture, the often layered structure of computer network architecture, to prevent entanglement between Http and Tcp, two protocols that are not at all in the same layer. There are several versions of the hierarchy, depending on the reference model, such as the OSI model and the TCP/IP model.

Here is an example of the more commonly seen 5-layer structure:

(For a clearer and more complete picture, please see “Computer Network Communication Protocol Diagram (Chinese Edition)”)

As shown in the figure above, the architecture of five layers is top-down, and finally can realize end-to-end data transmission and communication. What are their responsibilities, and how can they finally realize end-to-end communication?

1) Application layer: if HTTP protocol is adopted, it actually defines how to package and parse data. If HTTP protocol is adopted, data will be packaged in accordance with protocol provisions, such as in accordance with the request line, request header and request body, and the data will be sent to the transport layer after being packaged.

2) Transport layer: the transport layer has two protocols, TCP and UDP, respectively corresponding to reliable transport and unreliable transport. For example, TCP needs to provide reliable transmission, so it has to solve how to establish connections, how to ensure reliable transmission without data loss, and how to adjust flow control and congestion control. About this layer, we usually deal with Socket, Socket is a group of encapsulated programming call interface, through it, we can operate TCP, UDP connection establishment and so on. When we use sockets to establish connections, we usually need to specify the port number, so this layer specifies the corresponding port number to send data.

3) Network layer: this layer of IP protocol, as well as some routing protocols, etc., so this layer of data to be transmitted to which IP address. The middle involves some optimal routes, routing algorithms and so on.

4) Data link layer: THE most impressive protocol is ARP, which is responsible for resolving IP addresses into MAC addresses, namely hardware addresses, so as to find the corresponding unique machine.

5) Physical layer: this layer is the lowest layer, providing binary stream transmission service, that is, the real start of data transmission through transmission media (wired, wireless).

Therefore, through the above five layers of their respective roles, the realization of physical transmission medium -MAC address -IP address – port number – to obtain data according to the application layer protocol data analysis and ultimately achieve network communication and data transmission.

The following will focus on HTTP and TCP related things, about other layers, graduated so long also forget a lot, if you want to more detailed understanding of the following three layers such as routing algorithm, ARP addressing and physical layer and so on or to go to see “TCP/IP detailed volume 1: Protocol” ~

4. HTTP related

This section is mainly about some basic knowledge of Http, and some practical applications in Android and encountered problems and solutions.

For space reasons, this article will only give a brief overview of some of the knowledge points. If you want to fully understand the HTTP protocol, please read the following articles:

Web Programming Slacker’s Guide (7) : Understand HTTP in a Nutshell

From HTTP/0.9 to HTTP/2: Understanding the History and Design of the HTTP Protocol

Introduction to Brain-dead Network Programming (3) : Some must-know HTTP protocols

4.1 Correctly Understanding HTTP “Connectionless” and “Stateless”

Http is connectionless and stateless.

Connectionless does not mean that there is no need for connections. Http is only an application-layer protocol and ultimately depends on transport layer services such as TCP to connect.

Connectionless means that the HTTP protocol processes only one request for each connection and disconnects the connection after each request is completed to relieve the pressure on the server and reduce the usage of server resources by connections. My understanding is that establishing connections is really a matter of the transport layer, and in the case of HTTP for the application layer, it is connectionless because the upper layer is not aware of the lower layer.

Stateless means that each request is independent of each other and has no ability to remember previous requested transactions. So there are things like cookies, things that store state.

4.2 Request Packets and Response Packets

This section briefly describes the basic knowledge of the format of HTTP request packets and response packets.

Request message:

Response message:

The differences between Get and Post that we are all familiar with are as follows:

1) Get will concatenate the request parameters after the URL, and finally display in the address bar, while Post will put the request parameters data in the request body, will not be displayed in the address bar;

2) Length limitation of transmission parameters.

Question:

For the first point, exposing private data in the address bar in the browser is indeed inappropriate, but in App development, there is no concept of the address bar, so will this become a constraint for choosing POST or GET?

For point 2, the length limit should be a browser limit, not a get limit. If it is in App development, this point can also be ignored.

4.3 HTTP Caching mechanism

The reason why I want to introduce the following Http caching mechanism is because Okhttp uses the Http caching mechanism for network requests, rather than the client writing its own cache strategy like Volley and other frameworks.

Http caching is controlled primarily by two header fields: cache-Control and ETag, which are described below.

Private: Only clients can cache.

Public: both client and proxy servers can cache.

Max-age: cache expiration time.

No-cache: a comparison cache is required to validate cached data;

No-store: All memory will not be cached.

The server sends the cache policy to the client through the header for the first time.

Max-age: indicates the cache expiration time. If the cache expiration time is not exceeded, you can directly use the cache.

No – cache: it means you need to use the contrast cache to verify the cached data, if this field is open, even if Max – age cache without failure, is still need to initiate a request to the server confirm resource if there’s any update, whether to need to request data, as to how to do comparative cache, is this the Etag it. If the server confirms that the resource is not updated, the server returns 304 and obtains the local cache. If the resource is updated, the server returns the latest resource.

No-store: If this field is turned on, no cache will be performed and no cache will be fetched.

When the client sends the first request, the server sends the Etag of the current resource. When the client makes the next request, the client uses if-none-match in the header to add the Etag. The server compares the Etag sent by the client with the latest resource Etag. The resource is not updated and 304 is returned.

The Http caching mechanism is realized through the cooperation of cache-Control and Etag. For more information about HTTP caching, please refer to the relevant section of the article “Introduction to Brain-dead Network Programming (3) : Some of the necessary knowledge of the HTTP protocol” for detailed reading.

4.4 HTTP cookies

Http protocol is stateless, and cookies are used to remember some states in the local cache. A Cookie generally contains domin(owning domain), PATH, Expires(expiration time) and other attributes. The server can write state to the client’s Cookie through set-cookies in the response header. The next time a client makes a request, it can take the Cookie with it.

Speaking of cookies, they are not often encountered if they are just for App development, but they may be encountered if they are related to WebView requirements.

Here’s a heart-wrenching anecdote about WebView cookies THAT I encountered on the project: The requirements are as follows: the H5 page in the loaded WebView needs to be logged in, so we need to manually write ticket into the Cookie of the WebView after logging in to the native page, and then the H5 page loaded in the WebView will be verified by the server with the ticket in the Cookie.

But there was a problem: Use Chrome Inspect to debug WebView, manually written Cookie is indeed written in, but when the request is initiated, Cookie is not brought, resulting in a request verification failure, after the check, is WebView attribute default closed, through the following code Settings can be opened:

CookieManager cookieManager = CookieManager.getInstance(); if(Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) { cookieManager.setAcceptThirdPartyCookies(mWebView, true); } else{ cookieManager.setAcceptCookie(true); }

4.5 the Https

As we all know, Https guarantees the security of data transmission. Https=Http+Ssl, the main principle to ensure the security is the use of asymmetric encryption algorithm. The common symmetric encryption algorithm is not secure because both sides use a unified key for encryption and decryption. Then someone else can use the key to decrypt the data.

However, the core essence of asymmetric encryption algorithm to achieve secure transmission is that the information encrypted by public key can only be unlocked by private key, and the information encrypted by private key can only be unlocked by public key.

Server application certificate issued by the CA authorities, to obtain the certificate of public and private keys, only the server knows the private key, and the public key can inform others, if can pass the public key to the client, so that the client by the server from the public key to encrypt their data transmission, and the server using the private key can decrypt the data. Since the data encrypted with the public key on the client can be decrypted only by the private key, which only the server has, the data transfer is secure.

Here’s a brief description of how asymmetric encryption algorithms ensure data security, but Https is far more complicated than that (there are plenty of articles on the web that don’t have space to go into details) :

One is that the client also needs to verify the validity and validity of the CA certificate sent from the server. The CA certificate may be switched during transmission, which involves how the client verifies the validity of the server certificate to ensure that the identities of the communication parties are legitimate.

The other asymmetric algorithm ensures the data security, but its efficiency is relatively poor compared with the symmetric algorithm. How to optimize the algorithm not only ensures the data security, but also improves the efficiency.

Firstly, CA certificates generally include the following contents:

The issuing authority and version of the certificate; Certificate user; Public key of certificate; Certificate validity period; The digital signature Hash value of the certificate and the signature Hash algorithm (the digital signature Hash value is the value encrypted with the private key of the certificate); And so on.

Client authentication server to get the certificate of legitimacy through: first use of access to the public key to decrypt the digital signature certificate Hash value 1 (because it is the use of private key encryption), and then use the signature in the certificate of the Hash algorithm to generate a Hash value 2, if the two values are equal, the said certificate legal, the server can be trusted.

By the way, I used Android WebView to load the web page on the company’s test server in the project development, which resulted in a blank screen.

The solution is to temporarily ignore SSL errors in the test environment, so that the web page can be loaded, of course, do not do this in production, one is there will be security issues, one is Google Play should not pass the audit.

The best way to do this is to override WebViewClient’s onReceivedSslError() :

@Override publicvoidonReceivedSslError(WebView view, SslErrorHandler handler, SslError error) { if(ContextHolder.sDebug) { handler.proceed(); return; } super.onReceivedSslError(view, handler, error); }

Finally: For more detailed and comprehensive knowledge of HTTPS, please read im Security part 7: If this is how to understand HTTPS, one article is enough.

The Http 2.0 4.6

Okhttp supports configuration using the Http2.0 protocol, which is a huge improvement over Http1.x for the following reasons.

1) Binary format: HTTP1. x is a text protocol, and HTTP2.0 is a binary frame as the basic unit, is a binary protocol, a frame in addition to contain data at the same time also contains the frame Identifier: Stream Identifier, that is, identify the frame belongs to which request, making network transmission become very flexible;

2) Multiplexing: a big improvement over the original http1.x connection-per-request situation, which has a lot of limitations and causes many problems, such as the cost and efficiency of establishing multiple connections.

Http1. x may make as many concurrent requests as possible to load resources. However, the browser has restrictions on concurrent requests under the same domain name. The optimization method is usually to put the requested resources under different domains to avoid this limitation.

Http2.0 supports multiplexing to solve this problem, multiple requests share a TCP connection, multiple requests can be concurrent on this TCP connection, one is to solve the problem of establishing multiple TCP connections consumption, one also solves the problem of efficiency.

So what is the principle that allows multiple requests to be concurrent on a TCP connection? The basic principle is the above binary frame division, because each frame has an identity, so multiple requests for different frames can be sent out of order, the server will according to the identity of each frame, it will be sorted into the corresponding request.

3) Header header compression: It is mainly used to compress the header to reduce the size of requests, reduce traffic consumption and improve efficiency. The problem was that every request had a header, and the data in the header was usually at one level.

4) Support server push.

For more information about HTTP2, see HTTP/0.9 to HTTP/2: The History and Design of HTTP.

5. TCP

TCP is connection-oriented and provides reliable data transmission. At this level, we typically use the Socket Api to manipulate TCP, establish connections, and so on.

5.1 Three-way Handshake to Establish a connection

First: send SNY=1 to indicate that the handshake is a request to establish a connection, and then SEQ generates a random number X of the client

The second time: send SNY=1,ACK=1 indicates that the connection is established in response to the request, and then ACK= seq+1 of the client (so that the client can confirm that it is the server that wants to connect before receiving the request), and then send the server to generate a random number representing itself.

Third time: ACK=1. Seq = client random number +1, ACK = server random number +1

Why do you need three handshakes to establish a connection?

Very clear above all is two shake hands is the most basic, shaking hands for the first time, the C terminal sends a connection request message to the S, S end S after receiving end can know oneself with the C is the connection is successful, but C end right now does not know whether the S end receives the message, so I have to reply S end after the message is received, the c-terminal get S after the reply, To make sure that they can be connected to the S end, this is the second handshake.

The C terminal can send data only when it is sure that it can connect to the S terminal. So two handshakes must be basic.

So why the third handshake? Suppose instead of a third handshake, we think the connection is established after two handshakes, what happens?

The third handshake is used to prevent the invalid connection request packet segment from being sent to the server and causing an error.

The details are as follows:

C side sent the first network connection request for some reason stranded in the network node, lead to delay, until the connection release at some point in time to reach S side, this is a failure message already, but this time the s-video still think this is the first time C side to establish connections to shake hands, then responded to the c-terminal S end, a second hand.

If there are only two handshakes, then at this point, the connection is established, but the C side doesn’t have any data to send, and the S side just waits, wasting a lot of resources. So a third handshake is required, and only the C side responds again can avoid this situation.

For a better understanding of the TCP three-way handshake, don’t miss the following article:

Chapter 18: Establishment and Termination of TCP Connections

“Understanding TCP in Depth (PART 1) : Fundamentals”

Classic Theory: TCP three-way handshake and Four-way Wave

Wireshark Packet Capture Analyzing TCP Three-Way Handshake and Four-Way Wave

Learn TCP three handshakes and four waves with Animation

5.2 Four wave disconnects

After parsing the connection diagram above, this diagram should not be hard to understand.

The main question here is: why is there one more wave than when establishing a connection?

The server ACK(reply client) and FIN(terminate) messages are not sent at the same time. Instead, it ACK first and then FIN. When the client requests to disconnect, the server may have unfinished data, so ACK first and then FIN when the data is finished. That makes four handshakes.

TCP establishes and disconnects connections. The most important feature of TCP is to provide reliable transmission, so how to ensure reliable data transmission? This is the sliding window protocol to be discussed below.

Read more:

Classic Theory: TCP three-way handshake and Four-way Wave

Wireshark Packet Capture Analyzing TCP Three-Way Handshake and Four-Way Wave

Learn TCP three handshakes and four waves with Animation

5.3 Sliding Window protocol

The sliding window protocol is fundamental to ensure reliable transmission of TCP, because the sending window only receives the confirmation frame will move the window back to continue to send other frames.

Here’s an example: Suppose the send window is 3 frames

A began to send window frame in the first 3 [1, 2, 3], the first three frames can be sent later, temporarily can not send, such as [1] frame after send out, received a confirmation message from the receiver, then send window can move back 1 frame, at this time send window to [4] 2, only send the window frame can also be sent, An analogy.

The receiving window will put the frame into the corresponding position after receiving it, and then move the receiving window. The interface window has the same size as the sending window. If the receiving window is 5 frames, the frames outside the receiving window will be discarded.

The different Settings of the size of the sending window and the receiving window extend the different protocols:

Stop-wait protocol: Each frame must wait for a confirmation message before the next frame can be sent. Disadvantages: Poor efficiency.

N-frame backward protocol: The recipient sends a cumulative confirmation message to the sending window to confirm that the N frame is correctly received. If the sender does not receive the confirmation message within a specified time, the sender considers that it has timed out or data is lost, and resends all frames after the confirmation frame. Disadvantages: Error the PDU after the serial number has been sent, but it still needs to be sent again, which is wasteful.

Select the retransmission protocol: If an error occurs, only the PDU involved in the error is retransmitted, improving the transmission efficiency and reducing unnecessary retransmission.

The last problem remains: There is a mismatch between the sending and receiving efficiency between the sending and receiving Windows, which leads to congestion. To solve this problem, TCP has a set of traffic control and congestion control mechanisms.

5.4 Flow control and congestion control

Traffic control is to control the traffic on a communication path. The sender adjusts the sending rate dynamically by receiving the feedback from the receiver to control the traffic. The purpose is to ensure that the sending speed of the sender does not exceed the receiving speed of the receiver.

Congestion control is a global control that controls the traffic of the entire communication subnet.

① Slow start + Avoid congestion

Let’s start with a classic picture:

Using a slow start at the beginning, the congestion window is set to 1, then the congestion window exponential growth to slow start threshold (ssthresh = 16), the switch for congestion avoidance, namely additive increase, such growth to a certain extent, lead to network congestion, at this time will reduce the congestion window to drop to 1, namely to slow start, at the same time adjust to 12 new slow start threshold method, And so on.

② Fast retransmission + fast recovery

Fast retransmission: All the retransmission mechanisms mentioned above wait for a timeout and no reply is received from the receiver before retransmitting. The design idea of fast retransmission is as follows: If the sender receives ACK messages from three duplicate recipients, it can determine that a packet segment is lost. In this case, the sender can immediately retransmit the lost packet segment without waiting until the timeout period expires, which improves the retransmission efficiency.

Fast recovery: The congestion control will reduce the congestion window to 1 and restart slowly when the network is congested. One problem is that the network cannot quickly recover to normal state. Fast recovery is to optimize this problem. When congestion occurs, the congestion window will only decrease to the new slow start threshold (12), instead of 1, and then directly enter congestion to avoid additive growth, as shown in the following figure:

Fast retransmission and fast recovery are further improvements to congestion control.

For a more in-depth understanding of this section, read: TCP/IP In Detail – Chapter 21 – TCP Timeouts and retransmission, easy to Understand – In Depth TCP part 2: RTT, Sliding Windows, and Congestion Handling.

6, about sockets

Sockets are a set of APIS for handling TCP/UDP. HttpURLConnection and Okhttp are used to send network requests at the lower level, and of course they are also used to send network requests through sockets, while Volley and Retrofit are used to send network requests at the higher level. Finally, it relies on HttpURLConnection or Okhttp for the final connection establishment and request sending.

The simple use of sockets should be able to establish a Socket on both ends, the server is called ServerSocket, and then establish a connection.

For more information, please read:

“Network programming lazy introduction (8) : How to write tcp-based Socket long connection”

Introduction to Brain-dead Network Programming (II) : What are We Reading and writing when We read and write sockets?

7. Summary of this paper

Of course, the above content is only the computer network foundation that I know and think is very important, and there are a lot of basic knowledge of network that needs to be deeply understood and explored. Write a lot of, it is an arrangement of their own network foundation, may also have mistakes, right when the introduction of a brick, but also ask you cattle generous comments.

My [graphite document] (the article is too long and a large number of interview questions are not shown, interested can see!