Recently, the department organized a front-end performance optimization communication meeting, and many optimization points were proposed in the process from the input page URL to the final page display content. But at the same time, I found that many students could not connect their knowledge of HTTP protocol layer, so I sorted out this article, hoping to bring you a bit of inspiration.

What are we going through at the network protocol level when we make an AJAX request on a page?

// Initiate a request

fetch('https://baidu.com')

// Protocol layer 1...

// Protocol layer 2...

// Protocol layer 3...

.then(res= >

  // Get the result

  console.log(res)

})

Copy the code

As shown in the above code, we launched a network request to Baidu.com and finally got the specific response content in the then method.

The result of capturing packets using Wireshark is as follows:


As can be seen from the figure, when requesting Baidu.com, the connection is first established through TCP three-way handshake, then content is transmitted through HTTP, and finally the connection is disconnected through TCP four-way handshake.

The real process is more complex, and we mainly analyze the following points:

  • Establish connection phase
    • DNS Domain name Resolution (Application layer)
    • Establishing a TCP connection (Transport Layer)
      • Find the target server by IP addressing (network layer)
      • Find the server hardware interface by Mac addressing (data link layer)
      • Transmit bit information to server hardware interface through network cable (physical layer)
  • Data sending stage
    • Establishing an SSL Secure Connection (Transport Layer)
    • Sending HTTP requests (application layer)

Establish connection phase

To obtain the web content of Baidu.com, it is necessary to establish a connection with Baidu server. How to establish this connection?

  1. Obtain the IP address of Baidu through DNS.
  2. Establishing a TCP connection.

DNS Domain name Resolution

Through DNS resolution, we can find the CORRESPONDING IP address of Baidu server.

As shown in figure:


After DNS resolution, we can get the IP addresses of Baidu.com: 39.156.69.79 and 220.181.38.148. Usually, the client will randomly select an IP address for communication.

Domain name resolution steps

In fact, the IP address does not need to pass DNS resolution to obtain the IP address. The IP address is usually cached by the client. The CLIENT requests the DNS server only when the DNS cache is not matched.

The procedure is as follows:

  1. Check whether the browser has a cache IP address.
  2. Check whether the Host has cached the IP address, for example, check the Host file.
  3. Check whether the local DNS server has cached IP addresses of carriers such as China Telecom and China Unicom.
  4. The DNS root DNS server resolves the domain name IP address.
  5. The domain name IP address is resolved by the two root DNS servers.
  6. And so on, finally get the IP address.

Establishing a TCP Connection

With an IP address, the client and server can establish a connection, starting with a TCP connection.

TCP is a connection-oriented, reliable, byte stream – based transport-layer communication protocol.

At this level, the data we transmit is packed byte by byte into a message, which is sent when the length of the message reaches the maximum segment (MSS). If the transmitted packets are very long, they may be divided into multiple TCP packets for transmission.

The TCP header is as follows:


We mainly look at the following points:

  • Source port, destination port.
  • Serial number: SEQ, the unique identifier of a packet.
  • Acknowledgement number: ACK: identifies the packet to confirm whether seQ has received the packet.
  • TCP sign:
    • A SYN value of 1 indicates whether this is a connection request or a connection accept request. Used to create connections and synchronize serial numbers.
    • An ACK value of 1 indicates that the confirmation number field is valid. Note that the uppercase ACK is just a flag, not the same as an ACK for confirmation.
    • If FIN is 1, the connection needs to be released.
  • Window: indicates the number of bytes that the sender can receive, that is, the size of the receiving window, for flow control.

Now, how does TCP establish a connection?


As shown in the figure, establishing a TCP connection requires three steps, commonly known as three-way handshake.

  • First handshake: The client sends the identifier seq= X to the server to establish a connection.
  • Second handshake: The server sends back an ACK =x+1 identifier to acknowledge receipt of the first handshake and sends its own identifier seq=y.
    • The client confirms that the data sent by itself can be received by the server.
  • Third handshake: The client sends ack= Y +1 to acknowledge receipt of the second handshake.
    • The server confirms that the data sent by itself can be received by the client.

After three handshakes, the client and server can send and receive data normally, and the TCP connection is established successfully.

TCP Reliable transmission principle

As mentioned above, TCP is a reliable transport. Why?

This is because TCP uses the stop-wait protocol ARQ, which realizes reliable transmission of information through confirmation and retransmission mechanism.

Such as:


  • The client sends data M1
  • The server confirms that data M1 is received
  • The client sends data M2
  • The server confirms that data M2 is received
  • And so on…

During this period, if a piece of data has not been confirmed for a long time, the client retransmits the data. In this way, for every data sent, the server side has been confirmed, that is, to ensure the reliability of data.

Although ARQ can satisfy data reliability, only one request can be sent and acknowledged at a time, which is inefficient, hence the continuous ARQ protocol.

Continuous ARQ protocol sends a group of data continuously and then waits for the confirmation information of this group of data in batches. It is like changing single-threaded ARQ into multi-threaded ARQ, which greatly improves the utilization efficiency of resources.


Such as:

  • The client sends data M1, M2, M3, and M4.
  • The server confirms that data M4 is received, indicating that M4 and previous data are received.
  • The client sends data M5, M6, M7, and M8.
  • The server confirms that data M8 is received, indicating that data M8 and previous data are received.

In this process, the server does not need to return confirmation information for each data, but accepts multiple data together, which is called cumulative confirmation.

Here is a question, every TCP handshake, how to find the destination server?

A: Through IP.

Locate the target server based on the IP protocol

The purpose of IP protocol is to realize data forwarding at the network layer. It continuously jumps through the router and finally sends the data to the destination successfully.

Each of the TCP handshakes and data interactions mentioned above are transmitted over IP.

The IP header is as follows:


Let’s focus on the following two points:

  • The source IP address
  • Destination IP address

To initiate an IP request, perform the following steps:

  1. Build the IP request header (source IP, destination IP).
  2. The IP protocol calculates a path to the server through an algorithm.
  3. The sender queries the routing table, finds the IP address of the next hop (usually a router), and sends data.
  4. The router queries the routing table, finds the IP address of the next hop, and sends data.
  5. Repeat step 4 until the destination LAN is found.
  6. Send data.

A routing table exists on a computer or router and consists of the destination IP address, subnet mask, next hop address, and sending interface. Based on the destination IP address, you can find the IP address of the next hop for forwarding.

For example, A wants to send IP data to G.


The specific process is as follows:

  • A Generates the IP header (source IP: A, destination IP: G)
    • User A queries the routing table, finds that the next hop is B, and sends the data to B.
  • B Generates the IP header (source IP: A, destination IP: G)
    • User B queries the routing table, finds that the next hop is E, and sends the data to E.
  • E Generates the IP header (source IP: A, destination IP: G)
    • E queries the routing table and finds that the next hop is G and sends the data to G.
  • Arrive at destination G.

Are you wondering why IP sends data to G along this path?

In fact, there is not only one path in the figure above. We reach destination G through ABEG, and we can also reach G through ABCFHG. Both of these two paths can complete the task.

This involves IP addressing algorithms.

IP addressing algorithm

We can think of all the computers in the network as a point, and the connections between the computers as a line, and these points and lines are combined to form a graph.

Such as:


Using the diagram above, we can turn a complex network into a mathematical problem. IP addressing algorithm is actually the shortest path algorithm in graph theory.

The shortest path algorithm has two implementations in IP protocol:

  • RIP
    • The distance vector algorithm is used to minimize the number of IP route hops.
    • The principle of
      • Each node stores location information of other nodes (the number of hops and the IP address of the next hop).
      • Through data exchange with the neighbor node, update the shortest distance to the destination, repeatedly, can get the shortest path from the start to the end.
      • Simple implementation, low cost, suitable for small networks.
  • OSPF protocol
    • Dijkstra algorithm is used to ensure the fastest IP route hop.
    • The principle of
      • Starting from the starting point, the strategy of the greedy algorithm is used to traverse the adjacent nodes of the nearest unvisited vertex at the beginning point until it expands to the end point.
      • Applicable to large networks.

With these two protocols, we can find a path to our destination.

This raises a question: How does IP data jump from one router to another?

A: Over the Ethernet protocol.

Find the server hardware interface by Mac addressing

IP protocol is mainly used to find the optimal path, the specific transmission is done by Ethernet protocol.

Ethernet belongs to the data link layer, which is mainly responsible for the communication between neighboring devices. By querying the Mac address table of the switch, you can find the physical interfaces of both sides and start the communication.

The Ethernet header is as follows:


We are only concerned with the following three points:

  • The source Mac address
  • Destination Mac Address
  • Check code CRC: Checks whether the current frame is valid.

As you can see, the Ethernet layer communicates with each other by Mac address. Where did the Mac address come from?

A: Through ARP.

ARP is a protocol that searches for Mac addresses by resolving IP addresses. After an IP address is converted into a Mac address, Ethernet data can be transmitted.

Such as:


When machine A sends data to machine C:

  • A constructs Ethernet packets (source ADDRESS: A, destination address: C) and sends data frames through the network adapter.
  • When the data frame reaches switch B, the switch retrieves the Mac address of destination ADDRESS C.
  • B queries the Mac address table and matches the hardware interface of C based on the destination Mac address.
    • If C’s hardware interface is found, send data.
    • If the hardware interface of C is not found, a broadcast message is sent to all the machines directly connected to B to search for C. After the broadcast message is found, C is recorded in the Mac table.

After the above process, we find the hardware interface of the destination machine.

Using the Ethernet protocol, we find the hardware interface of the target machine. How do we send the message next?

A: Through the physical layer.

Transmits bit information to the hardware interface of the server through the network cable

In the era without WiFi, we could only access the Internet by plugging in a network cable, which is actually one of the devices in the physical layer.

A network cable can be made of various materials, the most common of which are optical fibers and cables.

Optical fiber and cable are similar in that they simulate binary data using two signals, one bit for each signal.

  • In cable: high potential means 1, low point means 0.
  • In the optical fiber: 1 is on, 0 is off.

For example, in optical fibers, binary data is transmitted by observing the flickering of light.

With these physical devices, we can convert complex data into optical or electrical signals for transmission.

Data sending stage

Sending data can be divided into two steps:

  • Establish SSL
  • Sending an HTTP request

Establish SSL

The example in this article is to send an HTTPS request, so before sending the data, an SSL security layer is created for data encryption.

There are two common encryption methods:

  • Asymmetric encryption
    • A has A key, B has no key, and they both have A common lock. When B sends data to A, it locks the data before sending it.
    • When receiving data, A unlocks the lock with the key to obtain data. Except for “A”, no one else has the key and can’t get the data.
    • Unidirectional communication encryption is realized.
  • Symmetric encryption
    • Both A and B have the same key and A common lock. Each time they send data, they put the data in the lock for sending.
    • Upon receiving the data, each party uses its own key to unlock it. The others don’t have keys, so they can’t access the data.
    • Two-way communication encryption is realized.

Internet communication is two-way, so we need symmetric encryption, but how can we ensure that both sides of the communication have the same key?

Current solutions:

  • Asymmetric encryption is first used to negotiate the secret key so that both parties get the same key.
  • Symmetric encryption is then used for encrypted transmission.

The secret key negotiation process is shown as follows:


Highlight in the picture:

  1. The client sends the encryption algorithm supported by itself.
  2. The server selects an encryption algorithm and returns the digital certificate.
  3. The client confirms that the certificate is valid.
  4. The client generates a random number, encrypts it with the server’s public key in the certificate, and sends it to the server.
  5. The server uses private key decryption to obtain random number.
  6. Both parties use the encryption algorithm determined in Step 2 to encrypt the random number and obtain the same symmetric encryption key.

Ok, after the key negotiation, our SSL security layer is set up.

There is a problem with key negotiation:

When negotiating a key, how can you ensure that you are negotiating with a real server and not a middleman?

A: Digital certificates.

Digital certificates focus on two parts:

  • Server Public Key
  • A digital signature

The digital signature is encrypted by the server public key and certificate private key to prevent the server public key from being tampered with.


With a digital certificate, the client can verify the certificate to determine whether the server is a real server.

The verification logic is as follows:


As you can see, the digital certificate is decrypted using the same algorithm. If it gets the same summary of information, it guarantees that the data is valid. If it is inconsistent, it will fail validation and reject subsequent requests.

At this point, all the work is in place to send the HTTP request.

Sending an HTTP request

HTTP protocol actually makes a communication rule, prescribing the communication format between the client and the server.

Take the front page of request Baidu as an example:


As shown in the figure above, the following rules must be followed when making an HTTP request:

  • Request method (required)GET
  • Requested address (required)/
  • HTTP protocol version (mandatory)1.1
  • Other HTTP header fields (optional)Host,User-Agent,Accept
  • Request parameters, placed after the blank line (optional)

When the server responds to the request, it also follows the HTTP response rules:

  • HTTP protocol version (mandatory)1.1
  • Response status code (required)200
  • Status Code Description (Mandatory)OK
  • Other HTTP header fields (optional)Date,Server,ETag,Last-Modified 等
  • Request parameters, placed after the blank line (optional)

As long as we follow this rule, we can communicate HTTP.

So far, we have analyzed all the data requests. Do you understand?

Thinking and summarizing

Through a network request, this paper analyzes the entire PROCESS of HTTP, TCP, IP, Ethernet and other protocols, and finally combs them:

  1. Request to baidu.com.
  2. DNS parses Baidu.com and obtains the IP address.
  3. Establishing a TCP connection.
  4. The IP protocol calculates an optimal path to the server through an algorithm.
  5. When an IP address hops along a path, the IP address is converted into a Mac address through ARP.
  6. The Ethernet finds the hardware interface of the communication party based on the Mac address.
  7. The physical layer transmits bit signals between two hardware interfaces through network cables.
  8. The TCP connection is established. Procedure
  9. Establishing the SSL security layer.
  10. Send an HTTP request.

Finally, welcome everyone to pay attention to the public number [front-end log], to participate in the technology sharing!