Connect the dots: from TCP, to HTTP requests, to three handshakes and four waves

Why does TCP have 3 handshakes and 4 waves after reading this? Can’t you shake hands twice? What is TCP?

Fat nerdy panicked, TCP didn’t know what it was, who gave me the courage to read this article.

So let’s start with TCP.

(In the end, I found nearly 10,000 words, which covered a lot of knowledge points. Would you like to read this article when reviewing next time?)

Understand TCP

TCP/IP protocol family

Most of the networks we use are based on the TCP/IP protocol family.

Since computers and network devices need to communicate with each other, they need rules to communicate. For example, which side initiates the communication, what language is used for communication, these need some rules. These rules are agreements.

There are all kinds of things in the protocol, such as cable specifications, the order in which the two parties establish communication, Web pages showing the steps that need to be processed, and so on.

The TCP/IP protocol family is a collection of protocols associated with the Internet. Some say that TCP/IP is TCP and IP, and some say that TCP/IP is a general name of the protocol family used in IP protocol communication.

Layered management of TCP/IP

The TCP/IP protocol family is divided into application layer, transport layer, network layer, and data link layer.

Why do people have to layer?

Assume that the entire protocol is a whole without hierarchical design. If a part of the protocol needs to be modified, the entire protocol needs to be modified. However, after layered decoupling, the contents of each layer are relatively independent and do not affect each other. If you need to change the protocol content of one layer, it does not affect the content of other layers.
Layering is also good for design, as each layer just has to do its job. Applications at the application level can only worry about their own tasks, regardless of where the other side is, what kind of transmission line the other side has, and whether the transmission can be guaranteed.

Layer of the TCP/IP protocol family

The application layer

The application layer determines the type of application for this communication, For example, file Transfer Protocol (FTP), Domain Name System (DNS), Simple Mail Transfer Protocol (SMTP), and so on. HTTP also belongs to the application layer. In plain English, the application layer decides what to do this time. For example, FTP is used to share network files, DNS is used to convert domain names and IP addresses (it actually works for other application-layer protocols), SMTP is used to transfer mail messages between systems, and HTTP is used to transfer hypertext from web servers to local browsers.
The transport layer

The transport layer provides data transfer between two computers. According to the requirements of different applications, the transport layer requires two different protocols: connection-oriented TCP and connectionless UDP. When the transport layer uses TCP, it does its best to ensure reliable communication, even though the underlying network is unreliable. But if UDP protocol is used, the whole communication is still unreliable.
The network layer

The network layer is used to process the packets flowing across the network. In this layer, it defines the path (transmission route) through which the packets reach each other’s computers and pass them to each other.

When we communicate with each other’s computers through multiple computers or network devices, the network layer can choose a suitable transmission route among many options.
Data link layer (also called link layer, network structure layer)

The data link layer is used to deal with the hardware part of the network, including the control operating system, hardware devices and so on. Hardware categories are within the scope of the data link layer.

What does TCP do?

TCP is a transport protocol specifically designed to provide reliable end-to-end byte stream communication over an unreliable Internet.

There are many key words in this sentence:

Unreliable Internet

On the Internet, different parts of the Internet may have completely different topology structure bandwidth, latency and other parameters, TCP is designed to dynamically adapt to these characteristics of the Internet, and can face various faults.
Reliable communication

The application layers of different hosts often need reliable, pipe-like connections, but the IP layer only provides unreliable packet switching, so the reliable connections are done by TCP.

The application layer sends data streams to the TCP layer. The TCP layer divides the data streams into packets of appropriate length in packet segments and transmits the packets to the IP layer. The IP layer sends the packets to the TCP layer at the receiving end over the network.

To prevent packet loss, TCP assigns the serial number seQ to each packet. When receiving a packet successfully, the receiving end returns an ACK. As shown in figure:
Byte stream communication

“Stream” : A sequence of bytes flowing into or out of a process.

“Byte stream oriented” : While applications (the application layer) interact with TCP one data block at a time (of varying sizes), TCP treats all data as a series of unstructured byte streams that can be partitioned at will.

TCP then has a buffer, and when an application sends a block of data that is too long, TCP can cut it down and send it again. If an application sends only one byte at a time, TCP can also wait for enough bytes to accumulate before sending a segment.

Basically, TCP doesn’t care what’s being sent to it locally (it’s all bytes anyway), it just cuts, and it looks at the current level of network congestion to determine how much to slice. He cares about the timing. He must number what he sends first and then number what he sends later, so as to ensure the order of transmission.

Of course, the receiver’s application must be able to recognize the byte streams received and then restore them to meaningful application-layer data (by sorting them in numbered order).

Other features of TCP:

Is a connection-oriented transport layer protocol.

TCP connections must be established before applications can use TCP again. After data transfer is complete, the established TCP connection must be released.
Each TCP connection can be one-to-one (point-to-point).
TCP provides reliably delivered services. Data sent over a TCP connection is error-free, not lost (retransmitted if lost), not duplicated (deleted if duplicated), and arrives in sequence (mentioned earlier in point 3).
TCP provides full duplex communication. Allows applications on both sides of the communication to send data at any time. Both ends of a TCP connection have send and receive caches.

When it does, the application sends the data to TCP’s cache, the application goes about its business, and TCP sends the data out when it sees fit.

Upon receiving, TCP puts the received data into the cache for the application to read if it wants.

What about UDP, by the way?

UDP does not need to establish a connection before transmitting data, which means it does not need to give any confirmation after receiving a UDP packet. TCP, on the other hand, provides connection-oriented services. A connection must be established before data is transmitted and released after transmission. Therefore, TCP has a much higher overhead than UDP, such as validation, flow control, timer, connection management, etc., which can take up a lot of resources.

When DNS and SNMP are used at the application layer, UDP is used at the transport layer. However, TCP is used when SMTP, HTTP, and FTP are used at the application layer.

The main features of UDP are:

UDP is connectionless (this was mentioned at the beginning).
UDP is delivered as best as possible and reliable delivery is not guaranteed, so there is no need to maintain complex connection states.
UDP is packet-oriented. The UDP of the sender directly adds a header to the packet sent by the application layer and submits the packet to the IP layer. Neither split nor merge. That is, UDP sends whatever the application layer gives UPD, one at a time. And the recipient’s UDP, the IP layer to hand over the data packet directly remove the head is done.

Since UDP delivers a complete packet each time, the application layer must select a packet of appropriate size. If the packet length is too long, the UPD sends the packet to the IP layer, which then fragments the packet, reducing the efficiency of the IP layer.

If the packet is too short, after UDP sends the packet to the IP layer, the header of the IP datagram takes up a larger proportion of the total datagram. This also reduces the efficiency of the IP layer.
The UPD does not have congestion control, so congestion on the network does not slow down the sending rate of the source host. Sending speed is important in some situations, like video, IP phone, etc. Both require the source host to send data at a constant speed and allow some data to be lost. UDP meets this requirement.
UDP supports one-to-one, one-to-many, many-to-one and many-to-many interactive communication.
UDP has a small header overhead of 8 bytes, which is shorter than TCP’s 20 bytes header.

Note that some real-time applications require UDP, but when many source hosts simultaneously send high rate real-time video streams to the network, the network may become congested and no one can receive them properly. Therefore, UDP that does not use congestion control may cause serious congestion problems.

There are some UDP real-time applications, to improve the unreliable transmission of UDP, reduce the loss of data, such as forward error correction or retransmission of lost packets and so on.

Not to mention the differences between TCP and DCP, these are all differences.

Walk through the HTTP request

With that in mind, we can walk through the HTTP request process.

Enter www.baidu.com in the address bar of your Web browser and press Enter
DNS domain name resolution:
1. Internal browser code splits the URL.
2. Resolve domain names.
  
  The browser searches layer by layer to see if there is one saved to the IP address of the domain name, and sends a request to the IP address if there is one. If not, keep looking.
  1. Start by searching the browser’s own DNS cache
  2. If not, the browser searches the system’s own DNS cache
  3. If not, the browser looks for the local hosts file
  If it is not found, go to the DNS server to find the domain name. If it is found, the domain name is resolved into the corresponding SERVER IP address and sent back to the browser.
  
  After parsing the IP, the application layer sends an HTTP request from the client.
The transport layer is working. To facilitate transmission, at the transport layer (TCP), the data (HTTP request packets) received from the application layer are divided into packet segments for management. Then, the serial number and port number of each packet are marked and transmitted to the network layer, so that the server can accurately restore the packet information when receiving the packet.

To ensure secure and reliable transmission, a three-way handshake is used. I’m not going to talk about it, but we’ll talk about it in more detail later.
Arrived after network layer, be IP protocol home ground! The function of IP protocol is to transmit various packets segmented by TCP to the receiver. All sorts of conditions need to be met to ensure that the transmission actually reaches the other side. Two of the most important conditions are IP address and MAC address.

So the IP address has been resolved by DNS at the application layer, except for the MAC address. In this case, the ARP protocol is used to find out the MAC address of the target based on the IP address provided by the communication party.

After the MAC address is found, an address is added to the packet and the packet is forwarded to the link layer.

In fact, we rarely communicate on a local area network, so we usually send packets to routers, which relay them. So the general process is like this: first use ARP to find out the MAC address of the router, and then send the packet to the router, and the router looks, oh! It was destined for this IP address, and then he looked it up in the routing table to see where it was supposed to go next, and he found an IP address that had to be translated into a MAC address by ARP to send the data to the next router. The next router checks the IP, then finds the MAC address of the next router, then passes the data… Until the packet is delivered to the final destination.
The data link layer transfers data. The bridge and switch of the data link layer carry out port forwarding according to the MAC address, and then the repeater and hub of the physical layer carry out signal transmission and amplification. At this point, the client phase of sending the request is over.
Server receives data

The server at the receiving end receives the packet at the link layer and then erases the header information up to the application layer. The HTTP request sent by the client is received.
The server responds to requests

After receiving the HTTP request from the client, the service searches for the resource requested by the client and returns a response packet.

IP address and MAC address

Dizzy.. We’re getting off the subject. Here is a casual look, are directly quoted from other people’s articles, do not want to see the skip can also ~ anyway and the theme of the relationship is not much, just to deepen understanding.

An IP address indicates the network location to which a node is assigned, while a MAC address is the fixed address to which a network adapter belongs (also known as a LAN address, a physical address is an address used to identify the location of a network device). A MAC address uniquely identifies a NIC on the network. If a device has one or more nics, each NIC must have a unique MAC address. MAC addresses are used by the data link layer, and IP addresses are used by the network layer and higher layers.

I saw two articles written very easy to understand and fun, but one of them is not allowed to be reprinted, so I directly posted the two articles, interested can go to see ~

Why use a MAC when you have an IP?

Why use IP when you have a MAC?

Three-way handshake

To be honest, I was only going to write three handshakes, but TCP ran into over 2,000 words and, embarrassed, silently changed the title.

First look at some identifiers and some rules, I think it is quite important, the first time I saw the three handshakes, I thought wow, so many jumbled symbols look very impressive, I looked at it several times but did not understand. It’s easier to understand when you understand their meaning and rules.

Some identifiers and some rules

identifier

The SYN (synchronous)

Request to establish a connection. Sync is a handshake signal to request to establish a connection.
ACK (acknowledgement)

Acknowledgement character. If the receiver successfully received the data, an ACK message will be sent indicating that the data was received correctly.
FIN (finish)

If FIN=1, the sender finishes sending the packet and requests to release the link.
PSH (push) transmission
RST (reset) Resets
URG urgency
Seq (Sequence Number)

Serial number, consisting of four bytes, indicating the number of the first byte of the packet segment carrying data.

Because TCP is byte stream oriented, in a TCP connection, each byte in the byte stream is numbered sequentially.

For example, if the serial number of a packet is 301 and the data carried has 100 fields, it is obvious that the serial number of the next packet (if any) should start with 401.

Why is it random?

First, suppose there is A TCP connection between A and B, which is usually A quad consisting of two IP addresses and two port numbers of A and B.

In this case, user A sends A TCP packet segment whose quad and sequence number are the same as those of the previous TCP connection.

If seQ is fixed, then some malicious attacker can forge a TCP segment to interrupt a normal TCP connection by choosing the appropriate serial number, IP address, and port. However, it is difficult to guess the serial number randomly generated by the algorithm to reduce malicious attacks.

And if a fixed sequence number is used, conflicts between datagrams can easily occur. For example, A establishes A connection with B, and A sends A datagram with sequence number 1, but it lingers on the network for A long time. At this time, A suddenly failed and restarted, and re-established A connection with B, still communicating happily with the previous serial number 1. At this point, the lingering datagram finally arrives at B. B mistakenly thinks it is a datagram from a new connection and accepts it. This will cause the data to be out of order.
Acknowledgment (Acknowledgment Number)

The confirmation number is the serial number of the first byte of the next packet segment expected to receive from the peer party.

For example, B receives the packet segment from A, whose serial number seq is 1 and the data length is 100 bytes. This indicates that B correctly receives the data from A with serial numbers ranging from 1 to 100. Therefore, B expects the next data sequence number from A to be 100+1, so B sets the confirmation number to 101 in the confirmation message to A

Some of the rules

According to TCP, the SYN packet segment cannot carry data, but consumes a SEQ sequence number. This is why we see SYN followed by seQ every time.
An ACK packet segment can carry data, but does not consume serial numbers if it does not. But they still have to bring a confirmation number. If you want to carry data, it’s going to be followed by this SEQ.
The FIN segment consumes a sequence number even if it does not carry data, so the FIN is followed by a SEQ.

Three handshakes

Server preparation: Create transport control block TCB, ready to accept client process connection request, at this time the server enters the LISTEN state

Client preparation: Also create transport control block TCB.

First handshake:

The client sends a packet to establish a connection and enters the SYN-sent state.

Where SYN=1, an initial sequence number seq=x is randomly generated. Indicates that you want to establish a connection with the server.
Second handshake:

After receiving the request packet, the server sends a SYN and ACK packets to confirm the connection. Therefore, SYN=1 and ACK=1. ACK =x+1 because it contains ACK, and seq because it contains SYN. The server enters the SYN-RCVD state.

I was like, what? Why x plus 1? Ack = x+ packet length +1 Baidu check:

When TCP establishes a connection with a three-way handshake:
```
[SYN] Seq=0, Len=0
-------------------------------->

[SYN, ACK] Seq=0, ACK=1, Len=0
<--------------------------------

[ACK] Seq=1, Ack=1, Len=0
-------------------------------->
Copy the code
```
Since SYN can’t carry data, Len=0 of the packet, why would ack become seq+1 of the previous packet instead of SEQ?

To confirm the client’s SYN, the server takes LEN+1 as the ACK value. In this way, the sequence number is incremented by 1 each time a SYN is sent. If it is lost, it will be retransmitted.

If a packet contains SYN or FIN, the server uses the SEQ +1 of the previous packet to acknowledge receipt, even if no data is contained in the packet. The same goes for the other handshakes and waves.
Third handshake:

The client sends an acknowledgement packet. Therefore, ACK=1 and ACK= y+1 are contained in the packet.

At this point, the TCP connection is established and the client enters ESTANBLISHED state.

Client: Indicates that I am ready to send data to me.

The server then received an acknowledgement message and started communicating with ESTANBLISHED.

Some problems with the three-way handshake

Why does TCP use three-way handshake? Is it ok to use two-way handshake?

If the client sends the first request connection, and it’s not lost, it’s just stuck on the network. This is an invalid message, but the server does not know, it still sends back a normal acknowledgement message, and then establishes the link. But in fact, we have been waiting for too long for the server to respond, so we already think that the connection is invalid, and may initiate a second connection, or may not connect. I’m just not sending anything over this connection. But the server still doesn’t know, and he’s just waiting, waiting for you to send him something, waiting for you to close the connection. This results in a waste of resources.

So if we use three handshakes and the server sends us a confirmation message, and we ignore it, he knows the connection didn’t make it, and he ignores you.

Can you carry data during a three-way handshake?

The third TCP handshake allows data to be carried.

Do not carry data for the first time. Do not cache the data and submit it to the application program after the handshake succeeds. In this way, SYN FLOOD attacks are amplified (described later). If someone were to attack the server and put a lot of data into the SYN packet in the first handshake each time, and then sent the SYN packet again and again, the server would spend a lot of time and memory receiving and processing the SYN packet. In other words, the first handshake that releases data makes the server more vulnerable.

However, the host that can send the third handshake packet must receive the second handshake packet, because the host with forged IP address will not receive the second handshake packet. Therefore, the user who can send the third handshake packet must be a legitimate user.

What if the client suddenly hangs?

The client suddenly hangs during normal connection, and if nothing is done to deal with it, the client and server will be idle for a long time. Therefore, we need a SYN Timeout. The server will reset the timer as soon as it receives a SYN message from the client. The Timeout period is usually set to two hours. If there is no response after a total of ten message segments are sent, the client is considered faulty and the connection is terminated.

Four times to wave

If either server or client does not want to communicate, they can issue a disconnection request.

First wave

The client process sends a connection-release packet, so FIN=1 consumes the sequence number, so it is followed by a randomly generated sequence number, seq=u (equal to the sequence number of the last byte of the previously transmitted data plus 1).

The client enters fin-WaIT-1 state,
Second wave

After receiving the FIN, the server sends an ACK (ACK=1) to confirm the connection. As specified, the confirmation number ACK = U +1 is followed by a randomly generated serial number seq= V.

At this point, the server enters lose-wait state. At this point, TCP is in the semi-connected state. Although the client has nothing to send, the server may have nothing to send, so the client still needs to accept the server to send something. This process also lasts for a period of time, i.e. the duration of the close-wait state.

The client, on the other hand, enters fin-WaIT-2 upon receiving the server’s confirmation request, waiting for the server to send the last data to be sent, as well as the connection release message.
Third wave

The server sends a FIN to the client and then closes the client connection. So FIN is equal to 1, and ack is the same u plus 1. The server may be sending some more data in the semi-shut state, so instead of using seq=v, let’s say seq=w.

The server enters the LAST_ACK state and waits for confirmation from the client.
Fourth wave

The client needs to confirm that it has received all data, so it sends an ACK =1, ACK =w+1. The sequence number is the previous ACK confirmation number, which is seq=u+1.

The client then enters the time-wait state. At this time, the TCP connection is not released, and the client enters the CLOSED state after 2∗MSL (maximum packet segment life) and the TCB is revoked.

The server will enter the CLOSED state as soon as it receives the confirmation from the client. As you can see, the server ends the TCP connection earlier than the client.

Some problems with the four handshakes

Why four waves?

Since it was the client that asked you to close the connection, it would be too abrupt if you immediately agreed. If the client suddenly shuts down, it should give the server some time to react. It may still have some data to send.

Therefore, the server sends an ACK message to acknowledge receipt of the request, which tells the client: Oh, I see, but wait a minute! I have something else for you. Then brush the hair data.

Then we can send a FIN packet to formally close the connection. It’s like telling the client: I’m done and I can close it.

Then this is not the end, you have received the data I sent to tell me ah, if not, I will send again. Therefore, the client also needs to send an acknowledgement message to the server, indicating that I really received it!

Why wait for 2MSL?

2MSL is the event set by the time-wait timer, and MSL is the maximum packet segment life. The maximum duration for which any packet exists on the network before being discarded. Two minutes is generally recommended, but for today’s networks two minutes may be a bit long, so a smaller MSL value is also allowed.

Now, using 2 minutes as an example, it would take 4 minutes after the client enters the time-wait state to enter the CLOSED state and start the next connection. Then the client cancels the corresponding transfer control block TCB to end the TCP connection.

There are two reasons to wait for 2MSL:

To ensure that the last ACK packet segment sent by the client can reach the server.

Because the ACK segment may be lost, the server in the last-ACK state cannot receive the acknowledgement, so it waits and waits for a certain amount of time (the waiting time should be longer than MSL to confirm the packet loss) and retransmits the FIN-ACK segment. In this case, the client will receive the retransmitted ACK segment within the 2MSL wait time, and then the client will retransmit the ACK segment and restart the timer. Finally, the client and server enter the CLOSED state.

Then, if the client does not wait and sends the ACK, the connection is closed. When the packet is lost, the server retransmits the FIN-ACK packet segment, the client cannot receive it, and the ACK is not transmitted, and the server cannot enter the closed state. The server can be closed only after receiving an ACK packet from the client.
Prevents invalid link requests from appearing in this connection.

If some data is stuck in the network during the last connection and arrives at the server after a new connection is established, the server will think that the delayed data belongs to the new connection and the packet will be confused.

After the client sends the last ACK segment, it can make all segments generated during the duration of the connection disappear from the network (because the maximum lifetime of the packet is only MSL), thus preventing the old connection request segment from appearing in the next new connection.

SYN (flood) attack

Principle of flood attack

If the client sends a SYN-ACK packet to the server and hangs, the server returns a SYN-ACK packet, but does not receive an ACK packet from the client, the server is in the half-connected state, so the server waits for an ACK packet from the client. A large number of these half-connected invalid connections can exhaust the server’s SYN connection queue. Then the normal connection will have no place and will be rejected.

Fortunately, the server has a timeout period, after which the server will disconnect without waiting for you. Currently, LINUX sends syn-ACK packets five times by default. The interval for sending syn-ACK packets is 1s, 2s, 4s, 8s, and 16s. After the fifth packet is sent, it takes 32 seconds to determine whether the fifth packet times out.

So it takes a total of 63 seconds, about a minute, for TCP to disconnect. This gives the attacker an opportunity to attack the server by sending a large number of SYN packets in a short period of time to exhaust the server’s SYN queue.

What is a SYN attack

In a SYN attack, a large number of non-existent IP addresses are forged in a short period of time and SYN packets are repeatedly sent to the server. The server replies with an acknowledgement packet and waits for the customer’s confirmation. At this point the connection is half open, swallowing server resources. Legitimate users attempted to connect to the server but were denied due to blocking service attacks.

Because the source address does not exist, the server needs to continuously resend until timeout. These forged SYN packets occupy the unconnected queue for a long time, and normal SYN requests are discarded, resulting in slow running of the target system, or even network congestion and system breakdown in serious cases. SYN attack is a typical DoS attack.

How do I detect SYN attacks?

When you see a lot of semi-connected states on the server and the source IP address is random, it’s basically a SYN attack.

How do I defend against SYN attacks?

SYN attacks cannot be completely blocked unless TCP is redesigned. We minimize the damage caused by SYN attacks. Common SYN attack defense methods are as follows:

Shorten SYN Timeout
Increase the maximum number of connections
The filtering gateway defends against SYN
Cookies technology

Finally: : this thing wrote for two days, because the knowledge point is really too much, write to write to find a new knowledge point, write to finally have no strength to think. Take a moment to reflect.

Reference article:

Computer Networks (7th edition) – Xie Xiren
Why does TCP have 3 handshakes and 4 waves? Can’t you shake hands twice?
Illustrated HTTP
Description of TCP/IP protocol family four-layer model
The phase of the HTTP request-to-response journey
A complete HTTP request process
How does a browser request go from sent to returned?
What happens from entering the URL to loading the page?
What is the TCP/IP protocol?
Deeply understand TCP three-way handshake, analyze the source code and track the handshake process
Understand TCP in depth: The three-way handshake
TCP three-way handshake in depth
TCP three-way handshake and four-way wave are colloquially understood
No reference but very detailed article, interested can understand: DNS resolution address
Why invent IP addresses when you can identify a computer with a MAC address?
Explain the MAC address table, ARP table, and routing table during network transmission
What are the differences and relationships between IP addresses and MAC addresses?
Why use a MAC when you have an IP?
Why use IP when you have a MAC?
TCP three times handshake four times wave understanding and interview questions
Network – a little confused about ACK
Why ack=seq+1 on three handshakes
20-1-TCP Connection — Initialize the sequence number (ISN)
Talk about TCP protocol, finally understand what it is
Packet Oriented (UDP) and Byte Stream Oriented (TCP)
What is the relationship between TCP byte stream and packet segment? – Che xiaopang’s answer – Zhihu