What is “Three handshakes, four waves”?
TCP is a connection-oriented unicast protocol. Before sending data, communication parties must establish a connection.
The so-called “connection” is actually a piece of information about each other, such as IP address and port number, stored in the memory of the client and server.
TCP can be thought of as a byte stream that handles packet loss, duplication, and errors at the IP layer and below.
During the establishment of a connection, the two parties need to exchange some connection parameters. These parameters can be placed in the TCP header.
TCP provides a reliable, connection-oriented, byte stream, transport layer service that establishes a connection using a three-way handshake. Use four waves to close a connection.
TCP service model
Now that we’ve looked at the “three handshakes and four waves” of establishing and closing a connection, let’s take a look at TCP.
A TCP connection consists of two IP addresses and two port numbers. A TCP connection is typically divided into three phases: start, data transfer, and exit (close).
When TCP receives data from the other end, it sends an acknowledgement, but this acknowledgement is not sent immediately, usually after a while.
Acks are cumulative, and an ACK with an acknowledgment byte number N indicates that all bytes up to N (excluding N) have been received successfully.
The advantage of this is that if one ACK is lost, it is likely that subsequent acks will be sufficient to confirm the previous packet segment.
A complete TCP connection is bidirectional and symmetric, and data can flow equally in both directions, providing a duplex service to upper-layer applications.
Once a connection is established, each TCP segment in one direction of the connection contains an ACK for the segment in the opposite direction.
The sequence number enables a TCP receiver to discard duplicate segments and record the segments that arrive in disordered order.
Because TCP uses IP to transport segments, IP does not provide the ability to eliminate duplicates or ensure correct order.
TCP, on the other hand, is a byte stream protocol that never sends data to upper-layer programs in a random order.
So the TCP receiver is forced to hold data with a large sequence number from the application until the missing segment with a small sequence number is filled.
The TCP header
The source port and destination port determine the processes of both parties at the TCP layer. The sequence number indicates the first byte number in the packet segment data, and the ACK indicates the confirmation number.
The next sequence number that the sender expects to receive is the last successfully received data byte sequence number plus 1. This field is valid only if the ACK bit is enabled.
When a new connection is created, the SYN bit of the first packet segment sent from the client to the server is enabled. This is called the SYN segment.
The sequence number field contains the first sequence number to be used in this direction of the connection, the initial ISN, after which the ISN plus 1 is sent.
So the SYN bit field consumes a sequence number. This means that retransmission is used for reliable transmission, whereas ACK without consuming serial numbers is not.
The header length (the data offset in the figure) is in 32-bit words, that is, in 4bytes. It has only 4 bits and the maximum size is 15, so the maximum header length is 60 bytes and the minimum is 5, that is, the minimum header length is 20 bytes (the variable option is empty).
- ACK — confirm, making the confirmation number valid.
- RST — Resetting the connection (often seen as reset by peer) is the result of this field.
- SYN – The sequence number used to initialize a connection.
- FIN: The sender of the packet has stopped sending data to the other party.
When a connection is established or terminated, the segment exchanged contains only the TCP header and no data.
State transition
The transitions between the three handshakes and the four waves are shown below:
Why “Three handshakes, four waves”
Three-way handshake
Put the three-way handshake in a more understandable light.
The client and server need to connect before communication. The “three-way handshake” is used to ensure that the receiving and sending capabilities of the client and the other side are normal.
First handshake: The client sends a network packet and the server receives it. In this way, the server can conclude that the sending capability of the client and the receiving capability of the server are normal.
Second handshake: The server sends the packet and the client receives it. In this way, the client can conclude that the receiving and sending capabilities of the server and the client are normal.
From the perspective of the client, I received the response packet sent by the server, indicating that the server received the network packet sent by me during the first handshake and successfully sent the response packet, which indicates that the receiving and sending capabilities of the server are normal.
On the other hand, I received the response packet from the server, indicating that the network packet I sent for the first time successfully reached the server, so that my own sending and receiving capabilities are normal.
Third handshake: The client sends the packet and the server receives it. In this way, the server can conclude that the receiving and sending capabilities of the client and the sending and receiving capabilities of the server are normal.
After the first and second handshakes, the server does not know whether the receiving capability of the client and its own sending capability are normal.
On the third handshake, the server receives the client’s response to the second handshake. From the server’s point of view, my response data from the second handshake is sent out, and the client receives it. So, my ability to send is normal. The reception capacity of the client is also normal.
After the above three handshakes, both the client and server confirm that their receiving and sending capabilities are normal. Then you can communicate normally.
Each time, the person receiving the packet can draw some conclusions, while the person sending the packet has no clue.
Although I have the action of sending a packet, but how do I know I have sent, and the other party has received it?
As you can see from the above procedure, a minimum of three handshakes are required. Both failed to reach the conclusion that their own and the other’s ability to receive and send were normal.
In fact, each time the party receiving the network packet can at least get: the other party’s sending, our receiving is normal.
Each step is related, and the next “response” is triggered by the first “request,” so each handshake can lead to additional conclusions.
For example, if the server receives data packets during the third handshake, it indicates that the server can only obtain the sending capability of the client and the receiving capability of the server.
However, in combination with the second time, it indicates that the client received the response packet sent by the server in the second time and made a response, thus obtaining the additional conclusion: the client received and the server sent are normal.
To summarize:
Four times to wave
TCP connections are two-way peer-to-peer, which means that both parties can send or receive data to each other at the same time.
When either party wants to close the connection, it sends a command to tell the other party that I’m closing the connection. An ACK is sent back, and the connection in one direction is closed.
But the other direction can continue to transmit data, and when all data is sent, a FIN segment is sent to close the connection in that direction. The receiver sends an ACK to confirm closing the connection.
Notice that the party that receives a FIN packet can reply only one ACK, but cannot immediately return a FIN segment to the other party. The “instruction” to end data transmission is given by the upper application layer. I am only a porter, and I cannot understand the “will of the upper layer”.
How to complete “three handshakes, four waves”?
In fact, the purpose of the three-way handshake is not only to let the communication parties know that a connection is being established, but also to use the options of the packet to transmit special information and exchange the initial SEQUENCE number ISN.
The three-way handshake means that three packet segments are sent, and the four-way wave means that four packet segments are sent. Note that both SYN and FIN segments are reliably transmitted using retransmission.
Principle of three-way handshake:
- The client sends a SYN segment with an initial sequence number, ISN(c), for the client.
- The server replies with its SYN segment, specifying its ISN(s). To confirm the client’s SYN, the ISN(c)+1 is used as the ACK value. Each SYN is sent, the sequence number is incremented by one, and if it is lost, it is retransmitted.
- To confirm the SYN on the server, the client returns the ISN(s)+1 as the ACK value.
Four wave principle:
- The client sends a FIN segment with its current serial number K that it wants the recipient to see. It also contains an ACK to confirm the latest data sent by the other party.
- The server adds K to 1 as the ACK sequence number, indicating that the last packet was received. The upper application is notified that the other end initiated a shutdown, which usually causes the application to initiate its own shutdown.
- The server initiates its own FIN segment, ACK=K+1, Seq=L.
- Client confirmation. ACK=L+1
Why is it three handshakes to establish a connection, but four waves to close it? This is because in LISTEN state, the server receives a SYN packet for establishing a connection and sends the ACK and SYN packets to the client.
When a FIN packet from the peer party is received, the peer party only stops sending data but can still receive data. The upper-layer application determines whether to close the data transmission channel. Therefore, the PEER ACK and FIN packets are sent separately.
“Three handshakes, four waves” advanced
ISN
An important function of the three-way handshake is that the client and server exchange ISN(Initial Sequence Number) so that the other side knows how to assemble the data by Sequence Number when they receive the data next.
If the ISN is fixed, an attacker can easily guess the subsequent confirmation number:
ISN = M + F(localhost, localport, remotehost, remoteport) Copy the code
M is a timer that increments by one every four milliseconds. F is a Hash algorithm that generates a random value based on the source IP address, destination IP address, source port, and destination port. Make sure that the Hash algorithm cannot be easily extrapolated externally.
Serial number winding
Because the ISN is random, the serial number can easily exceed 2^31-1. TCP’s judgment on packet loss and disorder depends on the comparison of sequence numbers.
TCP sequence wraparound (TCP sequence wraparound)
/*
* The next routines deal with comparing 32 bit unsigned ints
* and worry about wraparound (automatic with unsigned arithmetic).
*/
static inline int before(__u32 seq1, __u32 seq2){
return (__s32)(seq1-seq2) < 0;
}
#define after(seq2, seq1) before(seq1, seq2) Copy the code
The code above is the code in the kernel to resolve the wrap problem. __s32 stands for signed integer, while __u32 stands for unsigned integer.
When the sequence number is wound, the sequence number becomes smaller, and when subtracted, the result becomes a signed number, so the result is negative.
Suppose that SEQ1 =255 and seq2=1 (winding occurs). Seq1 = 1111 1111 seq2= 0000 0001 We want the comparison result to be seQ1 - seQ2 = 1111 1111-0000 0001 ----------- 1111 1110 since we converted the result to a signed number, Since the highest bit is 1, the result is a negative number, and the absolute value of a negative number is 0000 0001 + 1 = 0000 0010 = 2 so seQ1 - seq2 < 0Copy the code
SYN Flood attack
The most basic DoS attack is to use reasonable service requests to occupy too many service resources, so that legitimate users can not get the response of the service. SYN Flood is a Dos attack.
If a large number of SYN packets are maliciously sent to a server port, the server can open a large number of half-open connections and allocate Transmission Control blocks (TCBS), consuming a large number of server resources and failing to respond to normal connection requests.
After a TCP port is opened, the port is in the Listening state and continuously monitors the SYN packets sent to the port. Once a SYN packet is received from the Client, a TCB needs to be allocated to the request.
A TCB typically requires at least 280 bytes, and on some operating systems 1,300 bytes, and returns a SYN ACK command that immediately changes to a SYN-received half-open connection state, which deplets the system’s resources. Common anti-attack methods are:
① Release invalid connection monitoring
Monitors half-open and inactive connections of the system and disconnects them when a certain threshold is reached, freeing system resources.
This method treats all connections equally. In addition, the number of half-open connections caused by SYN Flood is large, and normal connection requests are drowned and mistakenly released in this method. Therefore, this method is an entry-level SYN Flood method.
(2) Delay TCB allocation method
Server resources are consumed because TCBS are allocated as soon as SYN packets arrive.
However, it is difficult to establish a normal connection for SYN Flood. Therefore, allocating TCB after the normal connection is established can effectively reduce the consumption of server resources. Common methods are Syn Cache and Syn Cookie.
(3) the Syn Cache technology
When the system receives a SYN packet, it stores the half-connection information in a dedicated Hash table until it receives a correct ACK packet and then allocates the TCB. This overhead is much less than that of TCB. You also need to save the serial number.
(4) the Syn Cookie technology
The Syn Cookie technique uses no storage resources at all, which is a clever approach that uses a special algorithm to generate Sequence numbers.
This algorithm takes into account the fixed information of IP, port, IP and port of the other party, as well as some fixed information that the other party cannot know.
For example, MSS(Maximum Segment Size) indicates the Maximum length of a TCP packet, excluding the length of the TCP header. Time, etc.
After receiving the ACK packet from the other party, it recalciates it to check whether it is the same as (Sequence number-1) in the reply packet from the other party, and decides whether to allocate TCB resources.
⑤ Use SYN Proxy firewall
One way is to prevent wall DQYWB connections from being valid after the firewall makes a SYN request to the internal server.
The SYN ACK packet sent by the firewall on behalf of the server uses the sequence number C, while the real server responds with the sequence number C ‘.
In this way, the serial number of each data packet is changed when it passes through the firewall. After confirming the connection security, the firewall sends the safe reset command. In this case, the Client reconnects to the firewall and permits SYN packets.
So you don’t need to change the serial number. However, the Client needs to initiate a two-handshake process, so the connection takes longer to establish.
The connection queue
When the external request arrives, the connection may be in the SYN_RCVD or ESTABLISHED state before it is finally sensed by the server, but not yet accepted by the application.
Similarly, the server maintains two types of queues: the half-connection queue in the SYN_RCVD state, and the full-connection queue in the ESTABLISHED state but not yet accepted by the application.
If the two queues are full, various packet loss situations occur:
Check whether any connection overflows netstat-s | grep LISTEN Copy the code
The half-connection queue is full
In the three-way handshake, the server maintains a half-connection queue, which creates an entry for each SYN packet from the client (the server creates a Request_SOCK structure to store in the half-connection queue when the server receives the SYN packet).
This entry indicates that the server has received a SYN packet, has sent an acknowledgement to the customer, and is waiting for one.
The connection identified by these entries is in the Syn_RECV state on the server. When the server receives an acknowledgement packet from the customer, the entry is deleted and the server enters the ESTABLISHED state.
In Linux, the syn-ACK packet is resent five times by default. The retry interval starts at 1s. The next retry interval is twice as long as the previous one.
It takes 32s for TCP to disconnect the TCP connection after sending the 5th timeout. Therefore, it takes 1s + 2s + 4s + 8s + 16s + 32s = 63s.
The SYN timed out takes 63 seconds. The attacker sends massive SYN packets to the Server in a short period of time to exhaust the SYN queues of the Server.
To handle SYN overload, Linux provides several TCP parameters to adjust the response:
- tcp_syncookies
- tcp_synack_retries
- tcp_max_syn_backlog
- tcp_abort_on_overflow
The full connection queue is full
On the third handshake, when the Server receives an ACK packet, it enters a new queue called Accept.
When the Accept queue is full, the Client does not respond even if it continues to send ACK packets to the Server.
At this point, ListenOverflows+1 and the Server determines how to return by tcp_abort_ON_overflow. 0 indicates that the ACK is discarded and 1 indicates that RST is sent to the Client.
The Client returns read Timeout or Connection reset by peer, respectively.
If tcp_ABORT_ON_overflow is 0, the Server will send SYN+ACK to the Client again after a certain period of time. If the Client timeout wait is short, an exception may occur.
If the client receives multiple SYN ACK packets, it considers that the previous ACK packets are lost. This prompts the client to send an ACK again and finally complete the connection when the Accept queue is free.
If the Accept queue is always full, the client receives the RST packet (at this point, the server sends SYN+ACK packets more times than the TCP_SYNack_retries).
The server simply creates a timer to retransmit the SYN and ACK to the server at regular intervals:
The command
[root@server ~]# netstat -s | egrep "listen|LISTEN"
667399 times the listen queue of a socket overflowed
667399 SYNs to LISTEN sockets ignored Copy the code
667399 times, which indicates the number of times the full connection queue overflowed, if executed every few seconds, the full connection queue must occasionally be full if this number keeps increasing.
Check whether the Accept queue overflows:
[root@server ~]# netstat -s | grep TCPBacklogDropCopy the code
Ss command:
[root@server ~]# ss -lnt
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:6379 *:*
LISTEN 0 128 *:22 *:* Copy the code
If State is Listen, send-q indicates that the maximum size of the full connection queue on the Listen port in column 3 is 50, and recV-q indicates how much the full connection queue is currently used.
In the non-Listen state, recv-q indicates the number of bytes in the receive queue. Send-q Indicates the bytes in the Send queue.
Summary: When external connection requests arrive, the TCP module first looks at the max_syn_backlog. If the number of connections in the SYN_RCVD state exceeds this threshold, incoming connections are rejected.
Discard or reset directly based on the tcp_ABORT_ON_overflow field.
On the Server side, the Server receives a SYN from the Client in the first step of the three-way handshake, puts the related information in the semi-connection queue, and replies with a SYN+ACK to the Client. Step 3 When receiving an ACK from the client, add the connection to the full connection queue.
Generally, the full connection queue is small and full first, while the half-connection queue is not full. If a SYN packet is received, the system enters the half-connection queue.
However, if step 3 (ACK) of the three-way handshake is received, the tcp_ABORT_ON_overflow field is used to determine whether to discard or reset directly.
At this point, the client sends an ACK, and the client thinks the three-way handshake is complete. It thinks the server is ready to receive the data.
However, the server may not be able to place the connection because the full connection queue is full, so it resends the SYN+ACK in step 2. If any data arrives, the server TCP module will queue the data.
If the Client does not receive any reply after a period of time, the Client automatically closes the connection.
“Three handshakes, four waves” Redis instance analysis
The Redis service is deployed on dev and port number is 6379. The following command is used to obtain data packets using tcpdump:
tcpdump -w /tmp/a.cap port 6379 -s0 -w Writes data to a file,-s0 Sets the size of each packet to 68 bytes by default. -s0 captures the complete packetCopy the code
Dev: 6379 (redis-cli) on dev2, send a ping to pong, stop packet capture, and read the captured packet with tcpdump:
Tcpdump - r/TMP/a.c ap - n - nn - A - x | vim - (- x in hexadecimal form features, the back analysis purposes)Copy the code
A total of seven packages were received.
The captured IP packet is divided into IP header and IP data part, and IP data part is TCP header and TCP data part.
The data format of IP is:
It consists of fixed length 20B+ variable length.
Unpack the meaning of the packet against the IP header format.
The rest of the data is related to TCP. TCP is also a 20B fixed length + variable length part.
For variable length, the protocol is as follows:
This completes the analysis of the first package. Dev2 sends a SYN request to dev. That’s the first of three handshakes.
SYN seq(c)=4133153791 Copy the code
The second package, dev response connection, ack=4133153792. Indicates that dev is next ready to receive a packet with this serial number for TCP byte annotation order control. The initial sequence number for dev(that is, server) is seq=4264776963, syn=1.
SYN ack=seq(c)+1 seq(s)=4264776963 Copy the code
The third package, client package validation, uses relative value replies. Seq =4133153792, equal to the ack of the second packet. Ack =4264776964.
ack=seq(s)+1, seq=seq(c)+1
Copy the code
At this point, the three handshakes are complete. The next step is to send the ping and pong data.
Then the fourth package:
The length of the TCP header is 32B, and the optional length is 12B. The total length of AN IP packet is 66 BYTES and the header length is 20 bytes. Therefore, part of the TCP data length is 14B. seq= 0xf65A EC00 =4133153792.
ACK, PSH. The data part is 2A31 0D0A 2434 0D0A 7069 6E67 0D0A:
0x2a31 -> *1
0x0d0a -> \r\n
0x2434 -> $4
0x0d0a -> \r\n
0x7069 0x6e67 -> ping
0x0d0a -> \r\n Copy the code
Dev2 sends the ping data to Dev, and the fourth packet is complete. The fifth package, Dev2 sends an ACK response to Dev.
The serial number is 0xFE33 5504=4264776964, and the ACK confirmation number is 0xf65A EC0E =4133153806=(4133153792+14).
In the sixth package, Dev responds to The Pong message to Dev2. The sequence number is FE33 5504, the confirmation number is F65A EC0E, the optional length of TCP header is 12B, the total length of IP datagram is 59B, and the length of the header is 20B. Therefore, the length of TCP data is 7B.
Data part 2B50 4F4e 470d 0a, which translates to +PONG\r\n. At this point, the analysis of the three-way handshake between Redis client and Server is completed.
conclusion
“Three handshakes, four waves” seems simple, but it can be extended to a lot of knowledge, such as semi-connected queue, fully connected queue and so on.
It’s easy to forget about the process of establishing and closing a TCP connection in the past, probably because you just memorized a few processes without delving into the underlying principles. So, “Three handshakes, four waves.” Do you really get it?
Note: This article is from the public account: Code nong Taohua Yuan. Author: Rao Quancheng