1, define,

TCP, Transmission Control Protocol. As the most important protocol in the transport layer, it plays many roles in the Internet because of its reliable transmission characteristics. The TCP application layer protocols include HTTP, HTTPS, FTP, SMTP, and DNS. TCP is used in browser, file transfer, and email…… TCP is indispensable in all scenarios that require reliable transmission.

2, the characteristics of

1. TCP is connection-oriented. Before an application can use TCP, it must establish a link. After the data transfer is complete, the established connections must be released. Just like when we make a phone call, we need to dial the number to establish a connection before the call and hang up to release the connection when the call is over.

2. TCP provides reliably delivered services. Data transmitted through TCP is not lost, repeated, or error-free.

3. TCP connections are point-to-point. Each TCP connection can have only two endpoints, called sockets. In dotted decimal notation, the IP address is followed by the number of the upper end, separated by colons (;) or commas (,). For example, 192.168.4.5:80 indicates a socket.

4. TCP is byte stream oriented. A stream is a sequence of bytes flowing into or out of a process. Byte stream oriented means that although the application interacts with TCP one data block at a time, TCP treats the data passing through the application layer as a series of unstructured byte streams in the send buffer. If the data is too large, it needs to be fragmented before being sent. The size of the data sent has no relation to the size of the packets from the application layer. Therefore, TCP is byte stream oriented. UDP does not have a buffer. Packets from the application layer are directly added to the header and sent to the network layer, which is responsible for fragmentation. Therefore, UDP is packet-oriented.

3. TCP header

The TCP header occupies 20 to 60 bytes, of which 20 bytes are fixed length of the header.

Source port, destination port

Like UDP, each takes two bytes.

The serial number

It’s 4 bytes. During transmission, each byte is assigned a number. After the connection is established, the serial number represents: the number of the first byte of the TCP data portion passed to the other party this time.

Confirmation no.

It’s 4 bytes. After a connection is established, the acknowledgment number represents the number of the first byte of the TCP data portion expected to be sent next.

Data migration

It occupies four bits. The value ranges from 0x0101 to 0x1111. Multiply this number by 4 to get the length of the header, which is 20-60 bytes. The length of the head is called the data offset. It is easy to understand that the head is arranged in front of the data part, and the amount of the data part offset is the length of the head.

Keep a

Occupy 6 bits, all zeros currently.

window

It takes 2 bytes. It is used for traffic control to inform the other party of the size (in bytes) of the data to be sent next time.

Inspection and

As with UDP, the verification and calculation method: false header + header + data part. The dummy header only works when calculating the sum and is not passed to the network layer.

Pointer to an emergency

It takes 2 bytes. Indicates the number of bytes of emergency data in the packet segment. Because the emergency data is placed before the ordinary data, the emergency pointer also indicates the position of the end of the emergency data in the packet segment. Note that the emergency pointer only works when URG=1.

options

The length is variable. The maximum length is 40 bytes. Used to store other auxiliary data, such as selection confirmation data.

Sign a

The 6 flag bits can be said to be the core of the TCP header data, used to illustrate the nature of this paragraph.

URG: Urgent. When URG is 1, the data in the emergency pointer takes effect, indicating that some parts of the packet data need to be processed urgently.

ACK: Acknowledgment, Acknowledgment bit. The confirmation number field is valid only when ACK=1.

PSH: Push bit. When two processes communicate with each other in an interactive manner, you can set PSH=1 to receive an immediate response after entering a command. In this case, the receiver pushes the PSH=1 packet to the application process as soon as possible, rather than waiting for the receive cache to fill up.

RST: Reset. When RST=1, it indicates that a serious error occurs in the connection. The connection must be released and then re-established.

SYN: Synchronization bit. Used to synchronize serial numbers when a connection is established. When SYN=1, ACK=0, this is a connection request. If the peer party agrees to establish a connection, it replies with SYN=1 and ACK=1.

FIN: Finish, indicating the end bit. If FIN=1, data is sent and the connection needs to be released.

Why is the TCP header so complex?

It’s all about the reliable transmission of data.

4, TCP several basic problems

4.1 Reliable Transmission

One of the most important features of TCP is reliable transmission. Our channel is generally unreliable, how to achieve reliable transmission in unreliable transmission channel? Let’s start with the simplest stop-wait protocol.

Stop waiting for ARQ

Stop waiting: Stop sending after each packet data is sent and wait for the confirmation of the other party. Send the next packet data after receiving acknowledgement. Here are four situations that need to be dealt with to stop waiting for a protocol.

A) In the case of no error, each packet data sent by A can be confirmed by B, and the cycle repeats. B) If there is an error in the data sent by A to B, B will discard the data of A, or B will not send an acknowledgement to A if B has not received the data sent by A at all. If user A does not receive an acknowledgement from user B after A certain period of time, user A triggers the timeout retransmission mechanism.

C) if B is indeed received A data, but the B to send A confirmation in the midway lost, A after A certain period of time, still not received the confirmation B, will trigger A timeout retransmission, B will once again received from A same data, but the data will be discarded, B only to send A confirmation. D) if B is indeed received A data, B to send A confirmation nor lost, but because the routing and the reason for the delay, send A confirmation of B has failed to communicate, A after A certain period of time, would trigger A timeout retransmission, B will once again received from A same data, but the data will be discarded, B only to send A confirmation. After A period of time, the late confirmation was finally delivered to A, but A did nothing after receiving the confirmation, because B’s confirmation had been processed once just now.

Using the acknowledgement and retransmission mechanism above, we can achieve reliable transmission over unreliable channels, but this mechanism also has an obvious problem: after sending one packet data, we have to wait for confirmation before sending another packet data, which is like a single thread operation, which is inefficient. In this case, we introduce the following continuous ARQ protocol and sliding window protocol.

Continuous ARP + sliding window protocol

In order to improve transmission efficiency, we can use pipeline transmission. The sender can send multiple packets consecutively at one time without having to pause for confirmation each packet, so that data can be continuously transmitted on the channel. This is the continuous ARQ protocol.

Sender rule: Maintains a window in which packets can be sent consecutively without waiting for confirmation. Recipient rule: After receiving several packets, the recipient sends an acknowledgement to the last packet that arrives in sequence. This means that all previous groups have been received correctly. Window sliding rule: Assume that the difference between the last received confirmation number minus the current confirmation number is 4. The sender receives this confirmation and moves the window four blocks forward.

During TCP communication, if a packet is lost in the sending sequence (for example, 3 of 1, 2, 3, 4, and 5 is lost), TCP retransmits the last confirmed packet and subsequent packets (the last confirmed packet is 2, and retransmits 3, 4, and 5). This operation is also called go-back-n, indicating that the N packets that have been sent need to be retransmitted again. In this way, packets that have already been correctly transmitted may also be sent repeatedly (e.g., 4 or 5), which degrades TCP performance.

SACK(Selective Acknowledgment) and Selective acknowledgment technology were developed to improve the above situation.

Selective confirmation SACK

Selective acknowledgement technique: tells the sender which data is missing and which data has been correctly received, so that the sender only needs to resend the lost packet rather than resend all packets following the lost packet.

SACK information is placed in the options section of the TCP header.

Kind: 1 byte. A value of 5 indicates that this is the SACK option. Length: contains 1 byte. Indicates how many bytes the SACK option occupies. Left Edge: 4 bytes, Left Edge. Right Edge: 4 bytes, Right Edge.

Here’s an example to help understand the left and right boundaries:

Suppose the receiver’s receive window is 1000 bytes, and the sender sends 1000 bytes in 10 packets. The brown packets are received by the receiver, and the white packets are not received.

1. The data on the left of 201 have been received accurately, so the confirmation number is 201. Next time, the sender will send the packet from 201. 2, [301,401), [501,601), [701,801), [901,1001) represent the received grouping fragment with left and right boundaries respectively, paying attention to the left and right boundaries. The receiver tells the sender these left and right boundary information, and the sender knows which groups to resend.

A pair of boundary information (left + right) takes up 8 bytes, and the options section of the TCP header is up to 40 bytes, so the SACK option carries up to 4 sets of boundary information.

The maximum number of bytes occupied by the SACK option: 4 * 8 + 2 = 34.

💡 doubt

1. If a packet is retransmitted and still fails, will it continue to be retransmitted until the retransmission succeeds? A: It depends on the system setup. For example, after retransmission fails for five times, some systems send a Reset message to re-establish a connection.

2. If the receiving window can receive up to four groups, but the sender only sends two groups this time, how can the receiver know whether there are two more groups? A: After A period of time, if the receiver does not receive the next two packets, the receiver sends an acknowledgement packet to the sender that the two packets are received.

3. Both the transport layer and the network layer have the ability to fragment data. Why is the data fragmented at the transport layer rather than at the network layer? A: Because the retransmission performance can be improved. To be clear, reliable transport is controlled at the transport layer. If the data is not sharded at the transport layer, if the data is lost, the entire transport layer data needs to be retransmitted. If the fragment is sharded at the transport layer, only the missing fragment needs to be retransmitted.

4.2 Flow Control

In general, we want data to move faster, but if the sender sends data too fast, the receiver may not have time to receive it, resulting in data loss. The so-called flow control, is to let the sender send rate is not too fast, to let the receiver in time to receive.

Principle:

1. You can control the size of the window and the sending rate of the sender by confirming the window field in the packet.
2. The window size of the sender cannot exceed the window size given by the receiver.
3. If the window size given by the receiver is 0, the sender will stop sending data.

The following example illustrates how to control flow if you control the size of the send window.

Suppose the receiving window given by B is RWND =400 when the connection is established. There are three times of flow control in the figure. The first RWND =300, the second RWND =100, and the third RWND =0.

💡 doubt

After sending a zero-window packet to the sender, the receiver has some storage space and sends a non-zero window packet to the sender. However, the non-zero window packet is lost in the sending process. In this way, the sender has been waiting for the non-zero window message from the receiver, and the receiver has been waiting for the data from the sender, so the two sides are in an impasse.

To solve this problem, after receiving the zero-window message from the receiver, the sender will first stop sending data to the receiver, and at the same time, it will start a timer and ask the size of the window of the receiver every once in a while until the size of the window of the receiver is not zero.

4.3 Congestion control

Function: Prevents too much data from being injected into the network and prevents overload of routers or links on the network.

Congestion on the Internet is similar to congestion on daily traffic. Look at the chart below:

Let’s say you can send up to a thousand copies of data simultaneously on the network, and ideally, even if you send 10,000 copies, 100,000 copies simultaneously on the network, that’s still a thousand copies of data. However, when the throughput reaches a certain level, as more data is transmitted on the link, the transmission rate becomes slower and slower until the link is completely blocked.

Just like our daily traffic, congestion control is a global process that involves all hosts, routers, and other factors that degrade network performance. Flow control, on the other hand, is point-to-point control.

There are four algorithms for congestion control: slow start, congestion avoidance, fast retransmission, and fast recovery.

Slow start

Start slowly, as the name suggests. Start slowly. When the host starts to send data, it is not clear about the network load. If a large amount of data is immediately injected into the network, the network may be congested. Experience has proved that the better method is to first detect and gradually increase the sending window from small to large, that is, gradually increase the value of congestion window from small to large.

MSS: Maximum Segment Size specifies the Maximum Size of each Segment. SWND: send window. RWND: receive window. 4. CWND: Congestion window.

Where, the size of the sending window is equal to the minimum value of the receiving window and the congestion window: SWND = min(CWND, RWND).

The initial CWND value is small and then grows exponentially as the packet is acknowledged (ACK is received) by the receiver.

Congestion avoidance

Congestion avoidance: When the congestion window increases to a certain threshold, it increases in a linear manner to prevent premature congestion.

Ssthresh: Slow Start threshold. When CWND reaches the threshold, it increases in a linear manner. Increase in addition: The congestion window slowly increases to prevent premature network congestion. Multiplication reduction: As soon as there is congestion on the network, ssTHRESH is reduced to half of the peak congestion, and the slow start algorithm is executed, and CWND is restored to the initial value. Ssthresh drops quickly when there is frequent congestion on the network.

Fast retransmission

The real and fake networks are congested

Sometimes, individual packet segments are lost on the network, but the network is not congested. If the sender does not receive an acknowledgement for a long time, a timeout occurs and the network is mistaken for congestion. This causes the sender to mistakenly start the slow start and set CWND to 1 again, reducing transmission efficiency.

Fast retransmission algorithm

1. For the receiver, after receiving a packet out of order, the sender should immediately send repeated confirmation, so that the sender can know that a packet has not arrived in time, instead of waiting for their own data to send confirmation.

2. As long as the sender receives three consecutive 3-ACK packets, it should immediately retransmit the unreceived packet segment without waiting for the retransmission timer to expire.

As shown in the figure above, the receiver received M1 and M2 and sent a confirmation message, but M3 was lost. Assume that the receiver receives M4 again later. According to the fast retransmission principle, the receiver should immediately send an acknowledgement of “last successfully received data “(M2). Similarly, the confirmation of M2 should be sent immediately after receiving M5 and M6. The sender knows that M3 is missing after receiving 3 consecutive confirmations of M2. Resend M3 immediately. In this way, there will be no send timeout and the sender will not be mistaken for network congestion.

Fast recovery

The sender knows that only individual segments are now missing, so instead of starting a slow start, it executes a fast recovery algorithm.

1. So let’s do the multiplication reduction algorithmssthreshReduced to half of peak congestion. And thecwndValue is set to newssthreshValue.
2. The difference from slow start is that the slow start algorithm is now not executed, i.ecwndNow instead of going back to the original value, I’m going to takecwndValue is set to newssthreshValue. Then start performing the congestion avoidance algorithm (addition augmentation) to makecwndIt increases slowly and linearly.

Note that fast retransmission and fast recovery are features of TCP Reno versions, which are different from older VERSIONS of TCP Tahao.

The graph below shows a combination of slow start, congestion avoidance, fast retransmission, and fast recovery.

Based on the above, TCP congestion control can be summarized in the following flow chart.

5. Connection management

The TCP connection establishment process must solve the following problems:

1. Make sure each party is aware of the other’s presence and that the other’s ability to send and receive is normal.
2. Both parties negotiate parameters such as: MSS, whether to use SACK, window size, window scaling factor, etc.

5.1 Three handshakes

The PROCESS of establishing a TCP connection is called handshake. Three TCP packet segments are exchanged between the client and the server. The following figure shows the process of establishing a TCP connection through the three-packet handshake.

CLOSED: the server is CLOSED. LISTEN: The server is in the listening state and waiting for the client to connect to SYN-sent: The client sends a SYN packet and waits for the second handshake from the server. Syn-rcvd: the server receives a SYN packet. After receiving ACK packets from the client, the client enters the ESTABLISHED state. ESTABLISHED: Indicates that the connection is ESTABLISHED

💡 doubt

1. Why three handshakes? How about twice? One might think, the client asks the server to establish a connection, the server agrees to establish a connection, and the server also asks the client to establish a connection, and the client agrees to establish a connection, so that the two handshakes will be enough. Why give the server a confirmation, three handshakes?

From the point of view of saving resources, this prevents the server from wasting resources by waiting. When the client requests to establish a connection to the server for the first time, retransmission is triggered due to network timeout. Then, the client requests to establish a connection to the server for the second time. This time the connection was established, the data was transferred, and the connection was disconnected. A long time after the connection is disconnected, the first request that was stranded on a network node finally reaches the server. Originally, this request is invalid, but the server considers this request as a new request created by the client and sends an acknowledgement to the client. If only two handshakes are required, the server enters the connected state. But the client does not want to establish a connection per se, so it does nothing after receiving confirmation from the server. However, the server mistakenly thinks that it has entered the connection state and still waits for the client to send data, wasting resources.

If the three-way handshake is used, the server knows that the client does not intend to establish a connection after receiving no further confirmation from the client.

1. The client sends a connection request to the server. If the server can receive the request, the server knows that there is no problem with its receiving ability. 2. The server then sends an acknowledgement to the client. If the client can receive it, the client knows that its ability to send and receive messages is ok. 3. At this point, the server does not know whether its ability to send messages is normal, so the client needs to send an acknowledgement to the server. After receiving the acknowledgement, the server knows that its ability to send messages is ok.

That’s why you need three handshakes.

💡 What happens if the third handshake fails?

After the second handshake, the status of the server is SYN-RCVD. If the ACK packet cannot be received from the client, the server resends the SYN+ACK packet. If the server fails to wait for the client’s ACK after repeated SYN+ ACKS, it sends an RST packet to forcibly close the connection.

5.2 Wave four times

After the data transfer ends, both sides of the communication can release the connection. Now both A and B are in ESTABLISHED state.

Fin-wait-1: indicates that it wants to close the connection. A FIN packet is sent to the peer and the peer enters the FIN-WaIT-1 state. Procedure

Close-wait: indicates that the system is waiting to CLOSE. After the FIN is sent to the peer, the peer sends an ACK packet to the peer and enters close-wait state. In this state, you need to check whether you still have data to send to the peer party. If no, you need to send FIN packets to the peer party.

Fin-wait-2: The active party stays in FIN-WAIT-2 state and waits for the FIN packet to be sent after the peer party sends an ACK acknowledgement.

Last-ack: Passively closes a FIN packet sent by a party and waits for the ACK packet sent by the other party. After receiving an ACK packet, the system enters the CLOSED state.

Time-wait: indicates that a FIN packet is received and an ACK packet is sent. The state can be CLOSED after 2MSL. If the FIN-WaIT-1 state receives a packet with both the FIN flag and ACK flag, the packet enters the time-wait state without passing through the FIN-WaIT-2 state.

After sending an ACK, the client needs to WAIT for a period of TIME before actually closing the connection. The Maximum Segment Lifetime is the Maximum Segment Lifetime. MSL is the maximum lifetime of TCP packets on the Internet. Each specific TCP implementation must choose a certain MSL value, and RFC 1122 recommends 2 minutes. Can prevent the packets generated in this connection from being mistransmitted to the next connection (because the packets in this connection will disappear in 2MSL time)

If the client immediately releases the ACK and the server does not receive the ACK due to network reasons, the server resends the FIN.

Here’s what might happen:

1. If no response is received from the client, the server may wait or even resend the FIN for several times, wasting resources.
2. The client has a new application that happens to be assigned the same port number. Upon receiving the FIN, the new application immediately starts to disconnect from the server, possibly because it wants to establish a connection with the server.

💡 doubt

1. Why do I wave my hand 4 times to release the connection?

Because TCP is in full duplex mode. Both hosts in communication need to be disconnected. If host 1 sends a FIN message to host 2, the FIN has no data to send to host 2, and Host 2 can still send data to host 1.

First wave: When host 1 sends a FIN packet, host 1 tells host 2 that it has no more data to send. However, host 1 still accepts the data from host 2.

Second wave: When host 2 returns an ACK packet, it indicates that host 2 knows that host 1 has no data to send, but host 2 can send data to host 1.

Third wave: When host 2 also sends a FIN packet, host 2 tells host 1 that host 2 has no more data to send.

Fourth wave: When host 1 returns an ACK packet, host 1 knows that host 2 has no data to send. Then the entire TCP connection is officially disconnected

2. Why do I sometimes see only 3 waves when USING the packet capture tool?

This is a combination of the second and third waves. When the server receives the FIN from the client, if the server has no more data to send to the client, the client can merge the second and third waves and tell the client two things:

1. It is already known that the client has no data to send
2. The server also has no data to send

6, summary

This article mainly introduces several core knowledge of TCP protocol, including TCP header, several basic problems, connection management, etc., hoping to bring help to you. The following articles will continue to help you and help you to build a network of knowledge.