TCP optimization

What is TCP?

Transmission Control Protocol (TCP) is a connection-oriented, reliable, and byte stream-based transport layer communication Protocol.

TCP Three-way handshake

All TCP connections must start with three handshakes.

Three-way handshake process:

  • The client generates a random serial number x and sends a SYN packet to the server, asking if the server is ready.
  • The server returns x+1 to the client, generates a random sequence number y, and sends a SYN and ACK packet. Tell the browser I’m ready, are you ready?
  • After receiving the SYN and ACK packets, the client increments x and y by one and sends the last ACK packet during the handshake. Tell the server I’m ready, let’s link!

The client sends data immediately after sending an ACK group. The server must wait for the ACK group to respond to the data.

Why do you need three handshakes? . .

Reason: TCP is unreliable, but we want to establish reliable connections to send reliable data, and a three-way handshake is the theoretical minimum.

Let’s say instead of three handshakes what happens?

We substitute A for client and B for server

  • Suppose there is only one handshake, and A sends A packet to B, and it passes, and A doesn’t even know that B has received the packet… (If the network is disconnected, A will periodically time out and retransmit!!)
  • If there are two handshakes, B receives the SYN from A, and B sends the SYN+ACK to A, but B does not know whether A has received the acknowledgement message. (If the network is disconnected when B sends the SYN+ACK, the acknowledgement message sent by B is lost, and B will also retransmit the SYN after timeout!!)
  • If it’s A three-way handshake, and A sends an ACK to B, A and B confirm that they’re communicating, so they can send data. (If A fails to send the last ACK, it will still be retransmitted due to timeout!!)
  • If it’s a fourth handshake, we’ll find that we don’t need any more, and that’s a waste of resources.

Why tune TCP?

Each of our TCP connections requires a three-way handshake, and if the client is too far away from the server, it can have a significant performance impact.

So, the key to optimizing TCP is finding ways to reuse links.

Solutions: Keep-alive, TCP load balancing, and TFO are commonly used.

Long links (keep-alive)

Function: It is used to monitor whether the Data link between two Data works properly or prevent the link from being interrupted.

HTTP/1.1 is enabled by default, which means that multiple data packets can be sent continuously over a TCP connection without being disconnected.

Its setup requires support from the server, such as Nginx:

  • Keepalive_timeout =0: establish TCP connection + transmit HTTP request + execution time + transmit HTTP response + close TCP connection + 2MSL
  • Keepalive_timeout >0: establish TCP connection + (last response time – first request time) + close TCP connection + 2MSL

TCP also has its own keep-alive mechanism for detecting the state of a TCP connection:

  • Net.ipv4. keepaliveTime: Idle duration of KeepAlive, or the period of each normal heartbeat transmission (how many seconds after no data packets are transmitted on a TCP connection to start probe packets). The default value is 7200s (2 hours).
  • Net.ipv4. keepaliveIntvl: interval for sending KeepAlive probe packets (the interval between the last probe packet and the last probe packet) The default value is 75 seconds.
  • Net.ipv4. keepAliveprobes: Number of probes.

Load balancing

Suppose: A and B are both clients, and there is A load balancing server C in the middle, and then to the server D.

Principle: USER A and user C shake hands three times and send an HTTP request. User C checks whether there is A idle long connection before making A request to user D. If there is, user C uses it directly. After the request is completed, A and C negotiate to close the connection, but C and D still maintain the connection. When B and C perform a three-way handshake and send an HTTP request, C uses the idle connection with D to directly send an HTTP request. This avoids the delay and server resource cost of re-establishing TCP.

TFO (TCP Fast Open)

Although a persistent connection is enabled, there are still some requests to re-initiate the connection. After three handshakes, there will be a delay (RTT delay).

The goal of TFO is to resolve this delay and allow data to be exchanged during handshakes.

Basic principles:

  • The client sends a SYN, followed by an FOC request, of only four bytes.
  • The server receives the FOC request, generates a Cookie (8 bytes) based on the source IP address, loads the Cookie at the end of the SYN+ACK packet, and sends it to the client.
  • The client caches the Cookie for the next request.
  • At the start of the next request, the client sends a SYN packet, followed by a cached Cookie, and the data is officially sent.
  • The server verifies that the Cookie is correct, sends the data to the upper-layer application to process the result, and then sends the SYN+ACK. It does not wait for the client ACK to confirm, but sends the response data.

As we can see, the core of TFO is a secure Cookie, so the potential problem is that the client will send the data without returning an ACK after receiving a SYN packet from the server. At this point, the client should at least temporarily disable TFO on the corresponding connection path.

When a client retransmits a SYN packet after a timeout (or a server retransmits a SYN+ACK packet after a timeout), the Fast Open option should be removed. To avoid connection failure due to incompatibility.

Network congestion and slow start

What is network congestion?

Congestion: The demand for resources exceeds the available resources, and the network performance deteriorates. The throughput of the entire network decreases with the increase of the load, and even the phenomenon of congestion crash occurs.

To reduce network congestion, TCP adds a number of mechanisms to control the speed of two-way data transmission. Such as traffic monitoring, congestion control, congestion prevention mechanism, etc.

How to solve the problem of slow start and its principle?

We know that when the client and server establish a connection, the client does not know what bandwidth is available.

Therefore, you need an estimation mechanism that can then dynamically change the speed according to changing conditions in the network.

The specific implementation is as follows:

  • The sender initializes and maintains a congestion window variable (CWND) over a TCP connection. The maximum amount of data that can be transmitted between the sender and the receiver is specified as the minimum RWND and RWND of the receiving window.

  • The initial value of CWND was only one TCP segment. In 1999, it was increased to four TCP segments and in 2013, it was increased to 10 TCP segments.

  • After sending the TCP segment to the receiving end, the sender stops and waits for acknowledgement.

  • Thereafter, each time an ACK is received, the slow-start algorithm tells the sender that the CWND window has added a TCP segment and that two new packets can be sent. This stage is called the exponential growth stage.

Setting: Add a proc parameter to the kernel to control initcwnd, /proc/sys/net/ipv4/tcp_initcwnd. This method is valid for all TCP connections.

Limitation: The initial congestion window cannot be very large, otherwise the switching node buffer will fill up, the extra packets will have to be deleted, and the corresponding host will create more and more datagram copies in the network, causing the whole network to collapse. All major CDN vendors in the industry have adjusted the value of init_cwnd, which is generally between 10 and 20

Disable slow start restart

Description: SSR (slow-start Restart) resets the congestion window of a connection after the connection has been idle for a certain period of time.

Cause: While the connection is idle, the network condition may change. To avoid congestion, you should reset the congestion window to the “safe” default value.

Check out: sysctl net.ipv4.tcp_slow_start_after_idle

Set up theNet: sysctl – w. Ipv4. Tcp_slow_start_after_idle = 0

Effect: This can have a significant impact on long-cycle TCP connections (such as HTTP keep-alive connections) that are prone to bursts of idle, depending on network performance and data volume

Change the congestion avoidance algorithm

Congestion control algorithms have a significant impact on TCP performance, and there are many other algorithms besides the AIMD algorithm mentioned above.

Proportional Rate Reduction (PRR) is a new algorithm specified in RFC 6937. The goal is to improve the recovery Rate after packet loss.

Results: According to Google’s measurements, the new algorithm reduces average connection latency due to packet loss by 3% to 10%.

Settings: Upgrade the server. PRR is now the default congestion prevention algorithm for Linux 3.2+ kernel.

Reference:

TCP Performance Optimization practice

Web Performance Optimization