(Photo: Dubai Mall, Dubai, 2019)

Tencent kernel team recently released TCPA unilateral congestion control algorithm for internal use, which can be used for free by webmasters, individuals and enterprises. For users, TCP unilateral acceleration in addition to Google BBR, sharp speed, there is now another option.

TCPA is developed by Tencent TEG operating system group and based on RHEL 7.4 source code. The advantage of TCPA over Google’s BBR algorithm lies in the performance improvement of small files. Small files can improve more than 40% over BBR, and large files can improve 5%~10% over BBR. By default, TCPA only accelerates port 80/443/8080, which is more suitable for site construction scenarios.

Introduction of requirement

The figure above is TCPA, the processing logic diagram after sending packets and receiving reply packets. A total of 9 modules are included. After the application sends the data through Sendfile, Write, etc., the data package construction module of the kernel will process it, and finally generate one SKB and put it into the sending queue. The smooth sending module gently sends these data (whether new data or retransmitted data) to the network according to the value of the sending speed/sending window generated by the congestion control module. It does not decide the sending rate itself, but only performs according to the given speed/sending window.

The timeout module is used to decide which packets time out, retransmit them, and eventually hand them over to the gentle sending module. The Probe module constructs some packets to Probe the quality of the network according to the specific needs of the network. Sending is sent to the gentle sending module. After receiving a reply packet, the Probe module sends the Probe reply packet to process it.

After receiving the reply packet,SACK is processed separately if there is SACK. The processing logic of SACK is complicated. It is necessary to know which packets are answered and which packets are not answered, and to mark the retransmission queue. In addition, the system determines which packets are lost and the change of RTT value in the current network according to the response in SACK. This will provide judgment information for the packet loss detection module and congestion control module. Packet loss detection module, used to determine which packets have been discarded, such as fast retransmission, RACK, etc. Only the discarded packets are marked, and the retransmission is performed by the retransmission packet processing module.

Congestion control algorithm, used to judge the current network connection, and based on this to determine the current sending speed and sending window value. Different congestion algorithms depend on different judgment information. The increase or decrease of the window is judged from here. Key information is provided by SACK module, packet loss detection module, RTO timeout module and Probe module. The retransmission module and the new data sending module are relatively simple. After the data is sorted out, it is handed over to the gentle sending module for sending.

In general, the modification of TCPA is not only a modification of congestion algorithm, but also a reconstruction of the whole logic of receiving and sending packets. For the existing functions of the system, we also provide an additional adjustable parameter interface for use. For example, the small table lists part of the adjustment parameters:

net.core.wmem_max = 3276800net.ipv4.tcp_fack = 0net.ipv4.tcp_init_cwnd = 10net.ipv4.tcp_loss_init_cwnd = 1net.ipv4.tcp_mem = 379479      505972  758958net.ipv4.tcp_metrics_minrtt = 0net.ipv4.tcp_rto_max = 120net.ipv4.tcp_rto_min = 200net.ipv4.tcp_timestamps = 1net.ipv4.tcp_wmem = 4096       16384   4194304Copy the code

Official TCPA and Google BBR test comparison report

  • Test plan

 

Use third-party Monitor for testing. Two rounds of tests were conducted in total. The first test cycle was from 0 o ‘clock on 12.9 to 0 o ‘clock on 12.11 for two days. The second test cycle from 0 o ‘clock on 12.12 to 0 o ‘clock on 12.14.

The TCPA installation package is installed and tested using the default configuration. The latest Google BBR and standard kernels were tested using the configuration of the default kernel.

The three servers belong to the same equipment room. Configure the telecom test task (the server belongs to the telecom carrier) for testing.

  • The test results

 

Testing tasks First test Second test Mean of 2 tests Longitudinal performance comparison
640 KB task Telecom requirement – 2047.401 2183.034 2115.218
TCP – telecom 1450.636 1383.674 1417.155 49.26%
BBR – telecom 1448.155 1536.517 1492.336 41.74%
8 MB task Telecom requirement – 2570.422 2552.981 2561.702
TCP telecom 1768.458 1626.909 1697.684 50.89%
BBR – telecom 2295.64 2358.646 2327.143 10.08%
25 MB task Telecom requirement – 2769.412 2713.632 2741.522
TCP telecom 1876.674 1915.549 1896.112 44.59%
BBR – telecom 2585.8 2590.205 2588.003 5.93%

Note: Vertical Performance Comparison refers to the BBR default configuration. When TCPA is enabled, the test results of TCPA compared with TCP and TCPA compared with BBR are displayed in Average of Two Tests.

  • The test results

1. Compared with the standard TCP Cubic algorithm, the performance of Google BBR algorithm is significantly improved. Small files differ little; The download performance of large files is obvious, about 35%.

2. After TCPA is enabled and system parameters are optimized, the size of files increases by more than 40% compared with standard TCP.

3. With TCPA enabled and system parameter optimization, small files can improve over 40% compared to Google BBR, and large files can improve 5%~10% compared to Google BBR.

More detailed test report can be found at https://linux.qq.com/?p=224

The installation requirement

  • Environmental requirements

System CentOS 7 or aboveThe Boot partition must be at least 500 MB (too small may cause kernel installation failure).Copy the code

  • Manual installation

1. Download the kernel and installation package

# Custom kernelLink: https://share.weiyun.com/5AeyuFg password: d1swc1# TCPA installation packageLink: https://share.weiyun.com/5RAyh7c password: qeetzpCopy the code

2. Install required dependencies

$ yum -y install net-tools bzip2Copy the code

3. Install the system kernel

$RPM - the ivh kernel - 3.10.0-693.5.2. Tcpa06. Tl2. X86_64. RPMPreparing... # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # [100%]Updating / installing...1: the kernel - 3.10.0-693.5.2. Tcpa06. Tl2 # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # [100%]Install kernelSet Grub default to "3.10.0-693.5.2.tcpa06.tl2" Done.Copy the code

4. Restart the operating system

$ rebootCopy the code

5. Install TCPA module

Unzip the TCPA installation package$ tar jxvf tcpa_packets_180619_1151.tar.bz2# installation requirement module, the default installation to the/usr/local/storage/tcpav2 directory$ cd tcpa_packets$ sh install.shEnable the TCPA module$ cd /usr/local/storage/tcpav2$ sh start.shCopy the code

6. Enable TCPA module

$ cd /usr/local/storage/tcpav2$ sh start.shCopy the code

7. Verify that TCPA is enabled successfully

$ lsmod|grep tcpatcpa_engine           224249  0Copy the code

8. Uninstall the TCPA module

$ cd /usr/local/storage/tcpav2$ sh uninstall.shCopy the code

  • One-click install script

If you find the manual installation method above too cumbersome, you can use the one-click install script below.

$ wget http://down.08mb.com/tcp_opz/tcpa/tcpa.sh$ sh tcpa.shCopy the code

Usage: The one-click installation script does not require manual intervention. It automatically installs dependencies (only epel-Release and net-tools) and kernel and restarts. After the restart, the installation is completed automatically.

  • Other skills

By default, TCPA accelerates only ports 80,443, and 8080. If you want to add other ports, you can add customized ports in the following format.

$BINDIR/$CTLAPP Access Add tip $IP tport User-defined port numberCopy the code

For example, the port to be added is 8989.

$ vim /usr/local/storage/tcpav2/start.sh# add after line 46:$BINDIR/$CTLAPP access add tip $ip tport 8989Copy the code

Reference documentation

https://www.google.comhttp://t.cn/EMGiz64http://t.cn/EMM9ur8http://t.cn/EMIHaJnhttp://t.cn/EMIrT7Uhttp://t.cn/EMIga9D

Today’s idea

Everyone has two parts of the brain, a higher logical part and a lower emotional part, which I call the two Yous, that fight for control of one person.

“– Ray Dalio, Principles

Recommended reading

  • Diagram of Docker architecture

  • Illustrates the Kubernetes architecture

  • Docker configuration and practice list

  • Use TC and Netem to simulate network exceptions

  • Teach you how to use terminal screen recording magic Asciinema