Click “LiveVideoStack” at the top to follow us
The author | leowong
Planning | bag enquiry, Alex
Edit | Alex
QUIC annual inventory# 001 #
QUIC(Quick UDP Internet Connections) is a new “faster” universal network transport protocol. QUIC offers a number of improvements over TCP and TLS to improve the performance of network transport. With the standardization of THE QUIC protocol, HTTP/3 on Top of QUIC has been supported by many browsers, including Chrome, Microsoft Edge(the Core version of Chrome), Firefox, and Safari. A number of client apps are also beginning to support and use HTTP/3. This article will take you through the history of the QUIC protocol and what I think will be the future of QUIC.
Development of QUIC protocol
In fact, the QUIC protocol we’re talking about now has two names: “Google QUIC” (gQUIC for short) and “IETF QUIC” (iQUIC for short). GQUIC is the original protocol designed by Google engineers in 2012. Google made THE QUIC protocol public in 2013 and submitted it to the IETF in the same year. This is similar to the development of SPDY and HTTP/2 protocols, which were first designed and tested by Google and submitted to the IETF working group for standardization with improved performance.
In June 2015, an Internet Draft of QUIC was submitted to IETF for standardization, and the QUIC working group of IETF was also established in the same year [1].
In October 2018, the HTTP Working Group of the IETF and the QUIC Working Group jointly announced HTTP/3, the HTTP protocol running on TOP of QUIC, but the specific standards for HTTP/3 were not finalized at this time.
In May 2021, IETF announced the STANDARD OF QUIC, RFC9000[2], meanwhile, RFC9000 was made up of RFC8999[3] (version-independent Properties of QUIC), Supported by RFC9001[4] (Using TLS to Secure QUIC) and RFC9002[5] (QUIC Loss Detection and Congestion Control).
At this point, the complete standard of QUIC protocol has been formed, iQUIC and THE original gQUIC has been quite different, it can be completely regarded as a separate protocol. IQUIC has improved on gQUIC’s original design, from packet formats to handshakes and HTTP mapping, thanks to the open collaboration of many organizations and individuals working together to make the Internet faster and more secure.
Author’s note: The rest of this article describes iQUIC unless otherwise noted.
Status quo of QUIC protocol
The birth of QUIC protocol is closely related to today’s Internet application scenarios and transmission performance. Over the course of the development of the Internet and HTTP, the HTTP underlying protocol has largely remained the same. However, as massive mobile devices drive up Internet traffic, more and more application scenarios require network transmission with low latency, high throughput, and QoS awareness. The original HTTP protocol is increasingly difficult to meet application requirements in providing smooth and efficient Web access.
All features of QUIC start “Built on UDP”
“Built on TOP of UDP” means that you can be flexible to adjust reliable transport mechanisms and congestion control algorithms without caring about or understanding kernel development, which greatly facilitates a variety of transport qOS-sensitive applications, such as real-time audio and video transmission, online games, etc. The main features of QUIC are as follows:
-
Choosing UDP also allows the QUIC protocol to be adjusted independently of the network’s middle-boxes.
-
The difficult problems of TCP protocol development, debugging, and observation will also be solved. Instead of collecting and debugging protocol stack programs through Berkeley Packet Filter (BPF), simple data collection programs can be deployed in user mode.
-
Application crashes do not affect the system kernel, and can leave enough crash scene for subsequent analysis.
-
User mode has more libraries to help us implement more complex and comprehensive policies, such as adaptive CC (Congestion Control) algorithms.
-
Using UDP packets without sequence dependence, multiplexing between QUIC multi-data streams, no queue head blocking problem, and each has an independent Flow Control.
-
TLS1.3 is coupled to achieve 0-RTT connection and connection multiplexing.
-
User-mode implementation and network quintuple independent ConnectionID, ConnectionID, connection migration, and subsequent Multipath (QUIC).
The use of QUIC is steadily increasing
With the emergence of QUIC V1 version protocol standard, more and more websites begin to use QUIC traffic. According to the statistics of W3Techs[6], about 23.8% of websites use HTTP/3 at present.
Browser Support
-
Both Chrome and Opera support gQUIC and iQUIC.
-
In May 2021, Firefox began supporting QUIC in Firefox Nightly Edition.
-
In April 2020, Apple introduced experimental QUIC support in the WebKit engine and officially supported it in macOS Big Sur and Safari 14 on iOS 14, but required manually turning on HTTP/3.
Client support
-
Cronet is the earliest client library that supports QUIC. At present, many clients use Cronet library, including Google Play Services, YouTube, Gmail and so on.
-
CURL version 7.66 (September 2019) also supports QUIC, although the QUIC protocol stack cURL references is the Quiche open source library developed by Cloudflare RUST.
-
In October 2020, Facebook announced that all of its apps, including Instagram, also support QUIC, and that 75% of its traffic uses the QUIC protocol.
-
Uber App also supports QUIC protocol.
Server support
-
Since 2016 and 2017, there have been a number of public implementations of the QUIC server, including Google open source its prototype server source: github.com/google/quic…
-
Open source Caddy Server also has a set of QUIC protocol stack based on Go language, which can provide QUIC services.
-
Starting in 2017, LiteSpeed also supports the QUIC protocol on its WebADC (Load Balancer) and LiteSpeed Web Server products.
-
In 2018, Facebook and Cloudflare also started supporting the QUIC protocol on their servers.
-
From 2021, Microsoft began to support THE QUIC protocol in its Windows Server 2022 and SMB, and open source its protocol stack MsQuic.
-
Subsequently, Citrix’s ADC (Application Delivery Controller) and NetScaler products also supported QUIC.
Partial list of open source QUIC protocol stacks [7]
Some of the issues that have hindered the development of QUIC
Integration of the difficulty
The integration of the QUIC protocol is not an easy task, both on the client side and the server side. If the current network does not support QUIC, it means that we need to modify the application to accommodate the network library adjustments. This is often not the starting point for application iterations.
Let’s take a look at what clients and servers need to consider when integrating the QUIC protocol:
The client
-
Apply trade-offs between costs and benefits.
-
The transition period may also require the existence of both the old and new network libraries, which is convenient to degrade fault tolerance and increase the application volume.
-
Different QUIC library interfaces are not unified, unlike Socket unified interfaces with portability.
The service side
-
The network event model needs to be adjusted to accommodate the QUIC protocol stack, as well as TCP compatibility.
-
Whether the back-end architecture supports QUIC and how to convert HTTP/3 QUIC traffic to HTTP/1 to the Backend Service?
-
QUIC connection multiplexing and connection migration between multi-regions and multi-nodes should be considered.
-
Power consumption ratio of server QUIC traffic and how to achieve the same power consumption performance as TCP traffic.
QUIC stack performance
Compared with TCP protocol that has been developed for more than 30 years, the emerging QUIC protocol still has a lot of optimization to do in the engineering of protocol stack implementation. According to the data published by Google in 2017 [8], the CPU consumption of QUIC was twice that of TCP for the same traffic at that time. The performance pain points of QUIC protocol stack mainly include:
-
UDP packet sending and receiving performance
-
Data packing and unpacking processing performance
-
Packet parsing performance
-
ACK processing performance
-
Encryption and decryption performance
-
Performance of other processes: packet Pacing, CC algorithm, memory usage, etc
UDP Packet Performance
UDP packets have not received the same optimization treatment as TCP packets in kernel receiving and sending. UDP packets occupy a large consumption of QUIC protocol stack.
-
The original UDP Socket interface for Sendto /write was weak.
-
Sending and receiving data in batches Sendmsg /sendmmsg is not significantly improved.
-
Kernel GRO/GSO can improve performance, but not enough.
-
Kernel XDP or DPDK can greatly improve performance, but the program modification is significant.
-
Hardware NIC Offload UDP transceiver can be the ultimate optimization, the same program modification is great.
The figure above is the statistics of QUIC power consumption by Google in 2020 [8] : Compared with TCP/SSL, QUIC has 20% extra consumption. Station B is about the same internally, but it is believed that with the implementation of more optimization schemes, the flow energy consumption of QUIC and TCP will converge.
UDP is restricted by carrier QoS
As far as I know, based on the operation of QUIC protocol deployed by ourselves, domestic operators do impose some QoS restrictions on UDP traffic, but the proportion that can be restricted by QoS is so small that it can be completely ignored. Carriers do this for two main reasons:
-
The aging time of UDP traffic quintuple connections in the NAT state is not precise enough. If the aging time is too long, UDP connections of low-frequency communications will be damaged. If the aging time is too short, UDP connections will consume much device performance. Generally, a compromise rule of thumb is chosen, which can hurt UDP traffic in certain scenarios.
-
The priority queue of packet classification on carrier devices is difficult to manage UDP quintuples because THE UDP quintuples change frequently and UDP traffic occupies queues at all levels but cannot be accurately controlled. Generally, carriers restrict UDP traffic globally in specific periods of time or under specific load conditions.
Debugging the QUIC protocol
With the increasing use of QUIC protocol, the method of debugging QUIC protocol has gradually formed a standard — QLOG (currently a draft [9]). Qlog is a packet capture tool similar to wireshark, but is a set of protocols embedded in the protocol stack to capture key data packets. For details, see the qlog draft. Qvis is a set of front-end tools of QLOG [10], which is used to help engineers debug and tune the interaction process of QUIC protocol.
In fact, Qlog can do more than that. We can combine big data analysis and machine learning model to analyze massive QUIC transmission data, and achieve automatic location of network faults, transmission algorithm optimization and network feature engineering. For details, please refer to [11] open source project, which will not be detailed in this paper.
The future of QUIC protocol
Although the QUIC protocol came about because of HTTP/3, I think the future of QUIC should be more than HTTP/3. QUIC already has features that not only make existing Web applications run faster and more securely, but can also be adapted to a wide range of application scenarios, such as IoT, VR, audio and video conferencing, and more. Based on THE QUIC protocol, engineers can customize the application of more scenarios, but also make the QUIC protocol has more capabilities.
Starting from myself and combining with the business scene of STATION B, I mainly focus on the future development of QUIC protocol in the field of audio and video transmission: QUIC Datagram[12] and QUIC Multipath[13].
QUIC Datagram — QUIC’s unreliable data packet transmission extension
QUIC v1 (RFC9000) provides a secure, reliable and multiplexed transmission scheme. QUIC Stream is the main data transmission carrier of QUIC, and reliable transmission is its basic function to be guaranteed.
However, there are some application scenarios that do not require strict and reliable transmission, such as real-time audio and video, online games, etc. Some of these scenarios are now implemented directly based on custom UDP or WebRTC. Custom UDP to achieve the transmission layer function is more troublesome, large amount of engineering, sometimes for security considerations also need to add DTLS support, and WebRTC to establish the data transmission channel process is also more cumbersome, WebRTC main scene is designed for P2P, not suitable for the client server scenario in the long run.
Therefore, QUIC support for unreliable data transmission can make THE APPLICATION of QUIC protocol wider, suitable for more application scenarios, and unreliable data transmission on QUIC can well reuse THE security features of QUIC.
QUIC Datagram characteristics
-
The same QUIC connection can contain both reliable Stream transport and unreliable data transport. They can share the once-handshake information, which can be applied to TLS and DTLS respectively to reduce the handshake delay.
-
QUIC is more accurate than DTLS in the grip. It adds a timeout retransmission timer to each handshake packet and can quickly sense the loss and recovery of the handshake packet.
-
Unreliable transmissions from A QUIC Datagram can also be ACK, an option that allows the application to sense a successful reception of a QUIC Datagram if there is an ACK.
-
QUIC Datagram also works with QUIC CC.
These features make delivery of applications such as real-time audio and video streams, online games, and real-time web services much more efficient.
WebTransport is just around the corner
WebTransport is a frame-based protocol [14], which enables the client to communicate with the remote server in a security model and adopts secure multiplexing transmission technology. Note: WebTransport is a framework under which is the actual protocol. The framework provides a consistent interface, so it consists of a set of protocols that can be securely exposed to untrusted applications and a model that allows them to be used interchangeably. It also provides a flexible set of features that provide us with reliable one-way and two-way traffic as well as unreliable data packet transmission.
QUIC Multipath
QUIC Multipath is simply a way to transmit data over multiple physical network paths, using WIFI, LTE/4G, or IPV4/IPV6 at the same time. As the consumption rate of audio and video is getting higher and higher, the throughput capacity of network connection is getting higher and higher, and the single path network is beginning to be inadequate, so MP-QUIC can play a larger role. For example, in ultra HIGH definition, ultra high bit rate video 8K, VR and other scenes.
The basic idea of MP-QUIC is to use multiple network paths to form a QUIC connection for data transmission. In fact, MP-QUIC is also an extension of the connection migration feature, and the protocol standard is also in the draft stage [13]. While it may seem simple to implement a QUIC connection with multiple network paths, there are a number of things to consider:
-
Heterogeneous network
-
ROI between packet redundancy allocation and traffic cost
-
Multipathing scheduling policy
Currently, the protocol standard only provides the mechanism of multipath implementation, that is, multiple unidirectional Uniflows form a QUIC connection. Multipath scheduling and redundant packet policy need to be independently implemented. However, with the development of business scenarios and the formal launch of protocol standards, I believe that MP-QUIC will be more and more used in audio and video transmission.
reference
[1] QUIC Wiki: en.wikipedia.org/wiki/QUIC
[2] RFC9000: datatracker.ietf.org/doc/html/rf…
[3] RFC8999: datatracker.ietf.org/doc/html/rf…
[4] RFC9001: datatracker.ietf.org/doc/html/rf…
[5] RFC9002: datatracker.ietf.org/doc/html/rf…
[6] W3Techs HTTP/3 Usage Statistics: w3techs.com/technologie…
[7] QUIC Workgroup Implementations List:
Github.com/quicwg/base…
[8] QUIC CPU Performance: conferences.sigcomm.org/sigcomm/202…
[9] QLog: datatracker.ietf.org/doc/html/dr…
[10] was Debugging QUIC and HTTP3 with QLog: www.researchgate.net/publication…
[11] QLog&Qvis Tools: github.com/quiclog
[12] QUIC Datagram: datatracker.ietf.org/doc/html/dr…
[13] QUIC Multipath:
Datatracker.ietf.org/doc/draft-d…
[14] the W3C WebTransport: www.w3.org/TR/webtrans…
About the author: Wang Sheng, responsible person of the VIDEO cloud CDN system of STATION B. With years of experience in audio and video backend development and network transmission development, I participated in the construction of video cloud platform from zero to TB level at STATION B. Led the research and development of an audio and video distribution network with tens of millions of DAU in STATION B, proficient in various network transmission protocols and network algorithm tuning sensitive to various scenarios.
The lecturer to recruit
LiveVideoStackCon 2022 Shanghai station, is open to the public recruitment of lecturers, whether you are in the size of the company, title level, old bird or rookie, as long as your content is helpful to technical people, the other is secondary. You are welcome to submit your personal information and topic description via [email protected], and we will give feedback within 24 hours.