XLINK multiplexing technology, jointly developed by the Architecture team of Alibaba Tao Department technology Department and XG Laboratory of Dharma Institute, related paper “XLINK: “Qoe-driven Multi-Path QUIC Transport in Large-scale Video Services” has been officially accepted by SIGCOMM 2021, a top academic conference, This is also the first paper on multipath QUIC in SIGCOMM history.

review

Have you ever experienced

(1) When you are watching the video and binge-watching, you suddenly find that the video is jammed. How can you reconnect it?

(2) When you make a voice call walking from the mall to the parking lot, the phone suddenly cuts off and you have to dial again?

(3) When you’re racing to get to work on the freeway, but your email just won’t get through?

The generation of the above problems can be reduced to a problem, that is “weak network”. Due to the natural spectrum limitations of wireless networks, insufficient coverage of wireless signals, competing resources among multiple users, frequent base station switching in high mobile scenarios, etc., all may lead to frequent “weak networks”. Overcoming weak networks is critical to the user experience. To this end, Alibaba Tao Department technology Department Tao Department architecture team and DHARma Institute XG laboratory jointly developed XLINK multiplexing technology. XLINK enables Taobao users to use multi-path (5G/4G, WiFi) for data transmission at the same time, which fundamentally solves the user experience problems caused by single-path weak network. XLINK is based on the IETF Multipath QUIC draft proposed by Alibaba [1], which is the only Multipath QUIC standard draft that has been tested in large-scale practice at present.

QUIC technology is proposed by Google, Google in 2017 SIGCOMM conference published QUIC related papers and caused a huge response in the industry this year IETF QUIC 1.0 standard work will be formally completed, the next generation of HTTP protocol HTTP3 is based on QUIC to achieve. It can be said that QUIC is the most core and key transfer technology in the mobile Internet. Today, over 50% of Chrome browser traffic and 75% of Facebook traffic are transferred using QUIC. Through unremitting efforts in the past few years, Alibaba has rapidly grown from a follower of QUIC technology to an innovator of QUIC technology, and made breakthroughs in multipath QUIC technology. XLINK related papers have been officially accepted by SIGCOMM 2021, a top academic conference. This is also the first article on multipath QUIC in SIGCOMM history.

XLINK

Zhilong Zheng†, Yunfei Ma†, Yanmei Liu†, Furong Yang, Zhenyu Li, Yuanbo Zhang, Jiuhai Zhang, Wei Shi, Wentao Chen, Ding Li, Qing An, Hai Hong, Hongqiang Harry Liu, and Ming Zhang, XLINK: Qoe-driven Multi-path QUIC Transport in large-scale Video Services, to Appear in SIGCOMM 2021.

XLINK is implemented based on the IETF Multipath QUIC draft framework proposed by Alibaba. The draft is led by Tao Architecture and XG LABS, and the standardization Department of Alibaba Group, institute of Computing, Chinese Academy of Sciences, And Christian Huitema, the former chairman of the Internet Architecture Board, conducted in-depth discussion and cooperation, which is the only draft Multipath QUIC standard that has been tested in large-scale practice.

XLINK based on the DRAFT OF IETF Multipath (QUIC) proposed by Alibaba

Yanmei Liu, Yunfei Ma, Christian Huitema, Qing An, and Zhenyu Li. Multipath Extension for QUIC, The Internet Engineering Task Force, another awarding 2020. A Work in Progress. Tools.ietf.org/html/draft-…

Multipath technology: a ten-year exploration in academia and industry

The idea of using two paths simultaneously sounds simple, but it is difficult to do. At present, the only successful multipathing deployment in the industry is Apple’s Siri, Apple Music and other scenarios (Apple uses MPTCP RFC6824, which was standardized by IETF in 2013). These applications are Audio. They mainly use multipath to increase the robustness of transmission. However, in the past, the large-scale application of multiplexing technology is rarely seen in the industry Video application or real-time Video application, because such scenarios have very strict requirements on the bandwidth rate and delay of data. However, in wireless network, the path quality changes rapidly, and the past multipath protocol and scheduling algorithm will have obvious stall phenomenon in the high-speed changing scenario. Before XLINK, multipath transmission in audio and video has been delayed to display, the academic community has struggled for many years, we put forward a lot of MPTCP based optimization scheme, but no one can completely solve the following essential defects:

  • Kernel implementation, unable to provide customized optimization for application scenarios: Audio and video applications have a characteristic that the experience objective (QoS) of different applications is very different, and the transmission protocol and algorithm are also very different. For example, short video applications (Xiaotao short video) focus on second opening rate, while high-definition long video applications (Youku) have high bandwidth requirements. Video conferencing and live broadcasting (Dingding, Xiaotao live broadcasting) care more about delay and smoothness. These differences huge application scenario for the demand and the need to transport the algorithms and protocols do special optimized QoS demand of the application itself, but the MPTCP is located in the operating system kernel network protocol stack, all applications using the same set of algorithms, this leads to all tastes, at the same time transport protocol and iterative optimization of scheduling algorithm is also very difficult.

  • Heterogeneous network: the multipath aggregation effect of MPTCP is not ideal. As the multipath transmission on the public network is heterogeneous, the delay difference between 5G/LTE and Wi-Fi is large, and then the problem of multipath queue blocking (MP-HOL) will occur [2]. The MP-HOL blocking problem refers to that when some packets go through slow path and some packets go through fast path, the packet in the fast path will arrive first, but the packet in the fast path can only be transmitted to the application after the packet in the slow path arrives, resulting in increased delay. In some cases, it is even worse than the single path with good quality in the two paths. The bigger challenge is that wireless links change so quickly that it is difficult to predict bandwidth accurately (in wireless scenarios, it is difficult to predict bandwidth from the next second). So bandwidth prediction – based scheduling algorithms frequently fail in the long tail problem.

  • Traffic cost: In order to overcome the problem of heterogeneous network, some multipath transmission schemes choose to send redundant packets (sending the same packets on two paths repeatedly) to avoid the blocking problem of multipath queue, but two new problems are introduced: 1. Sending packets repeatedly can greatly increase the cost of additional data traffic. This can lead to high bandwidth costs for video applications, which is why Apple only uses MPTCP in audio scenarios such as Siri, where traffic is not as demanding. 2. Redundant data packets occupy bandwidth resources, which reduces the overall bandwidth utilization efficiency.

Other multipath schemes such as MPUDP and MPRTP also have their own difficulties and limitations. First of all, UDP does not guarantee the reliability of packet transmission, so the different delay of multiple channels will bring a large number of out-of-order packets to the upper application, and UDP does not recover packet loss, so it is rarely used at present. MPRTP has not been implemented on a large scale since it was proposed [6]. The reason is that WHEN MPRTP distributes data packets to various paths, it relies on the accurate estimation of bandwidth and delay of each path. However, unless there is a large amount of physical layer information, the prediction of LTE signal itself is a very difficult problem to solve.

XLINK: User experience Driven (QOE-driven) multipath solution

Why XLINK?

XLINK (= X+LINK), originally meant to LINK through multiple paths, where X represents an unknown quantity, and also represents the continuous exploration of unknown areas in Alibaba. XLINK technology based on QUIC protocol realizes WiFi/LTE/5G multi-path parallel transmission in user mode, effectively improves transmission bandwidth, greatly reduces transmission delay and lag rate, and shows excellent transmission stability in high mobility scenarios. At the same time, XLINK is the world’s first multipath QUIC communication protocol deployed and verified in large-scale video scenarios. XLINK is very convenient to use. XLINK is implemented in user-mode protocol stack XQUIC[3] and supports Android/iOS/Linux platform deployment. For mobile app developers, XLINK technology can be used only by integrating XQUIC protocol library. Users only need to upgrade the APP to enjoy the experience benefits brought by multipath transmission [4].

In order to break through the fundamental limitations of single-path transmission and solve the various landing problems of MPTCP and other multipath protocols in the past practice, we developed XLINK. The biggest difference between XLINK and all previous multipath technologies is that it directly uses the application QoE information to implement path selection, switching, and scheduling policies. From the technical point of view, XLINK breaks through the traditional multipath protocol design framework, on the basis of QUIC user mode characteristics, put forward the Client-Server QoE feedback driven multi-transmission scheduling scheme, It overcomes two major problems that have plagued the deployment of multipath protocols (such as MPTCP, MPUDP & MPRTP[5]) in the actual public network over the past decade: transmission stall caused by blocking of multipath queue and reduced aggregation efficiency caused by high extra bandwidth cost and traffic overhead introduced by sending redundant packets

The overall architecture of XLINK is shown in Figure 1, with the following features:

(Fig.1. Overall Scheme architecture of XLINK)

  • User mode deployment: XLINK works in user mode, is integrated in App and realizes reliable data transmission and congestion control over UDP. It comes with rapid application deployment and iteration, plug and play. XLINK applications on the mobile end can be updated by week, and XLINK applications and algorithms on the server can be updated by day or hour. Because XLINK is implemented in the user-mode protocol stack and integrated with APP, XLINK can be directly customized for different applications.

  • High performance: XLINK uses the QoE feedback of video applications as the control signal of the scheduler. The QoE signal can contain a variety of parameters related to user experience. Through this QoE feedback control scheduler, XLINK successfully overcomes the performance and cost problems brought by MP-HOL and makes the use of multipath technology feasible in large-scale video applications. When XLINK performs multipath scheduling, it does not need to accurately estimate the path bandwidth and delay. Instead, it adopts an adaptive compensation method of Reinjection of packets to achieve self-adaptive balancing of packets on multiple paths. In addition, XLINK can further optimize the user experience through its understanding of user QoE. For example, XLINK can be specially optimized for the first frame of a short video, reducing the user’s first frame download time and thus increasing the user’s break rate in seconds.

  • Low cost: XLINK’s scheduling algorithm can not only overcome the performance problems brought by MP-HOL, but also hardly increase the amount of additional data. The feedback of QoE helps XLINK adjust the strength of reinjection to achieve the optimal balance between performance and cost. Therefore, video applications with high bit rates can safely use multipath transmission on a large scale without worrying about traffic costs.

  • Lightweight: XLINK is completely developed based on C language (implemented in XQUIC user-mode protocol stack). XQUIC’s overall package size is only 300+KB, which is very suitable for various mobile terminals.

Landing effect of hand wash

XLINK has completed large-scale gray scale verification in mobile shopping. The test results show that using XLINK in weak network can reduce the average download time of short video fragment by 15.03%, and the download time of video fragment in weak network by 25.28%[5]. In addition, while on the road, XLINK users can take advantage of both WiFi hotspots and LTE on their phones to maintain a smooth video viewing experience in high mobility scenarios.

XLINK hand tao Demo show:

The following video shows the use effect of XLINK integration in Handtao. The application on the left uses XLINK with WiFi and LTE enabled, and the application on the right uses QUIC with single path WiFi enabled. It can be seen that XLINK starts faster and plays smoothly throughout the whole process, while the single path case has obvious lag due to WiFi network jitter during the playing process.

  • Click on the link to watch the video:

Mp.weixin.qq.com/s?__biz=MzA…

Looking to the future

XLINK, a multi-path QUIC technology jointly developed by XG Laboratory of Dharma Institute and Taodepartment Technology, not only has a good experience optimization effect in short video scenes such as taotao — short video download time can be reduced by more than 25% under the condition of weak network; This year has seen the gradual introduction of full hand-washing short videos and hand-washing – and the multi-path QUIC draft has received extensive attention in the IETF QUIC Working Group. As XLINK continues to be recognized by peers in the international arena, it can be said that XLINK has achieved exciting results in terms of technological leadership, international standard formulation and internal business empowerment.

We are at a critical juncture in the era of 5G and edge computing, which will revolutionize network architectures and technologies. 5G will further accelerate the explosive growth of diversified, end-computing and edge computing services and application technologies such as big data, artificial intelligence and mass storage. Therefore, it can be inferred that wireless network, as the entrance of these new services, is bound to usher in technological changes again and move towards a new generation of networks with high performance, high stability and high agility to respond to these new services.

At the same time, with the continuous enrichment of video applications and the heterogeneity of video QoS requirements, it is difficult to achieve better QoE by adapting a set of transport layer to all applications. The user-mode protocol stack represented by QUIC can achieve the ultimate effect that the kernel protocol stack could not achieve through joint optimization with video applications. XLINK demonstrates the power of this synergy by combining QoE with multipath. In the future, we will further promote the standardization of XLINK related technologies in IETF, and expect XLINK technology to better serve Alibaba users.

The appendix

[1] Multipath QUIC draft: tools.ietf.org/html/draft-… [2] Mp-hol blocking problem: [3] XQUIC is a standard protocol library of IETF QUIC developed by Alibaba. [4] It is up to users to decide whether to turn it on or off. [5] Here weak network statistics refers to the p99 bit long tail download time of video fragments [6] Singh, Varun, Saba Ahsan, and Jorg Ott.” multipath considerations for real-time media.”Proceedings of the 4th ACM Multimedia Systems Conference. 2013.

Tao architecture and basic service team recruitment

This is a team dedicated to providing basic core capabilities, products and solutions for Amoy and Ali

• Implementation and implementation of the next generation network protocol QUIC – XQUIC

• Adaptive high availability solutions with core capabilities – Noah

• A new generation of business development model platform – Gaia

• Mobile middleware product system (ten million level QPS access gateway, API gateway, push/message, mobile configuration center, etc.)

If you want to join us, please email your resume to: [email protected]

— — — — — — — — — — — — — — — — — — —

The author | yun-fei ma, Liu Yanmei Zheng Zhilong hong-qiang liu

Edit | orange

New retail product | alibaba tao technology