In the past five or six years, live broadcasting has undergone great changes almost every year, with different ways of playing and scenes being born, and the forms of live broadcasting have been continuously enriched. So what kind of “evolution” will live streaming technology have in the future? Recently, the volcanic engine live technical director on Monday nan in volcanic engine video cloud technology summit issued the force “is focused on the experience and growth, to explore evolution” broadcast technology in a keynote speech, the share in the new environment situation, live experience optimization challenges and broadcast technology in the evolution of two practical direction. Zhou Nan said that he expects to change live broadcasting from static capability to dynamic deployment, and change live broadcasting technology from basic “capability” to promote experience and business growth “power”, so as to bring greater value.

Zhou Nan, head of volcano Engine livestreaming technology

The following is the full text shared by Zhou Nan:

Distinguished guests, good afternoon! I’m Monday Nan, head of live technology for Volcano Engine. Today I am very happy to share with you the topic of reevolution of live streaming technology focusing on experience and growth. I hope to summarize the practice and thinking of bytedance internal and external customers supported by Volcano Engine in the past, and give you some different perspectives and input.

Challenges of optimizing live broadcast experience in the new environment

First of all, I would like to tell you my own story about live broadcasting. I believe that many guests present here have participated in or experienced the war of thousands of live broadcasting in 2016. Back then, it was mostly live shows, live games, and the resolution was still 360P, 480P, or 720P. However, in the five or six years from then to now, we will find that live broadcasting has undergone great changes almost every year. There are different ways of playing and different scenes, such as complex link, live e-commerce, paid live broadcasting, panoramic live broadcasting, life services and so on, and there are about dozens of them. The examples I give may not be complete, but we can see that the format of live broadcasting continues to enrich.

If you think of short videos as opening the black box of trust between people, then live streaming is about building connections between people. In various industries, live broadcasting has gradually become one of the essential basic capabilities. At the same time, due to the evolution of live broadcasting, customers have higher and higher requirements for live broadcasting, such as higher definition, lower delay and less lag. There are a lot of technical requirements behind this.

In addition, we can also see that the penetration of live streaming users continues to improve. There are several main points: in terms of geography, from first-tier cities to second-tier cities, and then to sunken cities, from China to globalization, this indirectly brings about the diversity of equipment and network, which undoubtedly brings great challenges to the large-scale optimal live broadcast experience. There are a lot of details to work out.

Next, let’s look at the key metrics within the technology. Usually, we pay attention to QoS indicators of live broadcast, such as first-level indicators, including connection success rate, lag rate, first frame and so on. There is no special list here. Then penetrate to the secondary indicators, will include a lot, such as back to source ratio, signaling success rate varies, more than dozens of indicators. There’s a lot of complexity, a lot of detail, and some of the metrics are mutually exclusive. Let’s review what we have just mentioned. Stacking different business scenarios, some hope for higher clarity, some hope for higher fluency, some hope for lower delay, and then stacking the differences of different network conditions and different hardware, it brings great difficulty to the technical optimization of the whole live broadcast experience.

In conclusion, with the diversity of livestreaming ecology and the change of permeability, the overall experience optimization is no longer a problem that can be solved by a single technical index. In order to achieve the continuous improvement of experience and business growth, complexity has shown an exponential increase. Of course, this is indeed the problem that we, as technical students, need to focus on, which is also the embodiment of the business value that technology can bring.

Re-evolution of livestreaming technology, establishment of feedback system and optimization map

The challenges facing technology are summarized above, so how do we find a clear optimization path without getting lost in the complexity of variables? Here I would like to share the summary and practice of Volcano Engine in the live streaming technology: First, I would like to introduce the establishment of data-driven feedback system. At Volcano Engine, we always emphasize data guidance. Why is that? There’s a classic summary: If you can’t quantify it, you can’t optimize it. We pay more attention to QoE as the quantitative target of the index, and all optimization methods focus on the change of QoE. Data-driven first focuses on A/B experiment of data, and A/B experiment is associated with live broadcast. Some general methodologies are also summarized here, including experimental design, flow calculation, experimental grouping, experimental recovery, improvement feedback, version iteration. In supporting A/B experimental platform, feedback optimization system of data experiment can be established. With key objectives, methods and implementation systems, based on past practice cases, no matter in parameter changes, function iterations or performance tuning, a lot of our work was finally quantified into the data of the entire live broadcast experience.

The second data-driven point, I define as a policy system. If data experiments are defined as coarse-density, manual competency tests, we expect this kind of competency tests to settle down. This chart is the data flow diagram of data strategy, from data burial point, data acquisition, data mining, model training, policy delivery to serial A/B experimental platform. Personalized use in different users, different scenarios, to achieve more fine-grained optimization effect, and with the front and back end of the strategy application, can achieve closed-loop data drive. Finally, whether it is data experiment or strategy system, we hope to transform live broadcasting from static capability to dynamic deployment, and evolve live broadcasting technology from basic “capability” to “driving force” to promote experience and business growth, so as to bring greater value to the business.

Re-evolution of live streaming technology and exploration of new technology upgrades

The third topic, I will focus on the exploration of new technologies. The focus here is on the field of ultra-low delay live broadcasting.

In the past, we have received a lot of feedbacks from front-line users. For example, the delay of livestream was too long, which led to the slow feedback from users and the response of anchors, especially for the livestream scenes with not too many fans. Of course, there was also the shopping spree on livestream e-commerce. Another example is the PK link of the anchor. In the PK countdown, due to the difference in delay, everyone feels unfair. There is also VR live broadcasting, in which the time delay needs to be minimized when the perspective is switched to reduce the user’s vertigo. This requires us to promote low latency upgrades and solve specific problems. What did we do? During the whole upgrade process, we did a lot of work, including the upgrade of UDP transmission mode, including a lot of anti-weak network packet loss congestion algorithm, signaling reconstruction, just to achieve the ultimate first frame experience, as well as node reuse, including upstream and downstream, here is not a repeat. Finally, take a look at the characteristics of our core technology: End-to-end delay, can achieve a wide range of distribution under the condition of 1 s, the theoretical value can be lower, such as 500 ms, with large-scale distributed ability at the same time, especially for UV big heat, have the ability to multistage distribution, and multi-resolution adaptation, achieve different equipment, different network, the end-to-end cover all link, Allow the whole business to have more options to address different demands.

Here is a review of the upgrading status of Volcano Engine in ultra low delay live broadcast. Currently, the total coverage of more than 200 million people, covering more than 10 kinds of scenes, covering more than 1000G peak flow, covering 3.6 billion minutes. In the past 10, 5 and 1s, GMV and reward rate have achieved very positive business benefits in the direction of e-commerce and mutual entertainment. Next we look forward to working with the industry to enable and upgrade the volcano engine with the latest technology. Today, the ultra-low delay live broadcast has been officially launched in Volcano Engine, you can connect to experience the whole product. In the end, the most important feature of ultra-low latency is that Volcano Engine will not be a closed system. We expect open standards and interconnection of live broadcast protocols. We also welcome joint efforts by industry service providers to provide customers with more choices and better services. Today, we have also invited students from Alibaba Cloud and Tencent Cloud to jointly issue ultra-low delay live streaming signaling standards and jointly build and communicate with each other.

This marks that the actual business of low-delay live broadcasting has been implemented, from the traditional 3-5s to the large-scale distribution within 1s. The details of the protocol interaction will be published on huohua Engine, Ali Cloud and Tencent Cloud official websites. This signaling protocol mainly made the following improvements:

First, redefine the signaling interaction process so that signaling and media can be smoothly completed. Second, build the ability to support fast connections, provide extreme first frame rendering times, and improve the split second and success rate of playback. Third, provide media feature extensions that are compatible with and support the characteristics of the live streaming industry to ensure that more media can be compatible. Fourth, support signaling security enhancements, from live signaling to data, to make live streaming more reliable and pay more attention to privacy.

The volcano engine and Ali cloud, Tencent cloud hand in hand, is a representative of pure and create technical cooperation. Since entering the cloud market, Volcano Engine has been adhering to the development concept of open interconnection. The joint construction of the open protocol standard, any company and developers can access according to the standard, jointly promote the development of video technology and application innovation.