With the progress of video technology and the iteration of standards, the video industry has entered the digital era from analog, completed the media transformation from film and TELEVISION to the Internet, and derived a variety of innovative forms such as ULTRA-HIGH definition, 3D, AR/VR. Especially in the post-epidemic era, we can see many new changes in the field of audio and video technology, such as the collaborative interaction between cloud and terminal, the deep combination of algorithm innovation and engineering application, and the penetration and promotion of scenes and demands. Under severe challenges, new scenes and vitality have been brought to all walks of life.
In the upcoming LiveVideoStackCon 2021 Beijing station, experts from Aliyun intelligent video cloud will explore and discuss the innovative exploration of video cloud technology in the cloud together with numerous industry partners. To this end, we interviewed Ye Yan, a researcher of Alibaba, and He Yaming, a senior technical expert, and had an in-depth conversation with the two experts about the codec technology and the new scene application of video cloud.
“Social video” : The video cloud is the new infrastructure
The rise of the network video, from 2006 to now entering the era, the socialization of video 5 g, cloud, AI has become the trend of the development of the society, video is no longer limited to film, television, advertising, and other areas of the traditional media, video conference, interactive video, new applications, such as electric live, make industry boundary flux, the video industry escalating demand and technology. With the development of technology and the consolidation of infrastructure, video will become a new form of interaction and information bearing.
(Source: IResearch — 2021 Chinese Video cloud scenario Application Insight White paper)
The video cloud has evolved into a critical piece of infrastructure for the highly competitive and rapidly iterating big video industry. As is known to all, the current video business to calculate the force of these resources, storage, bandwidth consumption is very high, a popular live concert, for example, there may be millions of people were watching, this not only requires strong end side real time video processing ability, and relying on large-scale CDN smooth distribution of task distribution network, Even some special visual effects of AR/VR need to be presented through the calculation of edge nodes, so just moving the server to the cloud can not meet the requirements of the future scene, how to take advantage of the advantages of cloud technology architecture and business evolution has become a common issue faced by the industry.
Yem: To promote the implementation of the next generation of video standards and unleash industry productivity
Ye Yan is a researcher of Alibaba and head of cloud video standards and implementation of AliYun Smart Video. She is responsible for the technical development of video cloud in ITU-T VCEG, ISO/IEC MPEG, AVS and other international and national video standard organizations, involving the research and development of advanced technologies such as video codec, AI video quality assessment, VR/AR. She participated in the development of many international standards for video codec and streaming media, including H.266/VVC, H.265/HEVC, SHVC and so on. She is the author of more than 50 academic papers, the inventor of more than 130 U.S. licensed patents and more than 230 U.S. patent applications. She is also an IEEE Senior fellow. She received her bachelor’s and master’s degrees from the University of Science and Technology of China and her doctoral degree from the University of California, San Diego.
Video can’t do without codec technology, and codec can’t do without standard guidance. Video standards have always been the infrastructure for the development of the video industry. Video standards cover a wide range of areas, from the system standard MPEG CMAF to the codec standard H.266/VVC. The continuous update and iteration of video standards play a crucial role in improving the efficiency of video production, reducing costs and creating new experiences, and is also related to the future trend of the entire industry.
Ye Yan as alibaba researcher, ali cloud video cloud video standards and implementation team, head of the depth of the work has been the international standardization of video players and agents, in Ye Yan view “video standards organization witness is cutting edge technology and grasp the pulse of the latest industry one of the best places, it is through the industry experts and open technical discussion, Listening to the market has enabled us to iterate on more efficient standards and continue to drive the industry forward.”
However, facing the new stage of development, the industry also put forward different voices to some video standards organizations. One view, such as MPEG standard organization has lost its dominant role, everybody also in tenths of a performance gain racking their brains, which brings greater computational cost, the innovation of the type of this kind of the hi more brush is a kind of sense, did not provide the essential technical advances and innovation, the industry should find new ideas for solving the problem of video compression.
In the face of such noise, Diem expressed his own judgment: “I don’t really agree with the idea that the traditional framework and the new framework are separate or even antagonistic. While performance mining in traditional frameworks is increasingly difficult, this direction is based on a familiar framework that benefits hardware and software implementations, and ECM demonstrates that this framework can still provide significant performance gains, so it should not be abandoned lightly. On the other hand, JVET is also exploring what new frameworks or tools can achieve significant performance gains overnight. At the same time, we are very concerned about the calculation cost of this new framework. To be honest, we are still trying to figure it out, so we have to rely on the two-legged approach to find the next generation of codec technologies that have the most potential and are achievable.”
Indeed, each generation of coding standards is a very difficult work, not overnight. Take VVC, the latest standard in the industry, for example, it took about three years to complete the pre-research work before it officially started. For this reason, less than a year after the VVC standard was finalized, JVET set up the ECM software platform in the first half of this year to carry out technical pre-research and development of the next generation coding standard. “Although the compression capability of ECM is about 14% higher than that of VVC, based on previous experience, it will take several years for this pre-research work to meet the compression performance gain requirements of the next generation standard,” yan said. I expect to see the emergence of many 5G video applications in the coming years as the market and business are changing so much.”
He Yaming: “cloud + terminal + service” is the future trend of video cloud
He Yaming is ali Cloud intelligent business group video cloud senior technical expert, video cloud technology research and development. Before joining Alibaba, I worked in Facebook and Microsoft in the United States. I worked as Principal Software Engineer in Microsoft, engaged in the research and development of video coding and video cloud, and was responsible for the research and development of real-time audio and video and live broadcast technology in Facebook. Facebook Messenger and Facebook Live went from zero to billion-user stars in just a few years.
“Audio and video have natural cloud native properties, ‘cloud + terminal + service’ is the future trend of audio and video development.” This is ali Cloud intelligent video cloud senior technical experts, video cloud technology research and development director He Yaming to make the judgment.
In he Yaming’s opinion, the development of audio and video has always been the best practice of cloud native: cloud infrastructure, including central node, edge node and CDN network, is the basis for guaranteeing the large-scale distribution and transmission of audio and video; The computing power and flexibility of the cloud bring unlimited computing power to the audio and video business, while effectively controlling costs and generating more new scenarios. In addition, in today’s av end side equipment is more and more rich, “cloud” and “end” synergy is more important, in 2020, ali cloud proposed integration “cloud” strategy, in such a context, its path advantages increasingly greater force, relying on strong ali cloud cloud, can make the more intelligent, lighter, more flexible, Let developers create thousands of innovative applications, its development efficiency, operation and maintenance cost, ductility have been greatly optimized. On the road of “cloud integration, cloud edge integration, soft and hard integration”, He Yaming stressed the important role of AI — “We especially emphasize the application of AI, from intelligent video coding, image enhancement to super resolution, from intelligent beauty, virtual background, bel canto sound change to cartoon video. We can say that we are using the AI power of the whole group to push the audio and video scene into a broader space.”
“The summit, ali cloud video cloud bring special theme is’ from the cloud to innovation, video cloud of new technology and new scene, ‘here I want to special emphasis on the word” innovation “, the cloud is the consensus of the video industry, and basically complete the original biochemical processes of cloud, we the problem really is how to complete the next stage of innovation on the cloud, Companies should shift from providing resources and tools to providing services and ecology as a breakthrough, “he said.
At present, most of the leading cloud manufacturers in China have strong technical service capabilities and complete content consumption ecology, so that video products can be servized. Through API-based, PaaS service, PaaS+, SaaS tools, on-end SDK, low-code platform and other means, the access threshold of video technology can be reduced to better serve developers. Ultimately better service video production and consumers.
Now, in the face of domestic manufacturers in the field of video cloud cloud head fierce competition, He Yaming see more opportunities: “this is the trend we are very willing to see, it is also the result of our constantly push forward, ali cloud also hope that more and more people with lofty ideals to join video cloud team, will audio-visual together into a new era.”
Technology and scene: Future innovation and challenge of video cloud
At the Ali Cloud Intelligent Cloud Summit held in Beijing in May 2021, Zhang Jianfeng, president of Ali Cloud intelligent business Group, announced that Ali Cloud would add “good service” as an important strategy on the basis of “deepening the foundation, thickeningZhongtai and strengthening the ecology”. Video cloud technology as cloud computing, artificial intelligence, network and other technologies are closely combined with the industry scene, Ali Cloud has always insisted on the deep cultivation of the underlying technology, the application of Taiwan technology and service scene innovation.
Video coding and decoding is a technical field in which Alibaba has always had a dominant position in the industry, and it is also a concrete action of the group to adhere to the basic technology research of audio and video. Ali Cloud video standard team has just finished the technical development of h. 266/VVC, a new generation of international video codec standard, in 2020, and has invested manpower to vigorously promote the development of H.266/VVC codec at the first time. Soon after, AliCloud released the real-time HD codec Ali266, which strongly promoted the implementation of H.266/VVC standard application and really opened the commercial road of H.266/VVC.
When talking about the difficulties in the development of Ali266, yan said: “A mature commercial encoder must be deeply optimized through the algorithm to meet the requirements of real-time coding speed. In order to get the powerful compression performance provided by H.266/VVC, it is necessary to choose the most reasonable coding tool quickly and accurately from the many coding tools provided by VVC for the input video content. Therefore, we develop Ali266 along this track, in-depth VVC coding tool set, through qualitative and quantitative research on each coding tool, to help us choose coding tools. At the same time, we also pay special attention to subjective quality in the algorithm optimization process. In case of conflict with objective quality indicators, we are more inclined to ensure higher subjective quality, that is, to ensure the final user experience. Ali266 can achieve real-time HD and real-time full HD coding speed for the first time, and at the same time open enough gap with the coding performance of HEVC, and we take such a development strategy is directly related to the emerging VR/MR needs higher resolution video format as the technical base support. The bandwidth savings provided by VVC are therefore more valuable. So we will continue to work on the Ali266 to make it go faster and faster, reaching ultra HD 4K or even 8K real-time coding capability in the near future. It will also provide a good landing scenario for more efficient codec standards.”
Not only in the field of audio and video technology, but also with the in-depth integration of Ali Cloud video cloud business with the overall business of Ali Group and the in-depth practice of industrial customers, Ali Cloud video cloud has increasingly enriched the scene cooperation with internal and external customers such as People’s Daily New Media, Taobao Live, LAZADA and Youku. In 2018, Ali Cloud and Olympic Broadcasting Service co., LTD jointly created Olympic broadcasting Cloud OBS Cloud. This year, the Olympic Broadcasting cloud was put into use for the first time at the Tokyo Olympic Games, providing support for global broadcasting organizations on the cloud. This is the first time in the history of the Olympic Games that cloud computing was used to support global video broadcasting, allowing global audiences to break through the barrier of the epidemic on the cloud.
In the face of the ongoing global epidemic, He Yaming predicted that the demand for video technology will continue to grow in live broadcasting, conferences, e-commerce, entertainment and collaboration — “With the development of 5G, AR and VR technologies and the improvement of infrastructure, lower latency (< 100ms), higher resolution (8K+), more immersive (3D holographic, The interactive way of surround sound will change many industries. In addition to people, audio and video will also make people and things, things and things to establish more connections, human interaction will be upgraded again. Remember the popular saying in the media: the beginning is the end. It means that human beings first received information and felt the world by sight, from the initial voice to text to pictures to videos, and finally returned to the original form. I don’t think that’s entirely true. Video interaction is still evolving, and Matrix and number one players, including the recent mega-universe, have given us a vision of the future of communication.”
From cloud to innovation, new technology and new scene of video cloud
Topic
⏰ Time: 2021/10/30 14:00-18:00 🚀 How to participate: Coordinate Beijing, offline participation (free)
Scan the QR code in the picture or click to read the original text for more information about the special event ↓↓↓
Scan code into group to learn more about LVS conference and video cloud information ↓↓↓
“Video cloud technology” your most noteworthy audio and video technology public account, weekly push from Ali Cloud front-line practice technology articles, here with the audio and video field first-class engineers exchange exchange. You can join ali Cloud video cloud product technology exchange group, and the industry together to discuss audio and video technology, get more industry latest information.