Have you watched the World Cup live? Are you impressed by the clarity and fluency of CCTV’s video and audio? Let’s take a look at the feedback:
As the copyright owner and content distributor of this World Cup, CCTV has high technical requirements for the support service of this World Cup live broadcast. CCTV network finally chose Tencent cloud video cloud speed HD products, to provide its users throughout the country with smooth and stable HD live services.
From the point of view of the effect, the speed HD service has performed well after the official launch and operation of CCTV.com. After the World Cup, the user experience has also formed a sharp contrast with other platforms, which has changed the previous users’ views on the live broadcast of large-scale events on CCTV.
This paper, starting from the working principle of speed HD, introduces in detail how Tencent cloud’s deep learning technology optimises the World Cup live scene, including how Tencent Cloud won the PK, how to complete the privatized deployment of CCTV net speed HD service within a week and other unknown stories.
Tencent Cloud Speed HD: How to maximize deep learning efficiency?
The main quality problems of common transcoding include transcoding distortion, low resolution blur, camera jitter, high noise, low bit rate sawtooth, etc.
Topspeed HD service is an intelligent dynamic coding technology newly launched by Tencent Cloud Video Cloud. The main principle is to process live video streams intelligently through deep learning, so as to provide users with higher resolution live video streams at lower bandwidth costs. Mainly through the following modules to complete cooperation.
- Intelligent classification of video scenes
First of all, through the method of deep learning, a dozen mainstream categories and dozens of small categories of scene model library including games, sports, shows, outdoor, animation, food, films and TV dramas are formed. During the beginning of ultra HD service, the real-time live streaming was detected and analyzed, and the corresponding scene model was matched. In terms of matching rate, the matching accuracy of CNN network model reached more than 98% for obvious scenes such as games, football, basketball and animation. CNN+RNN+LSTM analyzed the time domain and space domain with an accuracy rate of more than 85% in scenes where the screen features of TV dramas, outdoor sports, food and tourism were scattered and the motion between frames changed greatly.
- Encoding parameter matching
According to the real-time recognition results of scene classification in the first step, the optimal coding parameters are selected by combining the source rate, frame rate, resolution, texture and motion variation amplitude of the video, as well as the comprehensive dimensions of machine load and picture quality effect.
- pre-processing
According to different scene classification, customer’s different requirements for video picture quality, and combined with the video source image texture and real-time motion detection results, pre-processing such as sharpening, soft blur, de-interlacing, block removal, noise reduction, color level compensation, frame reduction, etc.
- Coding dynamic optimization
Tencent Cloud speed HD service for different video categories, or different video segments within the same video, the application of completely different coding parameters. Including but not limited to: IBP frame type, quantitative parameter QP, resolution, etc.; And support encoding parameters by frame real-time update effect.
- Bit rate intelligent control
Since both the standard H.264/JVT-G012 bit rate control algorithm and the X264 bit rate control algorithm, the theoretical convex curve of rate distortion obtained by preprogramming is as close to the optimal distortion curve as possible in the scene with obvious motion change switching. For the scene frame with obvious motion change switching detection in real time, On the basis of x264 bit rate control, we will integrate the frame timing information to do some compensation factors and picture quality control, so that the subjective viewing score VMAF value can be increased by 3-5 points.
Note: Speed HD has lower and more accurate bit rate control under the same VMAF score.
- detail reduce
Aiming at h264 video coding format, we design an algorithm to eliminate noise in frequency domain of video residuals. This algorithm adaptively performs macroblock-level video processing by combining the residual size of the current encoded macroblock, the QP value of the macroblock, and the historical frequency domain value, and selecting a matching video denoise template based on different scenarios. It can optimize noisy macroblocks with very low CPU consumption while preserving the integrity of clear macroblocks.
- ROI processing
SIFT+ differential image + moving target image detection or user specified (such as table standard, watermark) pay attention to the ROI area, the macro-block coding of this area for strengthening (zoom/zoom quantization step), sharpening, color level compensation and other processing, such as CCTV World Cup requires the image quality and brightness enhancement of table standard and subtitle area.
Tencent Cloud video cloud mainly through the above module cooperation processing in the cloud to open Tencent cloud speed HD service.
From simulation to cut: 500+ World Cup rehearsals
After the Spring Festival of this year, Tencent cloud Speed HD service began to connect to some head game live streaming platforms one after another, but we have not contacted customers of sports events, especially football events. Before the World Cup, we did a systematic machine learning and visual effect optimization evaluation for football match scenes, in order to get ready for the upcoming World Cup.
- Preview: Machine learning and visual tuning
We collected through web crawler 500 football games, video, video for every game, in high speed motion scene, football follow model training, long lens optimization, highlight replays ROI area increase, the audience, capturing details such as facial expression to make visual effect analysis and optimization effect evaluation, the optimal dynamic coding model in order to make the World Cup.
- China Super: test the water speed hd service
Although we have been optimizing the dynamic coding model of football matches by ourselves, we are still a little worried about the lack of real landing users. Around mid-April, we took the initiative to promote the super Hd service for Chinese Super League games live on Dragon Ball, and gradually added the super HD service for some Chinese Super League games. Although there were some problems at the beginning, the effect was gradually optimized to meet the needs of customers.
- Premier League: cut the amount of promotion
After the practice in the early stage of Dragon Ball Super League, we had a basic foundation in mind, and began to actively look for some customers for sports events in the later stage. New British sports as proprietors of the premier league in the mainland, basic dominance in the premiership game (and much faster Yu Zhongchao’s premier league game, this scenario will be more matching with World Cup, we has carried on the speed of hd services to new British sports promotion, under several rounds of testing and optimization, the final client authentication.
Invisible PK: Dozens of service upgrades and optimizations
CCTV as the whole video cloud industry are extremely valued customers, cloud manufacturers are competing. Following CCTV’s decision to introduce new technology to improve the clarity of matches during the World Cup, cloud service providers have offered their own solutions. After several rounds of testing, CCTV.com finally decided to choose between Tencent Cloud and a cloud manufacturer. As the World Cup is approaching, it is a great test for us. In the PK process with the opponent, we have carried out dozens of service upgrades and optimization on the modules of bit rate control, de-blurring, de-blocking, anti-interleaving, motion search and block decision making. After round after round of PK, CCTV.com finally chose Tencent Cloud.
Note: Ultra HD has a slight advantage in VMAF quality, bit rate is about 8%-10% less than other vendors.
How did we go private in one week?
Although we won in terms of technical indicators, due to the particularity and urgency of the World Cup and the customer’s temporary lack of matching hardware resources such as servers, the demand became: bring your own machine for private deployment. In fact, in the early stage of our communication with customers, we are clear about the “privatization deployment” and are also preparing for it. However, time is limited at this time. Faced with the requirements of “server delivered”, “switch delivered” and “operation and maintenance personnel provided by themselves”, there is too little time for the team to prepare. But we do not want to give up, and good service for the HD live broadcast of the World Cup, for users, for customers, for Tencent are extremely important. After urgent negotiations, we came up with a new privatization plan.
- Privatize deployment targets after upgrading
In CCTV network build a set of Tencent cloud speed HD transcoding cluster, support CCTV network during the World Cup CCTV5 channel live, RTMP/FLV pull source stream, after transcoding processing support CCTV RTMP/FLV/HLS pull stream.
- Privatized machine deployment after upgrade
The customer only has rack resources in the machine room. Considering that it takes too long for the group to apply for procurement if the customer prepares relevant hardware resources. Under normal procedures, there would be no time to serve this World Cup. Therefore, we borrowed 10 M10 transcoding machines from the company. Other related auxiliary equipment, such as switches, optical modules, optical fibers and network cables, were also purchased separately through the cooperation of relevant students. When deploying servers and other hardware, the developers and architects even wired and twisted screws themselves. (This is the real full stack engineer)
We were more nervous than the players at the opening whistle
Finally, before the World Cup we compatible with the customer hardware equipment and software code module of all kinds of adaptation requirements, then is the actual test.
Nerves were still high before the first day of competition, and if anything went wrong after the live broadcast, it would all go to waste. After the game started, we stared at the screen nervously: all the service indicators were normal, the streaming effect exceeded expectations, and the broadcasting effect was obviously superior to other platforms. After the first day of the competition, we got great reviews from our customers and the feedback from our external users was a relief. However, the competition has only just begun, a month of the World Cup, we will also continue to fight in the invisible place!
Author: Li Haiqi, senior product manager of Tencent Cloud