directory

1. Background: Why improve picture quality?
2. Picture quality evaluation: How to evaluate the picture quality?
3. Picture quality improvement: How to improve picture quality end-to-end?

Background:

Short video industry status quo

As can be seen from the third party data below, mobile Internet traffic is close to saturation, and the year-on-year growth rate continues to decline, with the monthly year-on-year growth rate dropping from 2.3% in 2019 to 1.7% in 2020. Mobile Internet has entered an era of stock competition.

But there is no doubt that the short video industry has grown into one of the hottest markets in the world. As can be seen from the data in 2020, the monthly number of short video users in China has reached 800 million. Affected by the continuing epidemic and other factors, the monthly number of short video users in China has reached 900 million. The average daily usage time of short video users increased rapidly, while comprehensive channels showed a slight weakness. Short video apps also led the industry in terms of usage time.

Douyin submission and consumption

Mp.weixin.qq.com/s/70-OfjFPc…

As a national short video community, Douyin has a large number of users and short video content, with more than 600 million daily active users, 10 billion daily broadcast volume and more than 400 million daily video search times. Daily output of UGC content 8,000W +, original short video library reached 100 billion. The balance between the cost and experience of processing multimedia content in the context of massive short videos continues to bring us new challenges and surprises.

Image quality affects users

With the improvement of infrastructure and the arrival of 5G, it can be seen that the number of 5G equipment has increased by more than 8 times in 19-20 years, and the demand for higher and higher picture quality is increasing. At present, we are constantly improving the resolution. Bit rate, frame rate, bit depth, and support HDR video full link acquisition, editing, playing and so on;

Next, I will introduce the factors that affect high picture quality, mainly from the five aspects of resolution, bit depth, frame rate, gamut and brightness.

1. Resolution = the degree of detail in the image

Resolution refers to the number of pixels in an image. At a given screen size, the higher the resolution, the more pixels and the finer the detail. 4K ultra HD has a resolution of 3840×2160, which means it displays four times as much image data as standard full HD.

2. Bit depth = degree of fineness of color gradient

Bit depth refers to the number of colors that can be displayed per pixel. The greater the bit depth, the more colors can be displayed, making the gradient smoother and more natural. For example, an 8-bit display can display about 16.77 million colors, while a 10-bit display can display about 1.07 billion colors.

3. Frame rate = smoothness of motion

Frame rate refers to the number of images displayed in a second. A movie typically has a frame rate of 24P (24 pictures, or frames per second), while standard television typically has a frame rate of 60i or 30p (30 frames per second).

The 8K broadcast standard BT.2020 includes a definition of frame rates (up to 120p) at which motion is almost as smooth as in the real world.

Gamut = vividness of color

Gamut refers to the range of all colors that can be displayed. The chart on the right shows the range of all RGB values perceived by the human eye. Triangles represent gamut: the larger the triangle, the more colors can be displayed. The 4K/8K broadcast standard BT.2020 (REC.2020) covers a wider range of colors than the existing full HD broadcast standard BT.709 (REC.709).

At present, P3 color gamut has been supported together with end-to-end students. Douyin is the first appP3 color gamut to support P3 in China

5. Brightness = intensity of image illumination

Brightness refers to the range of illumination intensity of the image that can be displayed. The range of difference (dynamic range) between the faintest and brightest objects perceived by the human eye is typically 1012, while conventional display devices can display up to 103 dynamic range. HDR extends the dynamic range to 105-100 times higher than current display devices-allowing light and shadow to be presented in a way that is closer to what the human eye can perceive.

Tiktok end-to-end full link

The end-to-end full link path of Douyin is very complex, and each link will have a great impact on the picture quality. Next, let’s sort out the current link from Android terminal. We can see that when we take a photo of a mobile phone, we need to go through sensor for photoelectric conversion, and then the electrical signal is processed by ISP. In this process, manufacturers will add many enhancement algorithms, such as multi-frame HDR, hypersegmentation, noise reduction, etc. And then through the Android Camera1 2 API collection, some major vendors will also provide SDK, we can use these SDK to call the same image processing capabilities as the system camera;

At the production side of Douyin, the business logic, such as special effects SDK, enhancement algorithm SDK, editing SDK, uploading SDK, etc., has gone through special effects algorithms such as beauty and skin grinding, as well as hard and soft coding and decoding in the editing process, and image quality enhancement algorithms of various software ISP have finally been transmitted to the server side.

The server has to go through heavy processing, including the basic information analysis of the video, and then enhanced processing, finally transcoding into each notch, distributed through THE CDN, this process server core processing is image enhancement algorithm pre-processing and transcoding (H264,HEVC, VVC, etc.);

When the player is delivered to the consumer end, it needs to do a lot of processing, such as unlocking and decoding, enhancing SDK processing through algorithms, and finally rendering processing on the screen, such as HDR and super points.

Can see the above link has roughly is very complicated, so the quality of each link effect of offline evaluation is made, we will through the media lab to carry on the subjective and objective evaluation, early insight into user algorithm effect of be fond of, ensure not attract new engineering process quality problems, timely adjust quality direction adjustment, To ensure that the algorithm can reach the optimal effect before online; At present, we have launched image quality enhancement algorithms such as hypersegmentation, noise reduction, frame insertion and HDR in the whole Tiktok link, and achieved remarkable benefits in image quality.

Image quality evaluation

The pain points

According to VMAF and PSNR values, the SROCC values of VMAF and PSNR are not high, which reflects that although there are objective evaluation indicators, objective evaluation indicators are actually difficult to quantify the effect of subjective viewing. For example, is the picture quality of videos with high PSNR necessarily good? Not necessarily, because PSNR is only a relative reference indicator. If the quality of the video source is poor, no matter how high the PSNR is, in addition, most of the time when we watch PSNR, we just look at the mean of all the frames of the whole video. However, if badCase occurs in some of the frames, such as a large number of mosaics in the transition, it will also greatly affect the subjective experience of users. All these can not be well reflected from the value of PSNR. At the same time, if the server goes through sharpening, hypersegmentation, noise reduction, frame insertion and other pre-processing algorithms, then the full reference index basically cannot be used. In general, PSNR and other full reference objective indicators have great limitations.

** There is no positive relationship between subjective quality and user behavior. ** Even if there is a certain assessment of subjective quality in the testing stage, the change of subjective quality must be perceived or felt by users before it will have an impact on users’ behavior. Such impact on a single user or a single behavior may be very small, such as watching one more video or watching a video for a few more seconds. However, there is still little knowledge about the operation of the human visual system (HVS). For another extreme example, we have greatly improved the bit rate and resolution, and the picture quality of users will certainly be better, but the playback fluency and power consumption of users will inevitably be unacceptable.

So we need through constant AB experiments to observe the user’s QoE/QoS metrics, reflect and understand the real behavior of users, from the side to find an optimal balance point, is the quality of visual experience positive earnings than mobile phone performance for the users bring extra consumption of negative experience, make the whole overall positive business data;

QoE**** Index core:

Picture quality positive > other negative

AB experimentThe core of theQoE**** indicators are as follows:

Number of users, content views, per capita playing time, per capita playing time, per capita finishing play, per capita likes, multi-day retention, etc

Generally, there are three stages in algorithm development:

1. Algorithm simulation tuning stage
2. Engineering test phase
3. Online AB experiment phase

Therefore, in the offline algorithm simulation development stage, we will heavily rely on subjective and objective quality evaluation to ensure that the effects of all aspects of the algorithm can reach the optimal state as far as possible. Our algorithm iteration process will be introduced later.

Video quality assessment

What is image quality?

With the wide application of image information technology, the evaluation of image quality becomes an extensive and basic problem. As image information has incomparable advantages compared with other information, reasonable processing of image information has become an indispensable means in various fields. In image acquisition, processing, transmission, and record the process, because of the imaging system, the processing method, the transmission medium and recording equipment, such as imperfect, combined with the object motion, noise pollution and so on reasons, inevitably bring some image distortion and mass reduction, this brings people into an awareness of the objective world, the research to solve the problem.

For example, in the image recognition, the quality of the collected image directly affects the accuracy and reliability of the recognition results; For example, systems such as teleconference and voD are affected by transmission errors, network delays and other adverse factors, which require online real-time image quality monitoring; Therefore, the reasonable evaluation of image quality has very important application value.

There are two definitions of image quality.

Definition 1: accuracy of signal acquisition, processing, storage, compression and transmission by different imaging systems.

Definition 2: Evaluating image quality from the perspective of human perception: a weighted mixture of all visually important attributes of an image.

Definition one concerns the accuracy of signals in the imaging system from the perspective of signal processing.

The second definition is to pay attention to the feelings of observers from the perspective of human vision. Observers with different aesthetic levels will have different feedback results.

So from the definition of image quality, it extends out two kinds of image quality evaluation methods, from the point of view of whether people participate in the differentiation, image quality evaluation method has subjective evaluation and objective evaluation of two branches.

• Subjective quality evaluation: Subjective evaluation takes people as observers to conduct subjective evaluation on images, striving to truly reflect people’s visual perception, and relying on human eyes to watch and score. Such scores are more accurate, but time-consuming and inconvenient for large-scale deployment.

• Objective quality evaluation, objective evaluation method with the help of some mathematical model, reflects the subjective perception of human eyes, gives results based on numerical calculation, evaluation of an algorithm is to measure the correlation coefficient between subjective score and objective score, generally speaking, the higher the coefficient, the better.

Objective quality assessment algorithms can be divided into three categories, mainly depending on whether lossless source video is used as a reference.

• Full reference, such as PSNR, is a typical full reference algorithm, which measures the quality of damaged video by comparing it with the source video at various levels.

• Without reference, some algorithms do not use the source video, but only the video at the receiving end, to measure its own quality.

• Partial reference, such as extracting a feature vector from the source video, which is sent to the client along with the damaged video to calculate the quality.

Objective evaluation of Pica

At present, there are three mainstream test systems in the world: IMATest, DXO and Image Engineering.

MTF

MTF, Modulation Transfer Function. Modulation transfer function. MTF is often used in various camera lenses to describe the MTF curve of the lens, indicating the capability of the lens. These curves are tested in an ideal test environment that minimizes the attenuation of the analytical power of other systems on the lens. Of course, MTF can also cover the analytical power evaluation of the entire imaging system.

Signal-to-noise ratio SNR

Since noise and signal are always mixed together, an image that removes noise and signal at the same time is not a good image. Therefore, in order to judge the quality of the image, SNR is often used to express.

We usually use dB to express:

So SNR of dB is equal to 20 log 10 S over N.

The most significant disadvantage of SNR is that it can be easily cheated by noise reduction algorithms. An image with a high SNR can be heavily smeared. In addition, two graphs with the same SNR may have very different noise perception.

Chromatic aberration calculation in CIE Lab

Build color space, it is to express color conveniently. Now that you have the coordinates of the two colors, you can start the chromatic aberration calculation.

This is where we come across the first problem, how to choose the color space.

Standard-setters consider two things.

1. Color space must conform to human vision and be able to compare linear colors.

2. To calculate in the color space unrelated to the device, not because of the change of a display mode, affect the calculation of color difference.

CIE Lab was born out of this need. L represents brightness and AB represents color components.

CIE Lab was considered to be a homogeneous linear space, so the original color difference formula was the Euclidean distance between two colors in the color space:

delta_Eab = sqrt(delta_L.^2 + delta_a.^2 + delta_b.^2); delta_Cab = sqrt(delta_a.^2 + delta_b.^2); Where Delta E is the overall color, deltaC excludes the influence of brightness (L value is not calculated)Copy the code

After the chromatic aberration formula is gradually recognized in the application of some shortcomings still exist. CIE updated the standard again in 2000. This time the formula is more consistent with human perception.

The formula is as follows:

I have omitted the calculation of the intermediate variable, showing only the final formula. If you are interested, go to the appendix to download the CIE 2000 chromatic aberration formula MATLAB functions.

Objective Laboratory Introduction

Objective laboratory believe everyone is familiar with, we mainly rely on standard drawing card and controllable light environment, to repeat the test on variables of controlled, the current laboratory in detail we can see our previous article: mp.weixin.qq.com/s/t3mDJzneN…

1. 24 color swatches,
2. • leaves figure
3. • SFRplus figure
4. • ISO12233
5. Light boxes,
6. • Tuka stand

No reference scoring algorithm

At present, our own non-reference scoring algorithm is also used internally.

1. Online video quality monitoring

Quality monitoring of massive video data is one of the thorny problems we have to face. Video quality service includes video quality detection, video quality evaluation and video quality monitoring. If you can provide a fast and correct review or monitoring solution for a large number of videos, you can check video quality 24/7, significantly reduce labor costs, and reduce the risk and accident rate of online services.

• The production end of Douyin releases video picture quality monitoring, the service end transcodes video picture quality monitoring, and the online market playback quality monitoring;

• In other lines of business multiplexing fly book, watermelon, cut ying and other lines of business such as short video, live, RTC scene;

2. Video pre-processing detection

At present, The end-to-end video processing technologies in Douyin include: block removal, sharpening, super resolution, video noise reduction and other technologies. ** The biggest feature of video processing technology is that it is difficult to find an objective indicator to tell you how to adjust the parameters to satisfy the user.

The following figure provides the ability to conduct comprehensive, multi-dimensional scene and picture quality analysis of images and videos from multiple sources, including but not limited to: Scene semantics, face detection, video type, motion degree, noise, brightness, exposure, clarity, color representation, artifacts, etc., can be used for QA evaluation monitoring system (such as image quality scoring, aesthetic scoring, online image quality monitoring, etc.) and image quality repair and enhancement algorithm system (such as: Scenario-based algorithmic routing, dynamic algorithm branching, algorithm automatic on/off, algorithm condition input, etc.). The following figure shows the analysis and detection modules that AnalysisKit may contain and the business or algorithm scenarios it can serve:

Centralized planning, development and care of scene and image quality analysis and detection algorithms can form a whole set of reusable solutions that can be used in different scenarios and task types, improve the applicability of the algorithm, avoid repeated development of detection algorithms from various demand sources, help clarify the alignment of requirements, and avoid cold start of requirements.

Current applications:

Production end edit page one key enhanced function noise reduction algorithm pre-detection;

Night scene detection function provides information for post-processing of other algorithms;

3. It is recommended to provide scoring labels

Short video is gradually becoming an important way of information dissemination on the Internet. Attaching various useful labels to short video can help optimize recommendation system or search engine, so as to provide users with accurate recommendation or search service for short video.

** Current Applications: ** Quality video screening on TikTok, optimizing recommendation weight;

4. Automatic **** inspection

A huge number of online market contribute daily video some video video produced due to the production of the series of link quality badcase, some of them are caused by a series of process of server-side image quality degradation, some of them are consumers after post-processing lead to quality problems, these problems in the hope of all users to find and report, with feedback link, soon lost, And many front-line users and operations are not sensitive to picture quality problems, it is difficult to distinguish;

However, due to the huge power consumption of the image quality detection algorithm, it is impossible for us to detect all the submitted videos. Therefore, we plan to develop a set of single-point diagnosis algorithm with the combination of upstream and downstream data and the badcase we encounter in daily life together with the video architecture, and then conduct online big-V video quality inspection through this algorithm. Any video that may have quality problems can be thrown out as soon as possible, and then attribution can be made again through artificial subjective analysis, so that various problems of online picture quality can be found as much as possible, and the direction of picture quality optimization can be mined.

Real scene subjective evaluation

• Different color temperatures

• Different scenarios

• Different evaluation latitude

• Laboratory conditions

1. Head mold – Different skin tones

At present, byte products are all over the world. Since most of the research and development testing system is in China, we simulate skin color groups in different regions of the world through highly simulated human head models to carry out portrait quality tests.

2. Live simulation

The biggest difficulty in subjective testing is the repeatability of the scene. Once the scene is confirmed to be problematic, it is difficult for us to repeat the same scene in nature, so we need to create such a real scene artificially in the laboratory for us to repeat the test.

3. Lighting environment

Light conditions are also very important, so we set up A hanging light source, which is convenient for us to simulate various lighting conditions such as D65, D50 and A light for subjective scene testing.

Picture quality evaluation tool

We also use a lot of self-developed tools to improve the efficiency and automation of picture quality evaluation, currently developed 24 color card, SFRplus picture card analysis algorithm, anti-vibration detection algorithm, video basic parameter analysis, stuck analysis, coding analysis read video macro block and so on, the following part of the UI interface;

Picture quality evaluation process

The iterative process of the internal image quality algorithm is very long in the middle, and the following is a brief introduction to the iterative process of the algorithm, so that we can clearly understand that the online of any algorithm is through research and development and evaluation of huge efforts, and finally presented to the user online;

Algorithm starting from the project, it needs to develop and review internal closed loop tuning classmates, debugging unceasingly, after the process to achieve deliverable state, and then through internal figure card and no reference of objective index calculation, in a reasonable range of the threshold, we will only be delivered to the expert review group (about 40 people) to measure, After the multi-dimensional evaluation conclusion of experts is passed, we send it to some online internal test users (about 300 people each time) through the online mass test platform. After the final mass test of users is passed, we decide to launch the project. If any of the above links are negative, we will call back and re-debug.

The measuring way

When it comes to mass testing, we need to focus on our mass testing system. The current mass testing is integrated on Douyin, and users of internal testing can participate in the mass testing process by means of in-site message or scanning two-dimensional code.

At present, there are four methods of mass measurement:

What is JND?

An important indicator of mass measurement is JND. We decide our online standard through JND quantization.

For example: add a candle to a room with one candle, and the increase in brightness is palpable. Ten candles in the room, add a candle, the feeling is not obvious. A bell is shaking, add another bell, and the noise is also obvious. Ten bells are shaking. Add one bell and you don’t feel it. So how do you quantify the physical increase and the psychological perceived increase?

Weber’s law is a law that shows the relationship between psychological quantity and physical quantity, that is, the difference threshold of feeling changes with the change of the original stimulus quantity, and shows a certain law, expressed by the formula, is △I/I=K, where I is the original stimulus quantity, △I is the difference threshold at this time.

The smallest noticeable △I is called JND, Just noticeable difference.

The minimum appreciable difference (continuous difference threshold) is taken as the unit of sensory quantity, that is, for each increase of difference threshold, mental quantity increases by one unit. The sense quantity is directly proportional to the logarithm of the physical quantity, that is to say, the increase of the sense quantity lags behind the increase of the physical quantity, the physical quantity grows geometrically, while the mental quantity grows arithmetically. This empirical formula is known as Fishner’s law or Weber-Fishner’s law. Suitable for medium intensity stimulation. S = k lg I + C (S is the sense quantity, k is the constant, I is the physical quantity, C is the integral constant)

Common applications:

Acoustics, decibel calibration

Classification of pain

Earthquake classification

…

Here are some of the actual cases we use,

Evaluation laboratory

So what do we mainly do? Mainly from the following four directions to control tiktok end-to-end link picture quality;

Engineering optimization — vendor cooperation — codec — algorithm enhancement.

**

**

Subdivided down, mainly through the mobile phone manufacturers new phone cooperation test, image enhancement algorithm test; Engineering iterative optimization test, such as 1080P full link test; Codec picture quality test; Competitive product research and analysis; Evaluation video library; Evaluation platform construction; Automated monitoring and so on.

Quality ascension

The production end

Software ISP

• All ISP pipelines are basically the same, mainly reflecting different processing styles. The ISP complexity depends on the application scenario.
• Typical are traditional algorithms, now there are also traditional +AI hybrid pipelines.
• The more mature ISPs are:
  • Cameras: Canon, SONY, Nikon, etc.
  • Phones: Qualcomm, Apple, Samsung, Hays, MTK.
  • Security IOT: Haisi, AMba, ARM, etc.
  • Driving: Mobileye, etc.
• Post-processing:
  • Various mobile phone factories, Almalence, Visidon, Morpho, IMint, Hongsoft, Shangtang, Kuangshi, etc.

Many positions on the picture processing link can be enhanced by post processing. The picture quality characteristics of different positions are different, understanding these characteristics is helpful to design better post-processing enhancement algorithm, so the software ISP of post-processing algorithm developed by us came into being;

Unable to copy content being loaded

Software ISP

• At present, byte provides image quality solution system, from business scene analysis, image quality intelligent analysis, image quality enhancement and optimization, testing and deployment and a series of capabilities.

Core Platform Capabilities

• Prism **** system provides core capabilities in the form of SDK, including scene strategy module, image quality intelligent analysis module, image quality enhancement module.
• Scene strategy module: business scene configuration, quality parameters debugging, preset quality style debugging, function cutting configuration, working mode configuration, algorithm pipeline configuration.
• Picture quality intelligent analysis module: scene change detection, picture quality real-time monitoring, picture quality problem diagnosis, picture quality subdivision dimension scoring, aesthetic scoring.
• Picture quality enhancement module: photo enhancement ability, video enhancement ability, time series enhancement ability.

Unable to copy content being loaded

Current one-click enhancement functions on Douyin:

Combined with the user and the image scene, the intelligent analysis and enhancement of the image and video quality, dynamic adjustment to a clearer, gorgeous image, to create an excellent video immersion experience, as can be seen from the following figure after the enhanced effect visual sensory experience is significantly better than the right;

The left is the enhanced effect, the right is the unenhanced effect;

Camera collection

What is CamerA1, CamerA2?

Both are two sets of Camera API officially provided by Google. After Android 5.1, the update to Camera1 was cancelled, and the main maintenance was Camera2.

There are big differences between Camera1 and Camera2 in implementation and use, as shown in the figure below.

Camera1 is mainly used in old models and some models with compatibility problems. Camera2 is mainly used in new models.

Compared to Camera1, Camera2 has many new features and more powerful functions, such as:

• Support 30FPS hd continuous shooting;
• Support RAW format picture shooting;
• Movie Snapshot support;
• Support more parameter setting, provide more operation space for picture quality optimization and professional ability expansion;
• Features such as anti-shake and wide Angle are only available on Camera2.
• Most of the mobile phone vendor cooperation functions are also based on Camera2 implementation;

At present, most scenes of tiktok have been switched to camera2. I also cooperated with mainstream mobile phone manufacturers in China, such as huawei, xiaomi, oppo, vivo, etc. I used the SDK provided by the manufacturers to call more capabilities only available to system cameras to further improve the quality of our production.

Edit the SDK

For Android, the process basically follows:

Basically, the YUV frame output from the camera is preprocessed and sent to the encoder to obtain the encoded video stream.

• Hard coding VS soft coding

At present, it is gradually switching from soft coding to hard coding. As can be seen from the figure below, hard coding has lower power consumption and stronger performance.

• H264VSH265

Switch from H264 to H265, which provides a higher compression ratio and provides sharper picture quality at the same bit rate. Here are some of the differences;

• Intelligent precoding

By scoring the performance and measuring the speed of the client’s model, and perceiving the video content in advance, the algorithm selects a set of coding parameters that are most suitable for the current scene.

• HDR

The imported video can be rendered on screen, HDR display is supported for transcoding, and HDR tone mapping to SDR is also supported to avoid badcase generated by users.

• Hd video

At present, the production end supports the maximum 1080P and 60fps video, and we will continue to explore in the future to provide users with higher resolution video as much as possible.

• Trade off

Of course, we are not just trying to improve the picture quality. At present, we have achieved a balance between the picture quality and the bit rate through a combination of various soft and hard coding and upload SDK strategies, so as to improve the picture quality as much as possible without affecting the user’s submission experience. At present, in the internal evaluation, the effect of the production end is the best among the competing products.

The special effects the SDK

On the production side we also did a lot of work on the effects SDK. Here are a few examples.

• Delicate exfoliating

Before our exfoliating effect will be to deal with the global scene, cause behind the figures is not clear, the background of exfoliating treatment at the same time lead to face texture details of too little, lack of emotion, we targeted online follow-up facial skin, only to face detection area for processing, on-line delicate skin, at the same time to preserve texture;

• Night view filter

In the case of night scene, the default filter will magnify the picture noise, through the night scene detection algorithm, automatic use of different effects of the day and night filter, to maximize the filter effect;

• HDR support

Although it is mentioned above that Tik Tok supports HDR for on-screen rendering of imported videos, it is very difficult to support it after special effects are added, requiring more than 2W props to traverse and produce 10bitHDR material. At present, we are also gradually supporting HDR for special effects.

The service side

At present, the processing pipeline of the server is roughly divided into three parts: video analysis, video pre-processing and video transcoding;

Video analysis

Video analysis above we have internal reference rate algorithm research, as shown in the previous article VQscore:mp.weixin.qq.com/s/ZREhRIMye…

At the same time, other analysis processes, such as ROI, basic feature analysis, complexity analysis, video basic parameters, etc., rely on the above analysis for final decision processing;

Video processing

In terms of video pre-processing, there are many types of algorithms that have been and are being studied. We combine end-to-end data for decision processing. At present, we mainly use super-score, low-quality enhancement, de-compression distortion and so on

Video coding

The coding standard defines the format and process of the decoder, while the coding end can be optimized by each company. For example, for the motion search module, algorithm design and optimization can be carried out in the range, size and module selection. Therefore, the same standard encoder can have different encoder algorithms, each company has its own unique design.

Similarly, our company has developed its own encoder algorithm, which has been applied to various businesses. We research the encoder also achieved good results in the MSU, 17 first mp.weixin.qq.com/s/Fa3kDWtw0…

Current applications:

Adaptive coding, ROI coding, HDR support, super points, low quality enhancement, compression distortion, frame insertion, noise reduction have been used in business scenarios, some algorithms to obtain very significant benefits, such as ROI coding, a year to reduce the cost of bandwidth up to several hundred million;

The consumer end

At the consumer end, we have also launched hyperscore and other image quality enhancement processes. The following is a simple pipeline for hyperscore on the player. After a long time of experiment, this process achieves a balance between performance and picture quality.

This is an online demo of the actual over-dividing effect. It can be seen from the video that the clarity benefits brought by over-dividing are very significant. The internal data showed that the core business indicators (per capita duration, per capita VV and per capita completion rate) of the super-score experimental group were significantly positive, the fluctuation of next-day retention of all users was positive, and the continuous positive trend of per capita active days was not significant.

At present, Chaofen has a wide range of internal business applications, such as Douyin, Vod, live broadcast, watermelon, Toutiao, pipi Shrimp and so on. Oversplitting not only brings benefits of picture quality, but also caton optimization and bandwidth optimization through different strategies.

future

At present, Chaofen has a wide range of internal business applications, such as Douyin, Vod, live broadcast, watermelon, Toutiao, pipi Shrimp and so on. Oversplitting not only brings benefits of picture quality, but also caton optimization and bandwidth optimization through different strategies.

• Continuous polishing without reference scoring algorithm, able to ground more fine-grained daily test scenarios;
• Explore more accurate subjective quantitative scheme, can give more accurate conclusions;
• AR VR MR and other future scenes to explore image quality evaluation methods and schemes;
• HDR’s more comprehensive and rigorous test program;
• High resolution, frame rate; Such as 8K, 120FPS evaluation scheme exploration;
• How beauty effects depend on subjective and objective to form systematic standards;

Join us

Tiktok Multimedia Evaluation Lab is byteDance’s medium center for multimedia picture and sound quality assurance. Through professional reviewers, laboratory equipment and industry-leading evaluation solutions, tiktok serves a wide range of products within Bytedance, including Apps such as Tiktok, Toutiao, Jianyin and Watermelon Video. End-to-end full-link image quality is evaluated through cooperation with mobile phone manufacturers, video encoding and decoding, image algorithm enhancement, engineering optimization and other aspects to comprehensively improve user audio-visual quality experience. The team develops rapidly, is young and vigorous, pays attention to the construction of technical atmosphere, actively participates in the domestic and foreign top industry technical conferences, outputs high-quality technical patents and related papers, and works in Beijing, Shenzhen, Hangzhou, Shanghai and many other places to choose. At present, the team urgently needs image test engineers and other positions to join us, so that each line of your code can serve hundreds of millions of users! Welcome to join us! job.toutiao.com/s/R58Us4j