Abstract: Is there a kind of presentation that does not pursue business as the first goal, does not spend a lot of money, does not need to remake, does not gild the lily, low cost can be appreciated to make the classic clearer?
This article is shared from Huawei cloud community “in addition to re-screening and remake, review the classic of the third possibility”, the original author: audio and video butler.
After a gap of 12 years, Avatar has been re-released, and its cumulative domestic box office is estimated at 1.48 billion yuan!!
Classic is classic!
Speaking of classic re-releases, it’s hard to think of a movie like Avatar without the oscars for cinematography, visual Effects and art Direction. Including “Titanic”, “Kung Fu” and so on, most of them spend a lot of money to repair, or even restore the 3D version, objectively speaking, the market effect is not good.
In order to reproduce the classic, remake is also a way, such as “A Chinese Odyssey III”, “A Chinese Ghost”, “The Legend of the White Snake”, the result of the audience did not pay. After all, in addition to the plot and director, there are also actors and characteristics of The Times and other factors to become a classic, so the remake will lose its original taste carelessly.
Is there a presentation that does not pursue business as the first goal, does not need to spend a lot of money, does not need to remake, does not gild the lily, low cost can be appreciated to make the classic more clear?
There are! Because people’s pursuit of the classic film wonderful again, never stop. In order to make the classics shine again, Huawei Cloud Video cloud has opened the way of hd restoration of Hong Kong classic movies.
This weekend, relive the memories of the post-80s generation in huawei video APP.
A, film repair to do what
Let’s start with a category setting: today we’re talking about cinematic AI fixes. So a costly, multi-human Titanic restoration is out of the question.
The most intuitive thing about film restoration is to make the image sharper. The first step is to understand why the image is not clear. Our intuition is: too long.
From a professional point of view, in addition to video repair, film repair also needs to do video enhancement processing, so as to achieve better results.
1. Video repair
The problems to be solved in video repair are: video quality problems caused by various reasons in video shooting, compression, transmission, storage and other links. These picture quality problems include scratches, compression distortion, noise and so on. To help you get a better sense of how these problems affect the picture, we have found some of the most prominent aspects of the problem of the picture.
Scratches caused by damage to storage or handling
Distortion caused by compression
Noise caused by limited filming equipment
Of course, there’s not just one problem with the above images. Similarly, in the real scene of film restoration, there are often many kinds of distortion, and the degree of distortion varies greatly. You have to take it all into consideration.
2. Video enhancement
What are the main parts of video enhancements? How can the naked eye tell the enhancement?
The effect of video enhancement is generally reflected from four aspects: resolution, frame rate, dynamic range and color gamut.
High resolution
Resolution in this case refers to the number of pixels a display can display. The more pixels a monitor can display, the finer the picture. Obviously, 4K is much sharper than 720P. Video enhancement can improve resolution.
High frame rate
We all know that a video is made up of thousands of frames of pictures. If you think of the video on the left as 24 frames per second and the video on the right as 60 frames per second, it’s clear that the video on the right looks smoother.
High dynamic range
High dynamic range is referred to as HDR, and standard dynamic range is referred to as SDR. Generally speaking, on the basis of SDR, HDR can make the bright parts of the picture brighter and the dark parts darker. The range of colors can be increased from BT709 to BT2020. And the color changes will be smoother.
Wide color gamut
Wide gamut, can be understood as a separate color range adjustment, from the original 709BT to 2020BT.
In a word, video enhancement is mainly to enhance the effect in the above four aspects, to solve the problem that due to the upgrade of display hardware, the video previously collected and produced cannot be optimally matched.
Two, the three major challenges of film restoration
At present, it is not easy to automatically restore old movies, facing three challenges: data, algorithm, evaluation.
1. Data challenge: Generalization ability of blind repair task is improved
Film distortion often contains many of the above problems, also known as hybrid distortion. Blind repair means that before repair, it is not clear what problems exist in the video and the generalized algorithm is used. The challenges are as follows: how to improve the adaptability of the generalized algorithm to deal with various repair scenarios; How to use a small amount of comparative data to improve the learning effect.
Hybrid distortion
Intensity distortion
2. Algorithm challenge: How to improve the quality of all aspects of images at the same time
As in the case of distortion, video enhancement is often used in multi-task combinations, such as upgrading 1080p 30fps SDR to 4K 60fps HDR video. That means changing the resolution from 1080 to 4000, increasing the number of frames per second from 30 to 60, and changing the standard dynamic range to high dynamic range. The best way to speak human language is to make a video with higher pixels, clearer lighting, fuller colors, smoother color transitions, and smoother motion. ** This requires joint enhancement for optimization between multiple tasks.
3. Evaluation challenge: To evaluate the quality of generated images
The traditional evaluation algorithms mainly include PSNR, SSIM and VMAF, but their performance is not good. The output signal characteristics of the repair and enhancement algorithm based on deep learning are greatly different from the original image. For example, after certain processing of the image Reference, Patch0 is obtained by using the traditional evaluation algorithm, while Patch1 is obtained by our subjective evaluation. It shows that the traditional evaluation algorithm is not consistent with the subjective evaluation algorithm, which requires a new evaluation algorithm.
How does the cloud repair movies
First come to intuitively feel huawei cloud video cloud repair movie before and after the comparison
Huawei Cloud video before and after the restoration of Huangfeihong
Click to see the video
1. Technical architecture of film restoration
In order to make the restoration lightweight, low-cost and adaptive, Huawei Cloud Video cloud has built a complete set of film restoration technology architecture. The architecture consists of quality detection, algorithm scheduling and subjective and objective evaluation, as follows:
2. Specific operations
- Mixed distortion repair
Huawei Cloud Video takes traditional intensity detection as a prior input when dealing with distortion problems, that is, to predict possible distortion problems, input them into the AI algorithm, build a multi-task hybrid distortion repair model, and finally repair them.
- Spatial enhancement: Video hypersegmentation based on semantic features
Hypersegmentation refers to the use of media processing technology to transform a low-resolution image into a higher-resolution image. Super points implementation principle is: first use of a large number of high resolution image accumulation and learning, learning with low resolution image with high resolution image learning model is introduced to recover, finally get the high-frequency details of the image, get a better effect of image restoration, image recognition and identification accuracy.
For most movies and TV dramas, audiences tend to pay more attention to faces and subtitles, so these two are important enhancement areas with strong priori. Huawei Cloud Video cloud realizes an adaptive hypersegmentation algorithm based on semantics from the perspectives of network lightweight, stability and comprehensive effect. For comparison, please refer to the following figure:
- Time domain enhancement: Interpolating frames
Airspace and time domain is a relative concept, if the airspace is understood as a certain picture in the film, then the time domain is more concerned with the connection between the pictures. If the spatial enhancement is the enhancement of each picture, then the temporal enhancement in visual perception, usually refers to the film between the picture and the picture of the connection more smooth and smooth. So in the process of video enhancement, frame insertion is strongly dependent on motion estimation. This estimation accuracy will be affected by occlusion, large motion of small objects, repeated texture, motion blur and other scenes.
Huawei Cloud Video cloud revises the optical flow by improving the network and combining traditional methods, such as adjacent motion vectors in the reference time airspace, as follows:
- Contrast and saturation enhancement
Existing contrast and saturation enhancement algorithms based on deep learning are difficult to achieve good results due to the lack of data. Huawei Cloud Video cloud can solve data problems better by searching for statistical rules of images in brightness and color space and serving as network prior input.
So, where can I see it?
Xiaobian not to keep in suspense, please move to Huawei video – movie – HD repair, welcome to compare suggestions.
This weekend, let’s revisit some of Chow’s old movies, as well as some of his old classics.
The same fragment, now you and once you, the mood is the same?
Click to follow, the first time to learn about Huawei cloud fresh technology ~