Make writing a habit together! This is the second day of my participation in the “Gold Digging Day New Plan · April More text challenge”. Click here for more details.
For more blog posts, seeTotal catalog of romantic carriages for audio and video systems learning
YUV color coding analysis video coding principle — From The film of Sun Yizhen (1) Analysis video coding principle — from the film of Sun Yizhen (2)
The last video basic knowledge literacy introduced the most basic knowledge of video. In the deep chapter, RGB color coding was mentioned, which is also the most commonly used color coding. However, in the process of video transmission, in order to save volume, another color coding is often used to achieve, that is, YUV.
Generally speaking, we usually contact with the color coding is RGB, such as most of the display is RGB, usually do development in the program to set the color is RGB, so what is YUV, and RGB what connection?
YUV is a color coding method. Often used in various image processing components. YUV allows for reduced chromaticity bandwidth, taking into account human perception when encoding photos or videos. For example, the image data captured by the Camera on the mobile end and the stream data of the live stream are all encoded in this format.
Closely related to color-coded YUV is color space YCrCb, where Y “represents Luminance and Luma, and is the weighted sum of RGB components:
“Cr” and “Cb” are Chrominance and Chroma. “Cb” refers to the difference between blue value and brightness, and “Cr” refers to the difference between red value and brightness:
But now the color coding YUV and the color space YCrCb are often confused, so the ‘U’ mentioned later can be regarded as equivalent to ‘Cr’, and ‘V’ can be regarded as equivalent to ‘Cb’.
RGB appeals to the perception of color by human eyes. Compared with the three components of RGB that are color related, YUV’s biggest feature is the separation of brightness and chroma.
YUV focuses on the sensitivity of vision to brightness. YUV allows for reduced chromaticity bandwidth, taking human perception into account when encoding photos or movies. In other words, encoding allows for more Y than UV, allowing for downsampling of the UV component of the image so that the data takes up less space than RGB (downsampling simply means sampling at a lower rate than the original sample). For details on Downsampling, please refer to This article on Zhihu. Oversampling, undersampling, downsampling, upsampling what are the differences and connections between the four concepts of oversampling, undersampling, downsampling, and upsampling? .
Y, U, and V in the image:
It’s a Bit abstract, so take a look at this famous article from Microsoft: Video Rendering with 8-bit YUV Formats
Here we mainly talk about two aspects of YUV, which are the sampling format and the storage format. The sampling format is simple to understand how each pixel samples each component of YUV in an original image. For example, y component (or U or V) is collected every few pixels. The storage format simply refers to the way in which the sample is stored, such as which byte is stored y and which byte is stored U.
Yuv sampling format:
The “YUV Sampling” section of the article explains in detail how yuVS of various formats are sampled. Here is a translation of the excerpt:
One of the advantages of YUV is that the sampling rate of chromaticity channel is lower than that of Y channel on the premise that the perceived quality does not significantly decrease. Generally, A symbol called A:B:C (y:u: V) is used to describe the sampling frequency of U and V relative to Y. For easy understanding, it is described in the figure, where y component is represented by X and UV is represented by O:
4:4:4:
It means that the chromaticity channel is not down-sampled, which means that all three channels of YUV are fully sampled:
4:2:2:
Represents a 2:1 horizontal down sampling, no vertical down sampling. Each scan line contains four Y samples corresponding to two U or V samples. That is, the method of sampling in the horizontal direction according to Y: UV with 2:1 and full sampling in the vertical direction:
4:2:0:
It represents 2:1 horizontal sampling and 2:1 vertical sampling. That is, the horizontal direction according to y: UV using 2:1 sampling, vertical direction according to Y: UV using 2:1 sampling:
Note that 4:2:0 here does not mean that Y :u:v = 4:2:0, it means that only one chromaticity component (U or V) is scanned in each line, and the y component is sampled in the manner of 2:1. For example, YU samples the first row in a 2:1 fashion, while YV components are sampled in a 2:1 fashion in the second row. So the ratio of y to either u or v is 2:1.
4:1:1:
Represents 4:1 horizontal sampling, no vertical sampling. Each scan line contains four Y samples for each U or V sample.
4:1:1 sampling is less common than other formats and is not discussed in detail in this article.
Yuv storage format:
YUV storage formats come in two broad categories: planar and Packed: Packed: Y, U and V components are stored in an array. The Y,U, and V of each pixel are continuously interleaved. And RGB storage format similar. The planar: Y, U, and V components are stored in three separate arrays.
Y, U, and V each sampling point uses 8 bits for storage.
Next, we will elaborate on the common YUV format storage methods in the next video:
4:2:2 formats:
There are two main specific formats:
YUY2:
It is of type Packed and YUY2 format. The data can be treated as an unsigned char array. The first byte contains the first Y sample, the second byte contains the first U (Cb) sample, the third byte contains the second Y sample, the fourth byte contains the V (Cr) sample, and so on, as shown in the figure:
UYVY:
Also belongs to the Packed type, and is similar to YUY2 and, but the storage direction is opposite:
4:2:0 formats
The format also contains a variety of storage methods, here will focus on the following:
YUV420P and YUV 420SP are stored in Planar mode. After storing all Y components, YUV420P stores all U or V components. YUV420SP is stored in alternating order of UV or VU, see the following figure (figure from: basic knowledge of audio and video – pixel format YUV) :
YUV420P:
(Here needs to type the blackboard, because this article plays yuV is YUV420P format, familiar with its storage format can understand the code to read the logic of video frame data)
Because YUV420P is sampled 2:1 horizontally and 2:1 vertically, the number of y components is equal to the width of the video, and both the U and V components are the width of the video times the height /4
YUV420SP
Yuv to RGB:
At present, the general decoded video format is YUV, but the general graphics card rendering format is RGB, so yuV needs to be converted to RGB.
There’s the concept of Color Range. Color Range is divided into two types, one is Full Range, the other is Limited Range. Full Range R, G, and B Range from 0 to 255. The values of R, G, and B in the Limited Range Range from 16 to 235.
For each Color Range, there are different conversion standards, the common standard is BT601 and BT709 (BT601 is standard definition, and BT709 is high definition standard).
So for different Color ranges and different standards, there are 4 conversion formulas combined.YUV & RGB: The original image is so colorful) :
conclusion
This paper mainly introduces the basic concept of YUV and YUV sampling format and storage format as well as YUV and RGB conversion, which is very key to understand video coding and rendering in the future.
The next chapter begins to enter the chicken frozen heart part of video coding: Analysis of video coding principle — from the film of Sun Yizhen (1)
Reference:
Downsampling (signal processing) Oversampling, undersampling, Downsampling, upsampling what are the differences and connections between the four concepts? YUV Formats (8-bit YUV Formats) Video Formats YUV Formats (8-bit YUV Formats
Original is not easy, if you feel that this article is helpful to yourself, don’t forget to click on the likes and attention, but also to the author’s affirmation ~