A, an overview of the
In the process of RTC communication, images need to be collected, beautified, compressed, encoded, transmitted, decoded and displayed. Many image formats will be involved in this process, such as YUV and RGB, among which YUV and RGB also have many categories respectively. Today we will introduce these image formats in detail.
Two, YUV format
YUV is a type of image format used in video, pictures, cameras and other applications. It is the name of the color space common to all “YUV” pixel formats. Unlike the RGB format (red-green-blue), YUV is represented by a “luminance” component called Y (equivalent to grayscale) and two “chromaticity” components, called U (blue projection) and V (red projection), hence the name.
YUV can also be called YCbCr, and while these terms mean slightly different, they are often confused and used interchangeably.
1. What is YCbCr
Y represents brightness and CbCr represents color. To represent colors, we can look at the following coordinate diagram:
U (Cb) represents chroma component: it is the blue part of the photo without brightness (Y).
V (Cr) represents the chroma component: it is the red part of the photo without brightness (Y).
To explain YUV, we always need to keep in mind that we need to look at both bits and memory structures, so we need to explain these two things separately:
YUV sampling format: space – space: different space, that is, the number of bits to describe a pixel is different, such as YUV444, YUV422, YUV411, YUV420. That is, how to obtain Y, U and V components of each pixel when we collect pictures and video frames.
Storage format of YUV: space-inside: Same space, that is, the number of bits of a pixel is the same, but the storage mode is different. For example, for YUV420, it can be subdivided into YUV420P, YUV420SP, NV21, NV12, YV12, YU12, I420. That is, how the values of Y, U, and V are stored in memory or files.
2. Classification criteria
First, we can divide the YUV format into three formats according to the data size, YUV 420, YUV 422, YUV 444. Since the human eye is far more sensitive to Y than to U and V, it is sometimes possible to share a set of UV with multiple Y components, which can greatly save space and not lose too much quality. These three formats are formulated according to the characteristics of the human eye.
- YUV 420 consists of four Y components sharing one set of UV components
- YUV 422 consists of two Y components sharing a set of UV components
- YUV 444 does not share, one Y component uses one SET of UV components
According to the way that multiple Y components share a UV, we can divide YUV into 420,422,444 types. Under these three types, we can subdivide YUV into many formats according to the sequence of YUV’s arrangement and storage. These formats are numerous and difficult to remember. This caused a lot of difficulties in our learning process. I will introduce you one by one.
Firstly, according to the arrangement of YUV, YUV can be divided into three categories: Planar, semi-planar and Packed.
- Planar YUV three components are stored separately
- Semi-planar Y components are stored separately, and UV components are interlaced
- Packed YUV three components are all interlaced storage
Note: Interleaved is Packed, but in 422, Interleaved is more visible.
With these three approaches, we can classify the YUV format in a more detailed way.
We summarized a chart for specific classification:
Three, RGB
So what is RGB image format? In fact, he is not unfamiliar with it. He has been exposed to the three primary colors of color light since junior high school, which tells us that the light we can see can be mixed by these three colors in a certain proportion. Later, when you set the color of an element/control in HTML and Android development, you can use a string of numbers to get a specific color. This is where RGB comes in.
RGB color model is a kind of color standard, industry is based on red (R), green (G), blue (B) the change of the three color channels and their superposition between each other to get a variety of color, RGB is on behalf of the red, green, blue three channels of color, this standard includes almost all the colors of the senses by human vision, It is one of the most widely used color systems.
Its principle is: RGB is designed based on the principle of color luminescence. Generally speaking, its color mixing mode is just like red, green and blue lights. When their lights overlap with each other, the colors are mixed, and the brightness is equal to the sum of the brightness of the three lights.
In the superposition of red, green and blue lights, the brightest superposition area in the center of the three colors is white.
The red, green and blue channels each have 256 levels of brightness, with the “light” weakest – off – at 0 and the “light” brightest at 255. When the gray values of the three colors are the same, different gray values are produced, that is, when the gray values of the three colors are 0, it is the darkest black color. When the grayscale of all three colors is 255, it is the brightest white tone.
RGB colors are called additive colors because you produce white by adding R, G, and B together (that is, all light reflected back to the eye). Additive colors are used in lighting, television and computer monitors. For example, displays produce color by emitting light from red, green, and blue phosphors. The vast majority of the visible spectrum can be represented as a mixture of red, green and blue (RGB) light in different proportions and intensivities. When these colors overlap, they produce yellow, blue, and purple.
The following is a summary of common RGB formats:
Four,
Digital image has developed for nearly a hundred years, and different image formats have also emerged in the process of development. There are many, many kinds of image formats, so we have summarized some common image formats classification, which can be collected by partners in need and can be quickly referred to in the process of development.