This is the 8th day of my participation in the First Challenge 2022. For details: First Challenge 2022.

Preface:

In “Getting to know iOS CVPixelBuffer (part 1)”, we learned how to create, modify, and check cvelBuffer parameters. In this article, we will learn about the difference between RGB and YUV in color space, and then go back to cvpixelFormatType in CVPixelBuffer.

I believe you are not unfamiliar with RGB, so YUV you know, what does it do for us?

I start with a question: Why use YUV? After this problem understood color space in us, believe everybody already had the answer in the heart. Then we begin our text.

Color space

The word “Color Space” comes from the English Color Space. In chromology, people have established a variety of Color models, representing a certain Color with one-dimensional, two-dimensional, three-dimensional or even four-dimensional spatial coordinates. The Color range defined by coordinate system is Color Space.

The color Spaces we often use mainly include RGB, YUV, CMYK, HSB and HSL, among which RGB and YUV are the two most commonly used color Spaces in videos.

RGB

RGB is composed of three colors, Red, Green and Blue, which we call the three primary colors. Each color can take on a different value, and when the three colors are mixed together, they form different colors. In the image represented by RGB, each pixel has three colors, red, green and blue, and each color occupies 8 bits, that is, 1 byte. So one pixel takes up 3 bytes.

The red, green, and blue color channels have 256 levels of brightness, ranging from 0 to 255.

When storing, we usually arrange them in order of R, G, B, or B, G, R, so in the previous paragraph, PixelFormatType has two similar types: kCVPixelFormatType_32RGBA and kCVPixelFormatType_32BGRA. These two types are just coded in different order;

Now let’s get familiar with another color space.

YUV

YUV is a general name of Y ‘UV, YUV, YCbCr, YPbPr and other color Spaces

YUV is composed of Y and UV, with the former representing Luminance and Luma and the latter representing Chrominance and Chroma. Chrominance includes two aspects: hue and saturation.

With YUV images, each pixel contains Y, U and V components. The Y and UV components are separated. If only Y has no UV, then the image is just like the old black and white TV sets. In order to be compatible with older TV sets, color TV sets also use YUV color space, but UV exists.

Sampling format

YUV sampling format is a key, there are three types of YUV sampling format:

  • YUV 4:4:4
  • YUV 4:2:2
  • YUV 4:2:0

YUV444

YUV444 represents that each Y component corresponds to a U component and a V component. Take four pixels as an example: [Y0 U0 V0] [Y1 U1 V1] [Y2 U2 V2] [Y3 U3 V3] Stores the code stream as: Y0 U0 V0 Y1 U1 V1 Y2 U2 V2 Y3 U3 V3

YUV422

YUV422 represents that each Y component shares one U component and one V component. Take 4 pixels as an example: [Y0 U0 V0] [Y1 U1 V1] [Y2 U2 V2] [Y3 U3 V3] The stored code stream is: Y0 U0 Y1 V1 Y2 U2 Y3 V3 The mapped pixels are: [Y0 U0 V1] [Y1 U0 V1] [Y2 U2 V3] [Y3 U2 V3]

YUV420

YUV420 represents each of the four Y components sharing one U component and one V component. Take 8 pixels as an example: [Y0 U0 V0] [Y1 U1 V1] [Y2 U2 V2] [Y3 U3 V3] [Y5 U5 V5] [Y6 U6 V6] [Y7 U7 V7] [Y8 U8 V8] Store the code stream as follows: The mapped pixels of Y0 U0 Y1 Y2 U2 Y3 Y5 V5 Y6 Y7 V7 Y8 are: [Y0 U0 V5] [Y1 U0 V5] [Y2 U2 V7] [Y3 U2 V7] [Y5 U0 V5] [Y6 U0 V5] [Y7 U2 V7] [Y8 U2 V7]

In summary, the amount of data stored by YUV444 is the same as RGB, YUV422 is 2/3 of RGB and YUV420 is 1/2 of RGB.

storage

YUV is stored in two ways: packed and planar.

  • packed: Store all Y components continuously, and then cross store U and V components;
  • planar: Store all Y components consecutively, then store U and V components consecutively (Each block is guaranteed to be continuous);

Once we understand how these color Spaces are encoded and stored, we can look at the above definitions of these types and deduce what these types represent based on these theories.

Such as: 444, 422 and 420 represent different sampling formats (compression ratio). 8, 10 and 16 represent the bits Planar, BiPlanar, BiPlanar, VideoRange and FullRange. Their significance is the value range of Y in the luminance part of YUV color space. The value range of fullRange is luma=[0,255] chroma=[1,255] while the value range of videoRange is luma=[16,235] chroma=[16,240].

kCVPixelFormatType_422YpCbCr8 = '2vuy', /* Component Y'CbCr 8-bit 4:2:2, ordered Cb Y'0 Cr Y'1 */ kCVPixelFormatType_4444YpCbCrA8 = 'v408', /* Component Y'CbCrA 8-bit 4:4:4:4, ordered Cb Y' Cr A */ kCVPixelFormatType_4444YpCbCrA8R = 'r408', /* Component Y'CbCrA 8-bit 4:4:4:4, rendering format. full range alpha, zero biased YUV, ordered A Y' Cb Cr */ kCVPixelFormatType_4444AYpCbCr8 = 'y408', /* Component Y'CbCrA 8-bit 4:4:4:4, ordered A Y' Cb Cr, full range alpha, video range Y'CbCr. */ kCVPixelFormatType_4444AYpCbCr16 = 'y416', /* Component Y'CbCrA 16-bit 4:4:4:4, ordered A Y' Cb Cr, full range alpha, video range Y'CbCr, 16-bit little-endian samples. */ kCVPixelFormatType_444YpCbCr8 = 'v308', /* Component Y'CbCr 8-bit 4:4:4 */ kCVPixelFormatType_422YpCbCr16 = 'v216', /* Component Y'CbCr 10,12,14,16-bit 4:2:2 */ kCVPixelFormatType_422YpCbCr10 = 'v210', /* Component Y'CbCr 10-bit 4:2:2 */ kCVPixelFormatType_444YpCbCr10 = 'v410', /* Component Y'CbCr 10-bit 4:4:4 */ kCVPixelFormatType_420YpCbCr8Planar = 'y420', /* Planar Component Y'CbCr 8-bit 4:2:0. baseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrPlanar struct */ kCVPixelFormatType_420YpCbCr8PlanarFullRange = 'f420', /* Planar Component Y'CbCr 8-bit 4:2:0, full range. baseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrPlanar struct */ kCVPixelFormatType_422YpCbCr_4A_8BiPlanar = 'a2vy', /* First plane: Video-range Component Y'CbCr 8-bit 4:2:2, ordered Cb Y'0 Cr Y'1; second plane: alpha 8-bit 0-255 */ kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange = '420v', /* Bi-Planar Component Y'CbCr 8-bit 4:2:0, Video-range (LumA =[16 235] Chroma =[16 240]). BaseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct */ kCVPixelFormatType_420YpCbCr8BiPlanarFullRange = '420f', /* Bi-Planar Component Y'CbCr 8-bit 4:2:0, Full-range (luma=[0,255] chroma=[1,255]). BaseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct */ kCVPixelFormatType_422YpCbCr8BiPlanarVideoRange = '422v', /* Bi-Planar Component Y'CbCr 8-bit 4:2:2, Video-range (LumA =[16 235] Chroma =[16 240]). BaseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct */ kCVPixelFormatType_422YpCbCr8BiPlanarFullRange = '422f', /* Bi-Planar Component Y'CbCr 8-bit 4:2:2, Full-range (luma=[0,255] chroma=[1,255]). BaseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct */ kCVPixelFormatType_444YpCbCr8BiPlanarVideoRange = '444v', /* Bi-Planar Component Y'CbCr 8-bit 4:4:4, Video-range (LumA =[16 235] Chroma =[16 240]). BaseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct */ kCVPixelFormatType_444YpCbCr8BiPlanarFullRange = '444f', /* Bi-Planar Component Y'CbCr 8-bit 4:4:4, Full-range (luma=[0,255] chroma=[1,255]). BaseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct */ kCVPixelFormatType_422YpCbCr8_yuvs = 'yuvs', /* Component Y'CbCr 8-bit 4:2:2, ordered Y'0 Cb Y'1 Cr */ kCVPixelFormatType_422YpCbCr8FullRange = 'yuvf', /* Component Y'CbCr 8-bit 4:2:2, full range, ordered Y'0 Cb Y'1 Cr */Copy the code

Write in the last

Now, why do we use YUV? I think everyone already has the answer in mind. YUV’s advantages are basically clear at a glance, because the compression rate is much higher than RGB and the data storage is relatively small, so it is suitable for a series of operations such as video data transmission or image processing in memory.

Next chapter: iOS CVelBuffer (Part 2)