directory

  1. Coding process and data format review
  2. 4 x 4 brightness block of 9 in the prediction mode
  3. Four prediction models for 16 × 16 brightness blocks
  4. Four prediction models for 8 × 8 chromaticity blocks
  5. JM code
  6. data
  7. harvest

Review of coding process and data format

Let’s take a look at the figure below to review the coding process

An encoded image is usually divided into one or more slices, and each slice can be divided into several macro blocks. A macro block is composed of a 16×16 brightness pixel, an 8×8 Cb and an 8×8 Cr color pixel block, namely the common 420 sampling format, as shown below:

We look at the first frame of H264 data through the code stream analysis tool H264 VISA. It can be seen that it is an I frame, composed of one macro block after another, and encoded in the frame by the way of intra-frame prediction.

The type of the macroblock is 16×16 and 4×4. Find out more by Elecard StreamEye. Include pmode, iPREd Intra_4x4, ipred chroma all represent what meaning and what value inside have what meaning?

In the intra-frame prediction mode, the prediction block P is formed based on the encoded reconstruction block and the current block. For luminance pixels, the P block is used for operations related to 4×4 sub-blocks or 16×16 macro blocks. There are 9 optional prediction modes for 4×4 brightness subblock, independently predicting each 4×4 brightness subblock, which is suitable for image coding with a lot of details; There are four prediction modes for 16×16 brightness block, which are suitable for flat region image coding. Chroma block also has 4 prediction modes, similar to 16×16 brightness block prediction mode. Encoders usually choose the prediction mode that minimizes the difference between p-block and code block.

Let’s look at the prediction model

2. Nine prediction modes of 4 × 4 brightness block

The pixels of the 4×4 prediction block are marked with lowercase letters A-P, and the reference pixels on the left and above the prediction block are marked with uppercase letters A-M:

The 4×4 brightness block has 9 prediction modes, respectively as follows

The predicted values of all samples in prediction mode 2 are equal to the average values of A-D and I-L. The prediction diagram of 8 song directions of the other 8 modes is as follows:

The nine prediction modes are described as follows: Mode Description Mode0 (Vertical): Calculate the value of A, B, C, and D vertically

Mode1 (Horizontal) : The Horizontal is calculated by I, J, K and L on the left

Mode2 (DC) : The predicted values of all samples in P are equal to the average of AD and IL

Mode3 (Diagonal down-left): Derived from the interpolation of the lower Left and upper right samples in the 45-degree direction

Mode4 (Diagonal down-right): And Diagonal down-right at a 45 degree Angle

Mode5 (vertical-right) : push the calculation in the direction of approximately 26.6 degrees to the Right of the Vertical downward direction (width/height = 1/2).

Mode6 (horizontal-down): it is estimated at an Angle of approximately 26.6 degrees to the right of the Horizontal direction.

Mode7 (vertical-left): computes approximately 26.6 degrees to the Left of the Vertical downward direction.

Mode8 (horizontal-up): calculated at an Angle of approximately 26.6 degrees to the right of the Horizontal.

Note that:

The Mode 2 rule can be modified based on the availability of the sample in a-M, because it takes an average of the reference sample, while other modes may be used when all required reference samples are available. Then note that if samples E, F, G, and H are not available, the values of sample D can be copied to these locations to mark the samples as available. For Mode 3-8, the values of the predicted samples are the weighted average of the reference samples A-M. Assuming Mode 4 is selected, the predicted value of d = round(B/4 + C/2 + d /4). Quote from: www.jianshu.com/p/4d8692d53…

Image from: H.264_AVC(2nd edition). Bi Houjie, Wang Jian

Three, four prediction modes of 16 × 16 brightness block

Mode0 (vertical): indicates the value of Mode1 (horizontal) based on the upper sample (H). Mode2 (DC) indicates the average value of Mode3 (Plane) based on the upper sample (H) and the left sample (V). Based on the top sample (H) and the left sample (V) obtained by a plane function, it works well in the area where the brightness changes smoothly.

Here is an example from the new Generation video Compression Code Standard h.264_AVC (second edition), compiled by Bi Houjie and Wang Jian:

Four prediction models of 8 × 8 chromaticity block

The choice of prediction mode for Cr and Cb components of chroma is the same. The description of the chroma prediction model is similar to that of the 16×16 brightness prediction model, except that the model numbers are different. DC (mode 0), horizontal (mode 1), vertical (mode 2), plane (mode 3).

Here I think I have my own answers to some of the questions we looked at at the beginning of this article with the code stream analysis tool H264visa and Elecard StreamEye.

JM code

In this section, we combine JM source code to simply analyze the realization of JM including JM Encoder and JM decoder, JM Encoder is a specific implementation of H.264 standard, JM decoder is the corresponding decoder. JM Encoder, like X264, is the implementation of H.264 standard, but it is implemented strictly in accordance with the protocol of H264. X264 simplifies the calculation method of JM cost, removes some codes, and optimizations such as estimation. JM encoding speed is very slow, and is not suitable for production environment. View its source code for understanding the H264 protocol is very helpful. JM source download address: iphome. Hhi. DE/suehring/tm… , you can choose a version to download and view

This source code has not really look, first make a record, according to the need to learn. Specific can refer to thor article: Thor -H.264 official software JM source code simple analysis – encoder LenCOD

Six, data

  1. Book: The new Generation of video compression code standard -H.264_AVC(2nd edition). Bi Houjie, Wang Jian (Ed.) – Chapter 6 principle of H.264/AVC encoder
  2. H.264 in-frame prediction
  3. Raytheon -H.264 official software JM source code simple analysis – encoder Lencod
  4. JM source code download
  5. [JM official document]

Seven, harvest

Through this study practice

  1. Understand the way of intra – frame coding and the role in the whole coding process
  2. Understand the 9 prediction modes of 4×4 brightness block, 16×16 brightness block and 8×8 chroma block
  3. Use the code stream analysis tool to view macro block information for better understanding.

Thank you for reading the next article we learn between h.264 frame prediction, welcome to pay attention to the public account “audio and video development journey”, learn and grow together. Welcome to communicate