This is the 20th day of my participation in the December Gwen Challenge. Check out the event details: The last Gwen Challenge 2021

Pure front-end video streaming is generally transmitted through Websocket to raw H.264 video streaming. If RTC type stream is played, video label is generally used. If RTMP type stream, canvas is generally used to draw frame by frame. RTC streams are preferred for transmission.

The previous article to everyone said h.264 SPS, PPS, I frame, P frame, B frame, we have a general understanding of H.264, today we will look at how JS decoding RTMP type naked H.264 video stream.

In H.264, there is usually a delimiter in front of the frame: 00 00 00 01 or 00 00 01. The first frame is usually SPS, PPS, followed by I frame.

00 00 00 01 The bytes below the delimiter are of type NALU, as shown in the following figure:

The NALU type needs to be converted to binary. After converting data, the h.264 raw stream reading rules are as follows:

  1. F: forbidden_zero_bit, is the first bit, is the forbidden bit. If there are syntax conflicts, it is 1. It can be actively set to 1 so that it can be dropped. \
  2. NRI: Nal_ref_IDC, rank 2-3, as the reference level; Indicates the importance level of the NALU. The larger the value is, the more important the current NALU is. \
  3. Type: bits 4-8 indicating the Type of NALU.

The NALU type can be used to judge SPS, PPS and I frames, as shown in the figure:

As can be seen from the picture, SPS, PPS and I frame can be determined by judging nal_unnit_type. I frame is 5, SPS is 7 and PPS is 8.

00 00 00 01 or 00 00 01 after 0x65, 0x67, 0x68, the frame type can be determined by converting the 4-8 bits of the binary code into decimal code. The specific determination method is as follows:

The binary code of 0x65 is 0110 0101. The binary code of 0x65 is 00101. The binary code of 0x65 is 00101.

The binary code of 0x67 is 0110 0111. The 4-8 bits are 00111, which is 7 in decimal notation, corresponding to SPS.

The binary code of 0x68 is 0110 1000. 4-8 is 01000 in decimal 8, which corresponds to PPS.

Therefore, the method to judge SPS, PPS and I frames is as follows (pseudocode) :

(NALU type &&0001 1111) = 5/7/8;Copy the code

Through the above method, we should also have a certain understanding of the decoding.