FLV protocol 5-minute introduction

FLV protocol Overview

Flash Video (FLV) is a kind of streaming media format. Due to its small size and relatively simple protocol, it quickly becomes popular and is widely supported.

The common HTTP-FLV live broadcast protocol uses HTTP to stream transmission of FLV encapsulated audio and video data. For those of you who want to understand HTTP-FLV, it is essential to understand the FLV protocol.

In a nutshell, FLV consists of FLV header and FLV file body, and FLV file body consists of multiple FLV tags.

FLV = FLV header + FLV file body FLV file body = PreviousTagSize0 + Tag1 + PreviousTagSize1 + Tag2 + … + PreviousTagSizeN-1 + TagN

FLV tags are divided into three types:

• Video Tag: Store video-related data;
• Audio Tag: Stores Audio related data;
• Script Tag: stores audio and video metadata.

Before the actual explanation of FLV protocol, the unit is agreed:

FLV header

FLV Header consists of the following fields:

1. The first three bytes are always FLV
2. The last 4 bytes of content are fixed at 9 (for FLV version 1)

FLV file body

FLV file body is regular and consists of a series of tagsizes and tags:

1. PreviousTagSize0 is always 0;
2. A tag consists of a tag header and a tag body.
3. For FLV version 1, the tag header is fixed to 11 bytes, so PreviousTagSize (except for the first) is 11 + the size of the previous tag body;

FLV tags

The FLV tag consists of tag header and tag body.

The tag header is 11 bytes:

In playback, the time sequencing of FLV tags depends on the FLV timestamps only. Any timing mechanisms built into the payload data format are ignored.

Audio tags

The definition is as follows:

Remark:

If the SoundFormat indicates AAC, the SoundType should be set to 1 (stereo) and the SoundRate should be set to 3 (44 kHz). However, this does not mean that AAC audio in FLV is always stereo, 44 kHz data. Instead, the Flash Player ignores these values and extracts the channel and sample rate data is encoded in the AAC bitstream.

AACAUDIODATA

When SoundFormat is 10, it indicates that audio is encoded by AAC. At this time, the definition of SoundData is as follows:

The AudioSpecificConfig is explained in ISO 14496-3. Note that it is not the same as the contents of the esds box from an MP4/F4V file. This structure is more deeply embedded.

About AudioSpecificConfig

The pseudocode is as follows: see here

5 bits: object type
if (object type == 31)
    6 bits + 32: object type
4 bits: frequency index
if (frequency index == 15)
    24 bits: frequency
4 bits: channel configuration
var bits: AOT Specific Config
Copy the code

The definition is as follows:

Video tags

The definition is as follows:

AVCVIDEOPACKE

When CodecID is 7, VideoData is AVCVIDEOPACKE, also known as H.264 media data.

AVCVIDEOPACKE is defined as follows:

Here are a few things to explain:

1. NALU: In H.264, an abstract logical unit (NALU) is obtained after data is formatted according to specific rules. The data here includes not only the encoded video data, but also the parameter set (PPS, SPS) needed for video decoding.
2. AVCDecoderConfigurationRecord: h. 264 video decoding the required parameter set (SPS, PPS)
3. CTS: When B frames exist, DTS and PTS may be different in the process of video decoding and presentation. The calculation formula of CTS is PTS-DTS /90, in milliseconds. If B frame does not exist, CTS is fixed to 0.

PPS and SPS are not expanded here.

Script Data Tags

Script Data Tags are usually used to store onMetaData related to audio and video in FLV, such as length, length, width, etc. Its definition is relatively complex, using AMF (Action Message Format) encapsulates a series of data types, such as strings, values, arrays and so on.

A SCRIPTDATAOBJECT is defined as follows:

The definition of SCRIPTDATAVALUE is as follows:

As you can see, the definition of Script Data Tag is relatively complex.

onMetaData

OnMetaData contains audio and video related metadata, encapsulated in Script Data tags, which contain two AMFs.

The first AMF:

• The first byte, 0x02, is a string
• Bytes 2-3: UI16 type 0x000A, indicating string length of 10 (length of onMetaData);
• Byte 4-13: Hexadecimal number corresponding to the string onMetaData (0x6F 0x6E 0x4D 0x65 0x74 0x61 0x44 0x61 0x74 0x61);

Second AMF:

• The first byte, 0x08, represents the array type;
• Bytes 2-5: UI32, indicating the length of the array. The properties of onMetaData are not fixed.
• Byte 6 + : for example, duration, then:
  • Byte 6-9:0x0008, indicating the length of 8 bytes;
  • Bytes 10-17 0x6475 7261 7469, duration;
  • The 18th byte, 0x00, represents a numeric type;
  • Bytes 19-26:0x… Is the specific duration;

More onMetaData definitions:

Write in the back

FLV protocol itself is not complicated, the difficulty in understanding, more often from audio and video codec related knowledge, such as H.264, AAC related knowledge, it is recommended to look up when you do not understand. In addition, the byte order of FLV is big-endian, which must be paid attention to when parsing protocols.

This article for the convenience of explanation, part of the content may not be rigorous, if there are mistakes, please point out.

A link to the

Video_file_format_spec_v10. PDF www.adobe.com/content/dam…

Mpeg-4 Part 3 en.wikipedia.org/wiki/MPEG-4…

FLV file analysis www.jianshu.com/p/e290dca02…

H.264 NALU syntax blog.csdn.net/qq_29350001…