directory

  1. Ffplay breakpoint debugging
  2. (unsealed part) common structure and the relationship between the analysis
  3. data
  4. harvest

If you want to do good work, you must first strong device, breakpoint debugging, it is very important for us to comb through the process to troubleshoot problems, ffMPEG debugging can be in XCode, VS Code and QT ide for convenient debugging analysis. This article we take XCode as an example to introduce the ffplay breakpoint debugging, ffMPEG4.4 version to analyze.

A, FFplay breakpoint debugging

X and 4.x for Android. The difference is that this time, instead of cross-compiling, we will compile, install and debug ffmpeg on Mac.

./configure --enable-static --disable-shared --enable-debug --disable-doc --disable-x86asm --enable-nonfree  --enable-libvpx --enable-gpl  --enable-opengl --enable-libx264  --enable-libx265 --enable-libvmaf
make -j8
sudo make install
Copy the code

After a successful compilation, we will see several important executables ffmpeg_g, FFPRObe_g, and ffplay_g, which will be used for subsequent runs and debugging. How under the Xcode configuration debugging ffmpeg source please refer to: www.jianshu.com/p/27a90b113…

We analyze the structure used in the ffPlay unwrapping (read_thread) process at the interrupt point of the ffplay.c main function.

Open media stream

VideoState *stream_open(const char *filename,const AVInputFormat *iformat)
Copy the code

Involves a structure: AVInputFormat

Start readThread to start reading

    is->read_tid     = SDL_CreateThread(read_thread, "read_thread", is);
Copy the code

Allocate AVFormatContext memory

 AVFormatContext   ic = avformat_alloc_context();
Copy the code

Open the streaming media file

int avformat_open_input(AVFormatContext **ps, const char *filename,
                        const AVInputFormat *fmt, AVDictionary **options)
Copy the code

Structures are involved: AVFormatContext, AVInputFormat, AVDictionary

Get stream information

int avformat_find_stream_info(AVFormatContext *ic, AVDictionary **options)
Copy the code

Structures involved: AVStream AVCodecParameters AVRational

Loop through the frame data

for (;;) {... int av_read_frame(AVFormatContext *s, AVPacket *pkt) ... }Copy the code

Structures involved: AVFormatContext, AVPacket, etc

Unsealed flow to here, visible if you want to learn ffplay source code, first of all to make clear the main process and process involved in the key structure. In the next section, we’ll look at these structures in detail.

Iii. (Unsealed part) Common structures and their relationship analysis Common structures and their relationship analysis

3.1 Common structures and their relationships

There are many structures in FFMPEG. The most critical structures can be divided into the following categories: a) Protocol solution (HTTP, RTSP, RTMP, MMS) AVIOContext, URLProtocol, URLContext Stores the type and status of the protocols used by video and audio. URLProtocol The encapsulation format used to store input audio and video. Each protocol corresponds to a URLProtocol structure. (Note: FFMPEG files are also treated as a protocol "file") b) Unpacking (FLV, AVI, RMVB, MP4) AVInputFormat Encapsulates the format used to store input video and audio. Each visual audio package format corresponds to an AVInputFormat structure. C) Decoding (H264, MPEG2, AAC, MP3) Each AVStream stores one video/audio stream related data; Each AVStream corresponds to an AVCodecContext, which stores the data related to the video/audio stream using the decoding method; Each AVCodecContext corresponds to an AVCodec, which contains the corresponding video/audio decoder. Each decoder corresponds to an AVCodec structure. D) For storing data and video, each structure generally stores one frame; There may be several frames before decoding audio data: AVPacket decoded data: AVFrame quote from: https://blog.csdn.net/leixiaohua1020/article/details/11693997Copy the code

Their relationship is as follows:

Image from: Relationships between the most critical structures in FFMPEG

3.2. AVFormatContext This structure is defined in libavformat/ avformat.h. It is a data structure that is used throughout and is used by many functions. The role of several main variables is as follows:

Struct AVOutputFormat *oformat: struct AVOutputFormat *oformat: AVIOContext *pb: Unsigned int nb_streams: number of streams AVStream **streams: number of streams AVStream ** Char filename[1024] : filename int64_t duration: duration (unit: Microsecond us, converted to second, divided by 1000000) int bit_rate (BPS, converted to KBPS, divided by 1000Copy the code

This structure, also defined in libavformat/ avformat. h, is used to unpack the main variables of the object as follows

const char *name: Const char *mime_type: mime type such as video/avc video/hevc audio/aac int (*read_probe)(const AVProbeData *); int (*read_packet)(struct AVFormatContext *, AVPacket *pkt); int (*read_close)(struct AVFormatContext *); int (*read_seek)(struct AVFormatContext *, int stream_index, int64_t timestamp, int flags); int (*read_play)(struct AVFormatContext *); int (*read_pause)(struct AVFormatContext *); int (*read_seek2)(struct AVFormatContext *s, int stream_index, int64_t min_ts, int64_t ts, int64_t max_ts, int flags);Copy the code

3.4 AVStream Each AVStream stores data related to a video/audio stream; Is a stream object separated by the decapsulator, the product of the decapsulation, which is stored in the AVFormatcontext.

This structure is also defined in libavformat/ avformat.h, and the main variables are as follows:

int index; Stream index int id; Flow id void * priv_data; Stream data AVRational time_base; Time base, through which PTS, DTS can be converted to real time; PTS*time_base= actual time int64_t duration: stream length AVRational sample_aspect_ratio; AVRational AVG_frame_rate: frame rate AVCodecContext * COdec: AVCodecContext to point to the video/audio stream (they are one-to-one correspondence)Copy the code

AVStream is the output of the decapsulation link and the input of the decoding link. Each AVStream corresponds to an AVCodecContext, which stores the relevant data of the video/audio stream using the decoding method. Each AVCodecContext corresponds to an AVCodec, which contains the corresponding video/audio decoder. Each decoder corresponds to an AVCodec structure. Decoding part of the data structure analysis we will analyze the study in the next chapter.

3.5 AVPacket A structure that stores information related to compressed encoded data, and stores the data before decoding after encapsulation, as well as the information of PTS, DTS, Duration and streamId, etc. This structure is defined in libavcodec/ packet-h, and the main variables are as follows:

uint8_t *data; For H.264. Data of one AVPacket usually corresponds to one NAL. Int size: data size int64_t PTS: display timestamp int64_t DTS: decode timestamp AVPacketSideData * SIDE_data; Additional informationCopy the code

Third, information

AVFormatContext ffMPEG structure analysis: AVStream FFMPEG structure analysis: AVFormatContext FFMPEG structure analysis: AVStream FFMPEG structure analysis: AVFormatContext FFMPEG structure analysis: AVStream FFMPEG structure analysis: AVPacket

Four, harvest

Through this learning practice, we have learned

  1. How to debug FFMPEG breakpoint and analyze ffplay unpack process under Xcode
  2. Understand the relationship between common structures: decoding protocol, decoding, decoding corresponding structures and the relationship between them
  3. Understand the main variables and functions of several key structures associated with decapsulation. AVFormatContext, AVInputFormat, AVStream, AVPacket

Thank you for reading the next article we analyze the FFMPEG decoding part of the common structure, welcome to pay attention to the public number “audio and video development journey”, learn to grow together. Welcome to communicate