While writing this player, I encountered some memory management issues that were tricky but gave me a better understanding of it, and much of the material didn’t follow the FFmpeg update, such as the use of AVBufferPool.
The ffMPEG version is 3.4
AVFrame and AVPacket memory management policies
The AVFrame:
av_frame_allocJust giveAVFrameIf you allocate memory, the buF inside it is still empty, which is the equivalent of building a box, but the box is empty.av_frame_refAdd a reference to SRC’s buF, using the same data, but with a reference count of +1.av_frame_unrefThe reference count of the data is -1.av_frame_freeUnref is still called internally, but the frame passed in is also null.
There is also a missing buffer initialization method inside the decoding functions AVCOdec_send_packet and AVCOdec_receive_frame.
Then there is a pit for decoding, for the avCODEC_receive_frame function:
Note that the function will always call
av_frame_unref(frame)before doing anything else.
If you use the same frame, each time you receive the decoded data, the previous data will be released each time, resulting in only one frame that is useful.
If you think frame’s alloc is expensive, you want to save resources, and then you don’t notice this comment, you probably do it.
There are two ways to do this:
- Continue to receive with only one frame, but when passed to the next step (render, play, etc.), the next module uses a new frame and
av_frame_refTo receive, rather than directly assign. - Build a new AVFrame each time before decoding and pass it to
avcodec_receive_frame, so that each time is a new frame, without interference. But at the end of the process, the frame is released.
Conveniently, the second option is better; However, from the perspective of modularity, the first one is better. In the single decoding step, we need to manage our own memory well, that is, alloc and unref of buffer are matched. In this way, memory management is perfect within the current module, and if anything goes wrong, it’s just another module. In contrast, the first method relies on the processing of other modules to free memory.
AVPacket is basically the same as AVFrame, except that the av_read_frame function that gets the packet does not perform unref, but sets buf to null. This problem can also be circumvented by using one of the two schemes above.
However, direct frame1=frame2 assignment is not desirable. Of course, it should be analyzed on a case-by-case basis, and always be aware that it uses reference counting to manage the data within the BUF.
No release at all
The AVFrame decoded is managed by a buffer. The frame in the buffer is temporarily stored but not released. I thought the buffer was retained, and then I added a release method to it. After that, each frame calls AV_packet_free, and then something strange happens.
It is clear that each frame is called free or unref, but the memory does not change. If the release is not clean, at least a little less. Is av_packet_free not working? I tried to cancel the free of the frame after it was played, but the memory skyrocketed as it played, indicating that it worked.
Then there is a maximum number of buffers, and if you increase the number, memory goes up, and if you decrease it, memory goes down. This makes sense, because all these frames are there, so they’re definitely taking up memory.
All frames in the buffer are not released after playback.
How to check? Look at the source code.
Av_frame_unref av_frame_unref av_frame_unref av_frame_unref av_frame_unref av_frame_unref
void av_buffer_unref(AVBufferRef **buf)
{
if(! buf || ! *buf)return;
buffer_replace(buf, NULL);
}
static void buffer_replace(AVBufferRef **dst, AVBufferRef **src)
{
AVBuffer *b;
b = (*dst)->buffer;
if (src) {
**dst = **src;
av_freep(src);
} else
av_freep(dst);
if(atomic_fetch_add_explicit(&b->refcount, -1, memory_order_acq_rel) == 1) { b->free(b->opaque, b->data); av_freep(&b); }}Copy the codeSo the key point is atomic_FETch_add_explicit, which has a series of functions that do atomic addition, subtraction, multiplication, and division. The function fetch then add, query then add, so the value returned is the value before the modification.
Atomic_fetch_add_explicit (&b->refcount, -1, memorY_ORDER_ACq_rel) == 1
Data and extend_data are referenced from data. Buf is the AVBufferRef type, indicating one reference to AVBuffer. If more references are added, the reference count of AVBuffer will be +1. One less is -1, no reference is released, AVBuffer is the real body of data. The memory management of AVFrame and AVPacket relies on av_xxx_ref and AV_xxx_unref functions.
B -> Opaque, B ->data; What function is called. Void av_buffer_DEFAULt_free (void *opaque, uint8_t *data); The default release function is called when AVBuffer is released. This function calls av_free, and av_free calls free, which simply frees memory.
If b – > free (b – > opaque, b – > data); If the default release function is called, the memory will drop. Synbolic breakpoints can be automatically located in the source code, and you can view the call stack data, which is all you can find. This allows you to see what b->free is at runtime, which is pool_release_buffer!!
static void pool_release_buffer(void *opaque, uint8_t *data) { BufferPoolEntry *buf = opaque; AVBufferPool *pool = buf->pool; .if (atomic_fetch_add_explicit(&pool->refcount, -1, memory_order_acq_rel) == 1)
buffer_pool_free(pool);
Copy the codeThere is no place to free data at all, again reference counting, and then buffer_pool_free.
/*
* This function gets called when the pool has been uninited and
* all the buffers returned to it.
*/
static void buffer_pool_free(AVBufferPool *pool)
{
while (pool->pool) {
BufferPoolEntry *buf = pool->pool;
pool->pool = buf->next;
buf->free(buf->opaque, buf->data);
av_freep(&buf);
}
ff_mutex_destroy(&pool->mutex);
if (pool->pool_free)
pool->pool_free(pool->opaque);
av_freep(&pool);
}
Copy the codeCombined with this function, the pool name, the two comments above, and my tests:
- A pool is a buffer pool that manages a number of avbuffers.
- Buffers generated from the pool, when released, are returned to the pool, and the pool reference count is -1. That is, this is a buffer pool that circulates, using reference counting to mark internal buffers.
- Pool build (
av_buffer_pool_init), the reference count is an initial value of 1, and is calledav_buffer_pool_uninitMarked as destructible, the reference count is reduced by 1, and the two match. - For each buffer generated internally, the reference count is +1, and for each buffer reclaimed, the reference count is -1. These two also match.
- Combined with the above two points, memory can be freed with proper operation. And not released, at least one of them.
- The purpose of the circular buffer pool is to avoid frequent and large memory allocation and release, especially for video frame data, which can be in the hundreds of K per frame. It also explains why memory is not freed at all, using pools, either all or nothing.
From inside to outside, check to see if any frames have not been released. And there was, in fact:
retval = avcodec_receive_frame(decoder->codecCtx, frame);
if(retval ! = 0) { TFCheckRetval("avcodec receive frame"); av_frame_free(&frame); // I missed thiscontinue;
}
Copy the codeAfter decoding fails, it simply continues. In my mind, it’s like the frame here is useless, has no data, so I just ignore it. Next. He died right here.
There’s still a problem with releasing all these frames, so that leaves av_buffer_pool_uninit. The outer layer used by the user in the call to this function is far away, and it is finally checked from avcodec_close. In logic is also reasonable, the end of the decoding, it is necessary to destroy the allocated memory. But instead of calling avCODEC_CLOSE directly, use avCODEC_free_context, freeing up everything else related to coDEC.
At this point, memory is finally freed. The key is to recognize the existence of a pool, which is not widely available online.