Ffmpeg is a powerful multimedia processing tool, the Swiss Army knife of multimedia processing, covering a large number of multimedia processing tools. However, FFMPEG is written in pure C language, which is difficult for Python users to use. Therefore, today I recommend a development package that uses FFMPEG in Python: PyAV

PyAV provides the Python interface to FFMPEG, but in reality it only uses FFMPEG as the back end, and uses Cython to encapsulate ffMPEG’s interface, so ffMPEG is still called.

PyAV installation

PyAV is cross-platform and can be installed according to your environment and platform.

Install PyAV on Windows:

Under the Windows installation PyAV can consult blogs blog.csdn.net/Dillon2015/…

Install PyAV on Mac OS X and Ubuntu:

To install PyAV on Mac OS X and Ubuntu, refer to the official installation instructions

PyAV use

PyAV provides a very convenient interface so that developers don’t need to pay too much attention to the low-level details.

The video is split into separate frames

Sometimes when you need to break a video into frames, you only need one command on the FFmpeg command line:

Ffmpeg -i test.avi -r 1 -f image2 image-%3d.jpeg

-r indicates the number of images extracted per second. If it is equal to the frame rate, all frames are extracted.

It’s easy to do the same thing in PyAV,

import av
​
container = av.open(path_to_video)
#path_to_video is the path to your video
for frame in container.decode(video=0):
    frame.to_image().save('frame-%04d.jpg' % frame.index)
Copy the code

Save keyframe

For a video sequence, not all frames are the same, because the video encoding is cross-referenced in inter-frame prediction. If the reference frame of a frame is lost or damaged, the frame cannot be decoded correctly, so it is more important for those frames that are used for reference.

In av. Video. Frame. VideoFrame class has an attribute key_frame to whether the frame is the key frames.

import av
import av.datasets
​
container = av.open(path_to_video)
# Signal that we only want to look at keyframes.
stream = container.streams.video[0]
stream.codec_context.skip_frame = 'NONKEY'
​
for frame in container.decode(stream):
    # We use `frame.pts` as `frame.index` won't make must sense with the `skip_frame`.
    frame.to_image().save(
        'night-sky.{:04d}.jpg'.format(frame.pts),
        quality=80.)Copy the code

In the above code, non-key frames are skipped and all key frames are saved.

Video encapsulation

Video to encapsulation is to change the encapsulation format of video without changing the encoding mode of video stream, audio stream, etc., such as from MP4 -> MKV

The process is as follows:

 

import av
import av.datasets
​
input_ = av.open(path_to_video)
output = av.open('remuxed.mkv'.'w')
​
# Make an output stream using the input as a template. This copies the stream
# setup from one to the other.
in_stream = input_.streams.video[0]
out_stream = output.add_stream(template=in_stream)
​
for packet in input_.demux(in_stream):
    # We need to skip the "flushing" packets that `demux` generates.
    if packet.dts is None:
        continue
​
    # We need to assign the packet to the new stream.
    packet.stream = out_stream
    output.mux(packet)
​
output.close()
Copy the code

Generate video

PyAV can also be used in conjunction with Numpy to convert NDARray directly into video frames, making frame manipulation more flexible and convenient.

from __future__ import division
​
import numpy as np
​
import av
​
duration = 4
fps = 24
total_frames = duration * fps
container = av.open('test.mp4', mode='w')
stream = container.add_stream('mpeg4', rate=fps)
stream.width = 480
stream.height = 320
stream.pix_fmt = 'yuv420p'
​
for frame_i in range(total_frames):
    img = np.empty((480.320.3))
    img[:, :, 0] = 0.5 + 0.5 * np.sin(2 * np.pi * (0 / 3 + frame_i / total_frames))
    img[:, :, 1] = 0.5 + 0.5 * np.sin(2 * np.pi * (1 / 3 + frame_i / total_frames))
    img[:, :, 2] = 0.5 + 0.5 * np.sin(2 * np.pi * (2 / 3 + frame_i / total_frames))
​
    img = np.round(255 * img).astype(np.uint8)
    img = np.clip(img, 0.255)
​
    frame = av.VideoFrame.from_ndarray(img, format='rgb24')
    for packet in stream.encode(frame):
        container.mux(packet)
​
#Flush stream
for packet in stream.encode():
    container.mux(packet)
​
#Close the file
container.close()
Copy the code

The above code generates a 480×320 video at 24fps.

summary

PyAV also has more powerful features, interested partners can install their own try oh.

If you are interested, please pay attention to wechat public account Video Coding