In addition to FFmpeg has powerful encapsulation/decapsulation, encoding/decoding functions, but also contains a very powerful component – filter AVfilter. Avfilter components are often used for multimedia processing and editing, FFmpeg contains a variety of filters

1. Parameter arrangement mode of FFmpeg filter

To make it easier to understand Filter, the following is the simplest way to describe the arrangement of parameters when Filter is used: [input stream or tag name] Filter parameter [temporary tag name]; [Input stream or tag name] Filter parameter [temporary tag name]... Here is a simple example: Enter two files, a video input.mp4 and an image logo.png, zoom the logo and place it in the top left corner of the videoCopy the code
ffmpeg -i test.mp4 -i logo.png -filter_complex "[1:v]scale=176:144[logo]; [0:v][logo]overlay=x=0:y=0"Filter.mp4 scaled the logo.png image six to 176*144 resolution, then defined a temporary tag name logo, and finally spread the scaled image [LOGO] in the upper left corner of the input video stream [0:v] of test.MP4Copy the code

2.FFmpeg Filter Time built-in variable

In the use of Filter, the need to operate according to the timeline is often used. In the use of FFmpeg Filter, time-dependent built-in variables of Filter can be used. The following table shows these related variables, which can be used in the following

variable instructions
t The timestamp is expressed in seconds, or NAN if the timestamp is entered in a position
n Enter the sequence number of frames, starting from 0
pos Enter the position of the frame, or NAN if so
w Enter the width of the video frame
h Enter the height of the video frame

3.FFmpeg adds watermark to the video

FFmpeg can add watermark to video, which can be used for text or pictures, and is mainly used to mark video ownership

3.1 Text Watermarking

There are many conditions to prepare for adding text watermarking in videos, such as relevant files for processing text libraries, support FreeType, FontConfig and Iconv when compiling FFmpeg. The system needs to have the relevant font library, add pure letter watermark in FFmpeg can be supported by the drawtext filter, the following is the drawtext filter parametersCopy the code
parameter type instructions
fontfile string The font file
text string The text
textfile string Text file
fontcolor string The font color
box string Text area background box
boxcolor string Display font colors in the area of fast colors
fontsize string Displays the font size
font string Font name (default Sans font)
x string The x coordinate of the text display
y string The y coordinate of the text display
Using an example
ffmpeg -i out2.mp4 -vf "drawtext=fontsize=100:fontfile=FreeSerif.ttf:text='hello world ':x=20:y=20"Waterfont. Mp4 After execution, you can add "Hello World" text watermark in the upper left corner of the video. The color of the text is pure black. For example, make it green ffmPEG-I out2.mp4-VF"drawtext=fontsize=100:fontfile=FreeSerif.ttf:text='hello world ':x=20:y=20:fontcolor=green"Waterfont. Mp4 can also call some system data, such as watermark based on the current time ffmPEG-re -I out 2.MP4-VF"drawtext=fontsize=100:fontfile=FreeSerif.ttf:text='%{localtime\:%Y\-%m\-%d %H-%M-%S}':x=20:y=20:fontcolor=green" waterfont4.mp4
Copy the code

3.2 Image Watermarking

In addition to adding text watermarks to videos, FFmpeg can also add image watermarks to videos, video running lights, etc. Movie filters can be used to add watermarks to videos. The following are some parameters of movie filters

parameter type instructions
filename string The file name can be a file, protocol, or device
format_name_,f string The encapsulation format of input
stream_index,f The integer The stream index number entered
seek_point,sp Floating point Numbers Seek Specifies the time and position of the input stream
stream,s string Flow information for multiple streams entered
loop The integer cycles
discontinuity Time difference value Support for pulsating timestamp differentials

For example, there are two ways to add image watermarks in FFmpeg. One is to specify the file path of watermark through movie, and the other is to read the stream of input file through filter and specify it as watermark. Here we focus on how to read movie image files as watermarks

ffmpeg -i out2.mp4 -vf "movie=water_pic.png[wm]; [in][wm]overlay=30:10[out]"Waterpic.mp4 the image will appear in the upper left corner of the video. Movie can also be used with colorkey filters to create a translucent effect, such as ffmPEG-I out 2.MP4-VF"Movie = water_pic. PNG, colorkey = black: 1.0:1.0 [wm]; [in][wm]overlay=30:10[out]"Waterpicalpha.mp4 At this point the image is displayed in the upper left corner with 30% opacity. [WM] is used to name the frontCopy the code

3.3 Overlay filter usage

Overlay, also called video overlay technology, is widely used. Common examples include the above image watermarking and the picture-in-picture function mentioned below. The value of the picture-in-picture function is that there is a small playback window in a large video playback window, and different video content is played in the two Windows at the same time. An overlay technique uses two Windows. Usually, the larger window is called the background window and the smaller window is called the foreground window. Both can play videos or display pictures.

Overlay filters are described below
describe instructions
grammar Overlay [=x:y[[: RGB ={0, 1}]] Parameters x and y are optional and default to 0. RGB parameters are optional and have a value of 0 or 1
x Horizontal coordinates starting from the upper left corner. Default is 0
y Vertical coordinates from the top left corner, default to 0
rgb RGB = 0… Input color space does not change, default value RGB = 1… The input color space is set to RGB
Main_w or W Main input (background window) width
Main_h or H Main input (background window) height
Overlay_w or w Overlay Input (foreground window) width
Overlay_h or h Overlay input (foreground window) height
eof_action How to handle eOF flags when encountered. Default is repeat

Repeat (value 0) : repeats the previous frame

Endall (value 1) : Stops all frames

Pass (value 2) : Keep the main layer
format Sets the pixel format for output. Defaults to yuv420

Yuv420 (a value of 0)

Yuv422 (a value of 1)

Yuv444 (value of 2)

RGB (a value of 3)
shortest Boolean value, all terminates when terminating shortest video (default off)
Overlay filter usage

Using overlay command line basic format is as follows:

Ffmpeg -I input1 -i input2 -filter_complex overlay=x:y output Using a simple example, overlay the image in the top right corner of the video, Ffmpeg -I test.mp4 -i chenyao.png -filter_complex overlay= w-w: 56-max_muxing_queue_size 1024 overlay. Mp4Copy the code

3.4 FFmpeg generates picture-in-picture

In addition to the above overlay adding ICONS to the video, sometimes you need to use the picture-in-picture effect when working with streaming media files using FFmepg. In FFmpeg, multiple video streams, multiple multimedia acquisition devices, and multiple video files can be combined into one interface using overlays to create a picture-in-picture effect. Most of the processing related to video manipulation will be used in conjunction with overlay filters, especially for layer manipulation and merging scenes, in the previous and later filters

As can be seen from the parameter list of 3.3 above, there are not many main parameters. In fact, in the use of overlay filter, there are many combinations of parameters. You can use some internal variables, such as the width, height, and coordinates of the overlay layer

ffmpeg -re -i out2.mp4 -vf "movie=output.avi,scale = 480*320[test]; [in][test] overlay [out]" -vcodec libx264 videoInvideo.mp4
Copy the code

The figure above is the most basic way to display a picture within a picture. If you want the child video to be displayed in a specified location, such as the bottom right corner of the picture, you need to use the internal variables of the X and Y coordinates in the overlay, just as the above image is used to overlay the video

ffmpeg -re -i out2.mp4 -vf "movie=output.avi,scale = 480*320[test]; [in][test] overlay=x=main_w-500:main_h-500 [out]" -vcodec libx264 videoInvideo.mp4
Copy the code

The above two kinds of picture-in-picture processing are static position processing. Overlay can also be used for picture-in-picture processing along with regular expressions

ffmpeg -re -i out2.mp4 -vf "movie=output.avi,scale = 480*320[test]; [in][test] overlay=x='if(gte(t,2),-w+(t-2)*20,NAN)':y=0 [out]"-vcodec libx264 videoinVideo.mp4 - the effect is a video moving slowly from left to rightCopy the code

3.4 FFmpeg filter diagram, filter chain, relationship between filters

A filterchain contains multiple filters. A filterchain contains multiple filters. A filterchain contains multiple filters. FFmpeg supports a variety of filters. View all filters $FFmPEG-filters

An example of a filter diagram is as follows:

ffmpeg -i out2.mp4 -vf [in[wm] scale = 640.0, 480.0]; movie='logo.png', scale = 92.25:30.0 (logo); [wm] [logo] overlay = main_w overlay_w - 24.0:24.0 [out] output. Mp4Copy the code

In this example, there are three filter chains:

  • [in] scale = 640.0:480.0 [wm]
  • Movie = ‘logo. PNG, scale = 92.25:30.0 (logo)
  • [WM][logo]overlay=main_w-overlay_w-24.0:24.0[out] overlay=main_w-overlay_w-24.0:24.0[out] Filters in the filter chain are separated by commas “,”; The filter chain does not specify input or output. By default, the output of the previous filter chain is used as input and output to the following filter chain is used as input.
Simple and Complex Filters (Simple and Complex FilterGraphs)
  • Simple FilterGraphs: This filter has only one input and one output, which actually adds the actions between the decoding and encoding steps, as shown below.

Simple FilterGraphs configure filter options for each stream (with -VF and -AF aliases for video and audio, respectively).

  • Complex FiltersComplex FilterGraphs are groups of filters that cannot be described as a simple linear processing chain. For example, when a filter group has multiple inputs and/or outputs, or when the output stream type is different from the input. They can be represented in the following diagram:

    Complex filter diagram use-filter_complexOptions to represent, and-vfThe difference is that he has multiple inputs. This option is global, and the -lavfi option is equivalent-filter_complexOne specific example is,overlayFilter, this filter has two video inputs, one video output, the output video is the result of one input video overlaid on top of another video, audio has equalamixA filter

-map :-vf and -filter_complex are combined

-vf = -filter_complex; -vf = -filter_complex; -vf = -filter_complex; This will use the media stream select-map example in article 3 (which I haven’t verified yet)

* * * * * * * * * * * * * * * * * * * * * * * * *, combining vf and filter_complex example, a * * * * * * * * * * * * * * * * * * * * * * * * * * ffmpeg -i input. Mp4 - vf"crop='if(gte(iw,ih),ih,iw):if(gte(ih,iw),iw,ih)', scale=720x720"   

-an -c:v libx264 -profile:v high -level 4.1  -preset superfast -crf 20 

output.mp4

========================================


ffmpeg -i input.mp4 -filter_complex \ 
"[0: v] trim = 0-4. 95, setpts = PTS - STARTPTS (v1); \ [0: v] trim = 4.95:6.75, setpts = PTS - STARTPTS (v2). \ [0: v] trim = 6.75:8, setpts = PTS - STARTPTS (v3); \ [v2] setpts = PTS / 0.1 [vslow2]; \ [v1][vslow2][v3]concat=n=3:v=1:a=0[out]"\ -map [out] -an-C :v libx264-profile: V high-level 4.1 \ -preset superfast -crf 20 -r 30 output-mp4 = = = = = = = = = = = = = = = = = synthesis = = = = = = = = = = = = = = = = = = = = ffmpeg -i input. Mp4 - filter_complex \"[0:v]crop='if(gte(iw,ih),ih,iw):if(gte(ih,iw),iw,ih)',scale=720x720,split=3[1v][2v][3v]; \ [v] 1 trim = 0-4. 95, setpts = PTS - STARTPTS (v1); \ [v] 2 trim = 4.95, 6.75, setpts = (PTS - STARTPTS) / 0.1 (v2); V \ [3] trim = 6.75:8, setpts = PTS - STARTPTS (v3); \ [v1][v2][v3]concat=n=3:v=1:a=0[out]"\ -map [out] -an-C :v libx264-profile: V high-level 4.1 \ -preset superfast -crf 20 -r 30 output-mp4Copy the code

Reference:

  • FFmpeg from entry to mastery – Liu Qi

  • FFmpeg filter knowledge point records

  • FFmpeg overlay Filter usage – Watermark and picture in picture