Realization of video seamless stitching technology based on IJkPlayer-Concat protocol

One, foreword

Hi, everyone, I am Chengxiang Ink shadow!

Get straight to the point. Today we’re going to talk about how you can stitch together multiple videos into one complete, continuous video, and then play it seamlessly.

Such demand should not be biased door, right?

The simplest one is some video apps, which will cut the large video into small video for playing, and some will interrupt the advertisement before playing the video. All these requirements can be covered by the content of this paper.

When it comes to multiple video splicing, if you’ve seen Google’s ExoPlayer, there’s a ConcatenatingMediaSource that allows you to splice multiple video sources together, and the Api is pretty good. It’s basically seamless.

However, ExoPlayer relies on MediaCodec, and in addition to Api Level restrictions (Api Level 16+), it also has hardware requirements for the device. On some low-end machines, you’ll get some weird freezes, mosaics, and screen splashes. And because the package is so good, if you want to add your own soft solution, unfortunately, it’s not easy. Ffmpeg support is limited to Audio and not Video in extensions on Github.

This article is based on another popular open source player, IJKPlayer, to see how to splice video.

Second, the IJKPlayer Concat

IJKPlayer is a lightweight Android/iOS video player based on FFmpeg, which is open source by Bilibili. It is a very popular open source player in China. Many apps use it.

Because it relies on FFmpeg, IJKPlayer can help you solve most of the problems you encounter in video encoding and decoding. It is a very good player.

IJKPlayer itself is already very useful, if you want to play multiple video sources, want to play in sequence, if the requirements are not high or the premise of its transition effect, it is not impossible. But if you want that kind of seamless connection, you’ll see a brief black screen because it takes a while to load a new video feed, and the worse the device, the worse the black screen.

The solution I recommend here is to use FFmpeg’s concat protocol.

2.1 What is Concat

Concat is a virtual cascading script resolver provided by FFmpeg (hereinafter referred to as the CONCAT protocol), which exists in the form of a regular script file. You can use concat to define a video playlist, and FFmpeg will parse it one by one in the order you define it, as if they were a video source themselves.

This may be a bit confusing, but if you know the format of.m3u8, you should be able to understand concat, which defines a list of videos to be played sequently by the player’s decoder.

For specific information, you can go to FFmpeg’s official documentation and refer to the corresponding content.

FFmpeg Doc:

https://ffmpeg.org/ffmpeg-formats.html#concat

2.2 the concat file

To use the concAT protocol, you first need to define a file to parse. It must end with the.ffcat or.ffconcat suffix, and the content header of the file must mark the current concat version number.

There are two configurable options:

File: Specifies a video source to be parsed. This can specify a local file path, or an online Uri. Both are legal.
Duration: Indicates the length of the video source specified in the previous file. According to the official documentation, this is related to seek, so that when you adjust the progress, it can accurately locate the file. Duration is optional because it is an auxiliary parameter that you don’t need if you concatenate videos with the same bit rate and so on.

Here’s an official example of what a complete.ffcat file would look like.

Ffconcat version 1.0# my first filenameThe file/MNT/share/file - 1. Wav duration of 20.0# my second filename including whitespace
file '/mnt/share/file 2.wav'
# my third filename including whitespace plus single quote
file '/mnt/share/file 3'\' '.wav'
Copy the code

2.3 IJKPlayer Options Configuration for Concat

When using IJKPlayer, there are some Settings that are set through the setOption() method, and if concAT protocol is to be supported, the Settings are also required.

Here are two main concerns:

Protocol_whitelist: indicates the protocol whitelist.
Safe: indicates the secure path.

In order for IJKPlayer to support concAT, you need to configure concat into its whitelist protocol, mainly to add both ffConcat and concAT.

ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_FORMAT, "protocol_whitelist"."rtmp,concat,ffconcat,file,subfile,http,https,tls,rtp,tcp,udp,crypto");
Copy the code

Safe specifies that unsafe paths are allowed. The default value is 1, and unsafe file paths are rejected. Sure, what is a safe path? You can test it yourself or check the documentation, but you can turn it off by setting it to 0.

ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_FORMAT,"safe".0);
Copy the code

Now, that’s basically the point, just throw the.ffcat file into IJKPlayer and it will play in the normal order.

Concat content is very small, if you run into problems in the actual operation, it is recommended to pay attention to the Logcat output to see the cause of the error. Of course, it is also helpful to read through the CONCat documentation of FFmpeg.

3. Concat

3.1 What is a Secure Path

As mentioned earlier, the file path following file in concat files must be a secure path. What is a secure path?

According to concat’s documentation, a safe path must be a relative path and contain only (letters) and no special symbols. Digits, periods (.), underscores (_), and the start of the path does not contain the period (.). Is considered a secure path.

Such as:

file a.mp4
Copy the code

It is considered to be the a.m. p4 file in the current. Ffcat file directory, which is a secure path.

On the contrary, the video source path, such as https://, file://, and./, is considered as an insecure path.

3.2 Whether the FFCAT file must be local

FFmpeg simply accepts a concat data stream, and it doesn’t matter whether it’s on a local or remote server.

3.3 Whether file can be mixed

Concat does not enforce that the address of the video source followed by file be all in the same place.

Ffconcat Version 1.0 file /sdcard/a.mp4 file http://down4.xxx.com/hash/c644d9e118417e56d91cba3dc467ab9b.mp4
Copy the code

An example of this is a.ffcat file, which is legally playable.

3.4 There is a black screen flashing in the video stitching

Concat requires that the video to be spliced must have the same stream (the same bit rate and time baseline, etc.), so if these parameters are inconsistent between the two video sources, it may cause a black screen.

This problem, which I have tested on very poor TV boxes, can be seamless if the file flow is consistent. So if this happens to you, don’t be suspicious of FFmpeg’s concat problem, re-transcode your video files with FFmpeg and try again.

https://github.com/alwaystest/Blog/issues/58

https://www.jianshu.com/p/ea794a357b48

Today, I reply to “grow up” in the background of the official account, and I will get some learning materials sorted out by me. I can also reply to “add group”, so that we can learn and progress together.