This is part 2 of the audio and video series — “FFmpeg introduces Android Studio”
Why should we Learn audio and Video Development Techniques? Audio and Video Series (I)
Next: Fundamentals of video Decoding – Encapsulation format
Welcome to continue to pay attention to, feel content good words can also give me to support oh ~
What is FFmpeg?
FFmpeg is a set of open source free cross-platform multimedia framework, it provides a complete solution for recording, conversion and streaming audio and video, in Windows, iOS, Android provides a framework for audio and video processing, FFmpeg from the 90’s until now is very hot.
If you learn FFmpeg thoroughly and learn it well, you will have no problem finding a very good job in China
FFmpeg is widely used in the software applications of top Internet enterprises, such as Google, B station, Douyin, YouTube, Meitu, and even wechat, QQ…
And there’s a very simple way to verify that it’s using FFmpeg
For example, if you have an App, you can decompile it, you downloaded the App before, you can change the suffix to zip, and then you can decompress it into the current folder.
If you go to the lib folder and open it, you will find that there are many lib directories in it. One of them is called FFmpeg, and the FFmpeg library is quite large, you can see that it is over 3M
This also proves that FFmpeg is a must for all applications, as long as they are related to audio and video.
At this point, one might ask, if I just want to play a video, do I need FFmpeg?
It depends on what kind of video you want to play. If you want to play MP4 or AVI, you don’t need FFmpeg, just use our MediaPlayer. However, if we want to add some special effects like Douyin, such as “dog’s head effect”, “out-of-body effect”, and some very beautiful skin whitening, FFmpeg is needed to decode and draw at this time.
So, if you want to learn audio and video codec, then you have to learn FFmpeg.
FFmpeg is awesome, so how do we use it?
After all, FFmpeg is not developed for Android, it is a solution for the entire platform, Windows, iOS, Android, development language is C language
So, if we want to learn FFmpeg, we need to know how to compile it
To compile, you need to have a cloud server. The system image is preferably CentOS, or Ubuntu
This is CentOS 7.3. Let’s copy the IP first, either using Xshell or making a link like my MobaXterm here, so we’ll use the SSH tool to make the connection
We will open the tool, fill in the cloud server, the user name is root, and then we will enter the password
So our cloud server has been successfully connected, and you can see that the cloud server does not have anything
Then let’s see how to build the environment to compile FFmpeg.
Build an environment to compile FFmpeg
FFmpeg it provides a complete solution, but this solution it is written in C, we will FFmpeg source download down after decompression to see, together to see FFmpeg source exactly what is like
This is version 4.1.3, and we’ll follow up with version 4.2.1
From the very beginning to the 4.x.x version of FFmpeg, the middle version has changed a lot, such as 2.0-3.0, 3.0-4.0 are quite different, the versions after 4.0 are compiled by NDK Clang, GCC will not be used for compilation. So we will mainly explain the process of Clang cross-compilation
We opened the FFmpeg source code, clicked on the folder and found that all the code in the folder was C and C++ code, hundreds of files, if we put these files into Android Studio to compile, then you would need quite a long time to compile an Apk. That’s why we need to cross compile to generate so and put it in our project, right
So how do we normally use FFmpeg?
We download the source of FFmpeg, and then compile to the Android system, compile into a dynamic library or static library, then this compilation process needs our compilation environment, not Windows, must be Linux. This Linux we use CentOS 7.3, and then use Linux to compile the source code to generate dynamic library or static library, put it into our App, and then call through the header file
Enter FFmpeg’s official website, we can download the source code in it, but if we download to Windows, there is no egg to use, we have to download to Linux
Go back to our server and download the FFmpeg source code
Create an FFmpeg, which we also use when we write live audio and video streams
So we have a FFmpeg is finished, finished, but also need to decompress
Let’s call the decompression command XVF, and then extract FFmpeg
In the process of decompression, you can find that it has a lot of C files
This is our FFmpeg source code, FFmpeg source code we need to compile, in Linux there is GCC, then we can use Linux GCC to compile FFmpeg?
Obviously not, because GCC in Linux is not the same as our Android GCC tools. Linux builds things that can only be used in Linux, and can not be ported to our Android system, this time we need to learn another technology – cross compilation.
Currently, after NDK 17, the recommended cross-tool is called Clang. In previous versions, we all used GCC for Android.
What’s the difference between GCC and Clang for Android?
To help us compile faster and execute C code faster, we developed a very handy compilation tool called Clang
FFmpeg will also be compiled using the latest NDK version, Clang
Now that the source code is down, we still need a compiler — NDK
Configure the NDK environment
Because there are many C compiler tools in the NDK source files, we have to use NDK compiler tools and not Linux compiler tools. We call this compilation cross compilation.
Back to the cloud server, we need to download an NDK, you can choose NDKr20 when downloading, we will use this version to compile
Then we need to environment configure the NDK
Mkdir NDK Creates an NDK directory
It is recommended that you buy a server here is the best choice of Hong Kong server, because Hong Kong server download these foreign software is very fast, such as my server, in fact, it is a Hong Kong server
We then use the wget command to download the NDK. The NDK is relatively large, but due to the Hong Kong server, the download is still very fast
After the download, run the zip command to decompress the downloaded file
Unzip, the decompression time may be a bit long, depending on the configuration of each server, the decompression time will be different
After the decompression is complete, we can configure the NDK environment. This configuration environment is a little different from Windows. In Windows, we can configure it directly in my computer, but in Linux, it is different, we need to edit etc/profile
To add our environment at the bottom, we set an NDK ROOT and add it to the PATH
Then exit and let the environment take effect
By now, our NDK environment has been set up, this time we need to compile FFmpeg will need to use the shell script
A shell script
Shell scripts are specifically designed for Linux to compile syntax, similar to Java syntax
Speaking of which, should we learn shell syntax?
Learning is definitely necessary, but just a general understanding of how shell scripts are written is enough for this series of articles
Let’s take a look at the contents of this script. The script is used to package the FFmpeg source code into the.so file, or.a static library, that we want to use in Android projects
Create an android_build.sh in the FFmpeg source folder
Copy the following code directly in
#! /bin/bash export NDK=/root/ NDK /android-ndk-r20 # Export HOST_TAG= Linux-x86_64 # ARCH= aARCH64 # export CPU=armv8-a export ARCH=armv7a export CPU=armv7-a # ANDROID_NDK_PLATFORM=android-21 export PREFIX=$(pwd)/android/$CPU export MIN_PLATFORM=$NDK/platforms/android-$MIN export SYSROOT=$NDK/sysroot export TOOLCHAIN=$NDK/toolchains/llvm/prebuilt/$HOST_TAG export AR=$TOOLCHAIN/bin/arm-linux-androideabi-ar export AS=$TOOLCHAIN/bin/arm-linux-androideabi-as export CC=$TOOLCHAIN/bin/$ARCH-linux-androideabi$MIN-clang echo "-----------------------------" echo $CC export CXX=$TOOLCHAIN/bin/$ARCH-linux-androideabi$MIN-clang++ export LD=$TOOLCHAIN/bin/arm-linux-androideabi-ld export NM=$TOOLCHAIN/bin/arm-linux-androideabi-nm export RANLIB=$TOOLCHAIN/bin/arm-linux-androideabi-ranlib export STRIP=$TOOLCHAIN/bin/arm-linux-androideabi-strip OPTIMIZE_CFLAGS="-DANDROID -I$NDK/sysroot/usr/include/arm-linux-androideabi/" ADDI_LDFLAGS="-Wl,-rpath-link=$MIN_PLATFORM/arch-arm/usr/lib -L$MIN_PLATFORM/arch-arm/usr/lib -nostdlib" sed -i "" "s/SLIBNAME_WITH_MAJOR='\$(SLIBNAME).\$(LIBMAJOR)'/SLIBNAME_WITH_MAJOR='\$(SLIBPREF)\$(FULLNAME)-\$(LIBMAJOR)\$(SLIBSUF) '/" configure sed -i "" "s/LIB_INSTALL_EXTRA_CMD='\$\$(RANLIB) \"\$(LIBDIR)\\/\$(LIBNAME)\"'/LIB_INSTALL_EXTRA_CMD='\$\$(RANLIB) \"\$(LIBDIR)\\/\$(LIBNAME)\"'/" configure sed -i "" "s/SLIB_INSTALL_NAME='\$(SLIBNAME_WITH_VERSION)'/SLIB_INSTALL_NAME='\$(SLIBNAME_WITH_MAJOR)'/" configure sed -i "" "s/SLIB_INSTALL_LINKS='\$(SLIBNAME_WITH_MAJOR) \$(SLIBNAME)'/SLIB_INSTALL_LINKS='\$(SLIBNAME)'/" configure sed -i -e 's/#define getenv(x) NULL/\/\*#define getenv(x) NULL\*\//g' config.h # sed -i "" "s/SHFLAGS='-shared -Wl,-soname,\$(SLIBNAME)'/SHFLAGS='-shared -soname \$(SLIBNAME)'/" configure # sed -i "" "s/-Wl//g" configure ./configure \ --prefix=$PREFIX \ --ar=$AR \ --as=$AS \ --cc=$CC \ --cxx=$CXX \ --nm=$NM \ --ranlib=$RANLIB \ --strip=$STRIP \ --arch=$ARCH \ --target-os=android \ --enable-cross-compile \ --disable-asm \ --enable-shared \ --disable-static \ --disable-ffprobe \ --disable-ffplay \ --disable-ffmpeg \ --disable-debug \ --disable-symver \ --disable-stripping \ --extra-cflags="-Os -fpic $OPTIMIZE_CFLAGS"\ --extra-ldflags="$ADDI_LDFLAGS" sed -i "" "s/#define HAVE_TRUNC 0/#define HAVE_TRUNC 1/" config.h sed -i "" "s/#define HAVE_TRUNCF 0/#define HAVE_TRUNCF 1/" config.h sed -i "" "s/#define HAVE_RINT 0/#define HAVE_RINT 1/" config.h sed -i "" "s/#define HAVE_LRINT 0/#define HAVE_LRINT 1/" config.h sed -i "" "s/#define HAVE_LRINTF 0/#define HAVE_LRINTF 1/" config.h sed -i "" "s/#define HAVE_ROUND 0/#define HAVE_ROUND 1/" config.h sed -i "" "s/#define HAVE_ROUNDF 0/#define HAVE_ROUNDF 1/" config.h sed -i "" "s/#define HAVE_CBRT 0/#define HAVE_CBRT 1/" config.h sed -i "" "s/#define HAVE_CBRTF 0/#define HAVE_CBRTF 1/" config.h sed -i "" "s/#define HAVE_COPYSIGN 0/#define HAVE_COPYSIGN 1/" config.h sed -i "" "s/#define HAVE_ERF 0/#define HAVE_ERF 1/" config.h sed -i "" "s/#define HAVE_HYPOT 0/#define HAVE_HYPOT 1/" config.h sed -i "" "s/#define HAVE_ISNAN 0/#define HAVE_ISNAN 1/" config.h sed -i "" "s/#define HAVE_ISFINITE 0/#define HAVE_ISFINITE 1/" config.h sed -i "" "s/#define HAVE_INET_ATON 0/#define HAVE_INET_ATON 1/" config.h sed -i "" "s/#define getenv(x) NULL/\\/\\/ #define getenv(x) NULL/" config.hCopy the codeThis is a shell script that calls our./configure file
This./configure file is the configure file in the build.sh sibling directory
Let’s go straight to the FFmpeg source code, and in the FFmpeg source code directory there’s this thing called configure, and then we’ll open it up
This./configure is also a shell file, it’s an executable shell file, just like we have a main function in Java
How do we know if it’s a shell file?
Its first words are #! So /bin/sh means that it’s a shell file, and the shell file is executable, so it’s basically like main and maybe some arguments are passed in, like we often go to Java and write main, and then main has a steing array, We can pass some arguments into this Java file by executing it on the command line. Again, it passes these arguments in shell files, but the arguments passed in configure are more complicated
Anyway, we execute this shell file only to call the phrase./configure to execute our shell file
Parameter Settings
The following are all parameters passed
The above is a variable to define some parameters to it, similar to the global variable
Let’s look at./configure and the first –prefix means which directory do we want to print
We finally compiled is a dynamic library, then the dynamic library it certainly has the folder, but the files in this prefix the variables, we find up, find the prefix define variable declarations, declarations in our current catalog
PWD is a command that can be invoked directly by the shell. For example, PWD can output to the current directory
We compile the armv7a. During the compilation process, we will generate an Android file, and then generate a subfile called armCPUv7a under the Android file. Finally, we will put our compiled results in this folder
— AR is a link compiler in Clang, we can open the NDK, I open it in the AndroidStudio NDK, and then we open it in the Android/SDK directory
There is an NDK-bundle in this directory, we directly open Windows, Windows and Linux are similar, because they are the same version, but the suffix of one is.exe and the other is the Linux executable file
We open the NDK bundle and then toolChains
Toolchains: Stand for toolchains
Going in, we can find a lot of executable files in Windows with the suffix.exe
Then it has the following files that need to be passed in FFmpeg
We need to specify the paths ar, AS, nm and strip, which are mainly used to do a link compilation and speed up our compilation of SO
So you’ll notice that when you pass arguments, the first one is –ar, and its value is the variable AR, and up here we’ll define a variable called AR
His paths are the ones below, the ones we just saw
These all serve as link compilation tools
There were no such files in previous NDK versions, there was only GCC, but now we need to specify the contents of the cross-compile tool chain with Clang
Paths like the above are used to specify the tools in our cross-tool chain that FFmpeg will compile with when we specify them
And then the two main parameters are the next –cc, — CXX, in the old days it was — GCC, and now it’s cc, which means Clang
$CC is the variable defined
If we look around, we’ll find a lot of tools for compiling Clang
Clang ++ is used to compile C++, and cang is used to compile C files. We only need to specify these two to implement Clang syntax for cross-compilation
Next — ARCH stands for platform, which we compiled called armv7a, and we found the armv7a platform on top of it
The CPU’s platform can be either arm or x86. If you specify multiple platforms, you only need to use Spaces
Target-os = Android means that what we’re compiling is running in the Android directory, it can be compiled for ios or it can be compiled for Windows, the platform is different
Enable-cross-compile means to allow mixed compilation, because we need to specify the compiler, which is Clang and Clang++ that we see in the NDK. If you disable enable, There is no way for it to compile for Android
Disable -asm then speeds up our compilation
Enable-shared means we compile a dynamic library, shared means a dynamic library, and static means a static library
If you are compiling a dynamic library, make sure you disable the static library
Next, in order to reduce the volume of our compilation, we disabled some things that are not commonly used, such as ffProbe and ffPlay. Ffplay is a player, which is usually used in Windows and Linux. It is an executable tool, we can use ffPlay to analyze the video
Debug is also turned off, and the following extra-cflags is a flag
These general syntax is to stop here, shell syntax we do not need to go to the deep, you just need to copy our shell script to the server, and then execute OK
If you want to compile a static library, you can enable it and disable a dynamic library
If you’re building for x86, let’s change the CPU’s build platform variable to x86
Compile FFmpeg
Let’s go ahead and execute this, using sh build.sh directly
The compilation process is a bit slow, maybe more than 10 minutes, and relatively fast if you have a good server
At this point we have not finished compiling and need to execute make
Since all commands are compiled through makefiles, we need the make command to further compile
After a long wait, the compilation of FFmpeg is here
Then we need to execute a command called make install
Make install outputs the files we just compiled to the Android directory
Let’s first see if the current directory has an Android directory
Obviously, no, at this point we go to make install
Make install it command execution speed is relatively fast, after the execution of we again to ls, this time will be more than an Android directory
Open the Android directory and we see a very familiar name called ARMv7a
Keep going until you get to Lib, at which point the final product is presented to us
There are seven so’s in total:
Avcodec: Is related to the codec, because we need to decode the video as it plays; To encode it when we’re editing the video, we’re going to use avCodec
Avdevice: Platform is different, is a device related so library
Avfilter: There is a filter, because FFmpeg itself has some beauty, whitening, then we can add some dynamic filter
Avformat: is related to the format, because in FFmpeg even FLV, MP4, RMVB, even some of our first teacher old teacher’s small film it is also supported
Avutil: this contains the utility classes used by FFmpeg. The utility functions are stored in Avutil
Swresample: it is used for resampling. For example, we need to resampling audio. For mp3 and WMA audio, we need to resampling it and output it to the player for playback
Swscale: is a converter that will be used later when we implement the FFmpeg universal player
We just introduced the seven modules of FFmpeg, and we will compile these so into AndroidStudio and then integrate the project into the FFmpeg environment
Here is the end of this article, friends, see you in the next article
Welcome to continue to pay attention to, feel content good words can also give me to support oh ~
Official account: Android Development House (a gathering place for first-line developers, mainly sharing Android related technical articles, learning materials, video tutorials, hot information, tool resources, course books, etc.)
Next: Fundamentals of video Decoding – Encapsulation format