【Android audio and video development hit strange upgrade: FFmpeg audio and video codec chapter

[sound Ming]

First of all, this series of articles are based on their own understanding and practice, there may be wrong places, welcome to correct.

Secondly, this is an introductory series, covering only enough knowledge, and there are many blog posts on the Internet for in-depth knowledge. Finally, in the process of writing the article, I will refer to the articles shared by others and list them at the end of the article, thanking these authors for their sharing.

Code word is not easy, reproduced please indicate the source!

Tutorial code: [Making portal】

You can read about it in this article

Use GCC or CLANG to cross-compile FFmpeg SO library for Android platform. In order to take the first step in FFmpeg development, we should not only know why, but also why. Not only do you need to know how to compile successfully, but you also need to know why. Before you start, it is recommended to read the entire article. I believe this article will give you some insight.

One, foreword

There has been a lot of sharing of FFmpeg SO library compilations, but most of them are directly posted. I think most people searching for “how to compile the FFmpeg SO library” are new to cross-compiling.

For mobile developers in particular, most of them spend most of their time developing in the Java layer and have little exposure to the NDK layer. If you look directly at a cross-compiled configuration, it’s probably going to go over your head.

It’s not unusual for an FFmpeg compiled article to have a lot of questions like “Why didn’t I compile according to the owner’s configuration?” Comments, that why people can compile success, we copy down but can not?

There are many reasons, most of which are actually concentrated in the following aspects:

1. No brain copy, pray for a fool configuration can be compiled successfully; 2. There are many FFmpeg and NDK versions, each of which may require different configurations; 3. Do not understand the meaning of each configuration item, even if the configuration is correct, but a little modification, and the normal compilation.Copy the code

Why is FFmpeg so difficult?

I think it’s mainly because it’s hard to get your foot in the door. You can’t even compile the SO library. The rest is bullshit.

What is cross-compilation

define

Cross compilation is the generation of executable code on one platform and another platform.

What does that mean? Basically, you build a program on one machine that can run on another machine. For example: Compiling an APK on a PC that runs on an Android phone is actually a cross-compilation process.

Why cross-compile

We know that software on the PC is compiled directly on the PC, so why can’t software on Android compile itself on Android?

In theory, yes, but the resources on Android phones are limited. It takes that long to compile an APK on a PC. Can you imagine how long it takes to compile an APK on an Android phone? Or can you imagine typing code on your phone?

We wondered, since there are so many resources on the PC, can we use the PC to compile software that can run on the phone?

So cross compilation comes in.

What does cross-compilation require

Compile environment

We know that the running environment on the PC is completely different from the running environment on the phone. If you compile directly on the PC, you can imagine that the compiled App will die in a minute.

Therefore, the most important thing for cross-compiling is to configure the relevant environment used during compilation, which is actually the environment in which the target machine (such as Android phone) is running.

Compilation tool chain

For C/C++ compilation, there are usually two tools: GCC and CLANG.

GCC is an old compiler that can compile not only C/C++, but also Java, object-C, Go and other languages.

CLANG is a more efficient C/C++ compilation tool that is compatible with GCC. Google has long recommended using CLANG for compilation, and since NDK 17, GCC has been removed in favor of CLANG.

How to cross-compile FFmpeg

What is the FFmpeg

Famous FFmpeg, not in the audio and video industry, even if a developer does not develop audio and video are also slightly heard.

The official introduction

A complete, cross-platform solution to record, convert and stream audio and video.

FFmpeg is a complete cross-platform solution for recording, converting, and streaming audio and video.

As you can see from this introduction, FFmpeg has the following features:

Powerful: recording, decoding, coding, editing, streaming and so on
cross-platform

The compilation process

From the previous introduction, we can basically summarize the basic process of FFmepg compilation:

Selecting compilation tools
Configure the cross-compilation environment
Configure build parameters (such as removing unnecessary features)
Start the compilation

As simple as that is, let’s take a closer look at how to compile using CLANG and GCC.

4. Build FFmpeg with CLANG

Note: This article is compiled for Mac, it is recommended to use Mac or Linux to compile, it is said that Windows has many pits.

Download the Android the NDK

Android NDK has been iterated over many versions. After R17C, Google officially removed GCC and no longer supports GCC. The new version of NDK is compiled using CLANG.

Here, the latest NDK R20B version is used to compile.

NDK download: Android-NDK

The NDK directory

The main ones are these two paths:

Compiler toolchain directory: toolchains/LLVM/prebuilt/Darwin - x86_64 / bin directory: cross compile environment toolchains/LLVM/prebuilt/Darwin - x86_64 / sysrootCopy the code

Compilation tool path

Different Clang tools are differentiated according to different CPU architectures and Android versions, so you can choose according to your needs.

This paper selects CPU architecture ARMV7A, Android version 21:

armv7a-linux-androideabi21-clang
armv7a-linux-androideabi21-clang++
Copy the code

Compile environment path

In the toolchains/LLVM/prebuilt/Darwin – x86_64 / sysroot directory, contains two catalogues: usr/include, usr/lib, corresponding to the header files and libraries.

Download FFmpeg source code

FFmpeg can be downloaded directly from the official website.

This article uses the latest version, FFMPEG-4.2.2.

Once you’ve downloaded the source code, go to the root directory and find a file called CongFigure, which is a shell script that generates some configuration files for FFmpeg compilation.

This file is so important that FFmpeg is compiled and configured using it.

We’ll look at some of these important things later, which are key to understanding FFmpeg compilation configuration.

With that in mind, FFmpeg is ready to compile.

The configuration script

Modifying the Configure script

newcross_prefix_clangparameter

Open the configure file at the root of ffmPEG-4.2.2, search for CMDLINE_SET, find the following code, and add a command line option: cross_prefix_clang

CMDLINE_SET=" $PATHS_LIST ar arch as assert_level build_suffix cc objcc CPU cross_prefix # Add command line parameter cross_prefix_clang Custom_allocator CXX DEP_cc # Omit others..... "Copy the code

Modify compilation tool path Settings

Search ar_default=”${cross_prefix}${ar_default}” to find the following code

ar_default="${cross_prefix}${ar_default}"
cc_default="${cross_prefix}${cc_default}"
cxx_default="${cross_prefix}${cxx_default}"
nm_default="${cross_prefix}${nm_default}"
pkg_config_default="${cross_prefix}${pkg_config_default}"
Copy the code

Change the middle two lines to

ar_default="${cross_prefix}${ar_default}"
#------------------------------------------------
cc_default="${cross_prefix_clang}${cc_default}"
cxx_default="${cross_prefix_clang}${cxx_default}"
#------------------------------------------------
nm_default="${cross_prefix}${nm_default}"
pkg_config_default="${cross_prefix}${pkg_config_default}"
Copy the code

As for why this modification will be in the laterconfigureDetailed explanation in the analysis

Create a compile configuration script

Create a new shell script in the ffmPEG-4.2.2 root directory and name it build_android_clang.sh

#! / bin/bash the set - x # target = 21 CPU = armv7 Android version API - a # so library OUTPUT directory OUTPUT = / Users/CXP/Desktop/FFmpeg/FFmpeg - 4.2.2 / Android / $# CPU Path of NDK, According to their own position of the NDK set the NDK = / Users/CXP/Desktop/FFmpeg/android - the NDK - r20b # compiler tools link size TOOLCHAIN = $the NDK/toolchains LLVM/prebuilt/Darwin - x86_64 # compiler environment SYSROOT = $TOOLCHAIN/SYSROOT function build {the. / configure \ --prefix=$OUTPUT \ --target-os=android \ --arch=arm \ --cpu=armv7-a \ --enable-asm \ --enable-neon \ --enable-cross-compile \ --enable-shared \ --disable-static \ --disable-doc \ --disable-ffplay \ --disable-ffprobe \ --disable-symver \ --disable-ffmpeg \ --sysroot=$SYSROOT \ --cross-prefix=$TOOLCHAIN/bin/arm-linux-androideabi- \ --cross-prefix-clang=$TOOLCHAIN/bin/armv7a-linux-androideabi$API- \ --extra-cflags="-fPIC" make clean all # Make -j12 make install} buildCopy the code

The shell script, in general, is pretty easy to understand, for example

— disable-static –enable-shared Is used to disable the output of static libraries and dynamic libraries respectively.

Arch — The architecture of the SO library used by the CPU to configure the output;

–prefix configures the path to the output so library.

Let’s focus on a few options:

target-os

– target – OS = android: In the old version of FFmpeg, the support for Android platform is not perfect. There is no Android target, so it is mentioned in some old articles that the compilation of Android platform so library needs to make the following changes to configure. Otherwise, the so library will be output in the standard Linux way, which is not named the same as Android so, Android cannot load.

SLIBNAME_WITH_VERSION='$(SLIBNAME).$(LIBVERSION)' SLIBNAME_WITH_MAJOR='$(SLIBNAME).$(LIBMAJOR)' LIB_INSTALL_EXTRA_CMD='? (RANLIB) "$(LIBDIR)/$(LIBNAME)"' SLIB_INSTALL_NAME='$(SLIBNAME_WITH_VERSION)' SLIB_INSTALL_LINKS='$(SLIBNAME_WITH_MAJOR) $(SLIBNAME)';  SLIBNAME_WITH_VERSION='$(SLIBNAME).$(LIBVERSION)' SLIBNAME_WITH_MAJOR='$(SLIBPREF)$(FULLNAME)-$(LIBMAJOR)$(SLIBSUF)' LIB_INSTALL_EXTRA_CMD='? (RANLIB)"$(LIBDIR)/$(LIBNAME)"' SLIB_INSTALL_NAME='$(SLIBNAME_WITH_MAJOR)' SLIB_INSTALL_LINKS='$(SLIBNAME)'Copy the code

However, in the new version of FFmpeg, this problem has been solved and FFmpeg has added android as the target. So we don’t have to change it manually anymore.

sysroot

Usr /include usr/lib –sysroot=$sysroot: specifies the root path used to configure the cross-compile environment.

$SYSYROOT/usr/include and $SYSYROOT/usr/lib are the headers and libraries of the R20b NDK system.

Basically, a lot of novices can’t find all kinds of header files when compiling, resulting in compilation failure. So this is the first path to check when you can’t find a file in a compile.

A little doubt

When compiling with the latest NDK R20B version, I found that the compilation can be normal even without sysroot configuration. I wonder whether Android clang tool has been processed, and will automatically find the corresponding path. The cause was not found in the configure file.

If there are those who know, still hope to inform ~.

Sysroot is another parameter to mention, isysyroot. This parameter has puzzled me for a long time because there are few articles about the connection and difference between the two parameters, but it is also a parameter that can cause inexplicable compile failures.

extra-cflags

Before introducing -isysroot, take a look at the extra-cflags option.

This option gives the compiler a header search path other than sysroot. Such as:

--extra-cflags=" -i $SYSROOT/usr/include" #Copy the code

-isysroot is a configuration of this option. Such as

--extra-cflags="-isysroot $SYSROOT"
Copy the code

-isysroot sets the following path to the default search path for header files. In this case, the previous sysroot configuration path is no longer the default search path for header files, but still the default search path for library files.

As you can see, the two configurations are somewhat the same:

--extra-cflags="-i $SYSROOT/usr/include" --extra-cflags="-isysroot $SYSROOT"Copy the code

extra-ldflags

This is similar to extra-cflags above, but is used to configure additional library file search paths, such as

--extra-ldflags=" -l $SYSROOT/usr/libCopy the code

You can see that extra-cFlags extra-LDFlags combined can replace sysroot.

cross-prefix

This option literally translates to cross-compile prefix, referring to the prefix of the cross-compile tool.

This option is often used in conjunction with another option cc.

What does that mean? The cc option is configured in two ways:

One is to configure only cross-prefix without cc, as in this article.

The other is to configure both cross-prefix and CC.

Such as:

--cc=$TOOLCHAIN/bin/arm-linux-androideabi-gcc \
--cross-prefix=$TOOLCHAIN/bin/arm-linux-androideabi- \
Copy the code

These are two completely different configurations, but the amazing thing is that sometimes they both compile successfully and sometimes you can’t find the compile chain tool.

In order to understand the effect of cross-prefix cc and how to use it, I took a close look at the configure configuration script in the FFmpeg root directory and found some clues.

Analyze the configure configuration script

Note: The following analysis is based on FFMPEG-4.2.2, other versions may be different, just master the basic principles.

Gets user configuration options

To open the Configure shell script (note: not double-click to run it), start by looking at how Configure fetches user-configured compilation options.

Search for opt do to find the following code

for opt do optval="${opt#*=}" case "$opt" in --extra-ldflags=*) add_ldflags $optval ;; --extra-ldexeflags=*) add_ldexeflags $optval ;; --extra-ldsoflags=*) add_ldsoflags $optval ;; --extra-ldlibflags=*) warn "The --extra-ldlibflags option is only provided for compatibility and will be\n"\ "removed in  the future. Use --extra-ldsoflags instead." add_ldsoflags $optval ;; --extra-libs=*) add_extralibs $optval ;; --disable-devices) disable $INDEV_LIST $OUTDEV_LIST ;; --enable-debug=*) debuglevel="$optval" ;; # omit some middle code... *) optname="${opt%%=*}" optname="${optname#--}" optname=$(echo "$optname" | sed 's/-/_/g') if is_in $optname $CMDLINE_SET; then eval $optname='$optval' elif is_in $optname $CMDLINE_APPEND; then append $optname "$optval" else die_unknown $opt fi ;; esac doneCopy the code

The shell script code has a lot of special syntax, and you don’t have to go through all the details to understand it.

The first line of the for loop gets the user-set option value optval by splitting =.

In addition to some special options, let’s look at the last wildcard *). The purpose of this code is to associate the user-configured options with values.

For example — CPU =armv7-a, the first three lines divide the CPU, assign optname, and assign optval to the CPU. This variable is armv7-a.

Android related configuration

Search the Android keyword and you will find the following code

# ffmpeg - 4.2.2 / configure

if test "$target_os" = android; then
    cc_default="clang"
fi

ar_default="${cross_prefix}${ar_default}"
cc_default="${cross_prefix}${cc_default}"
cxx_default="${cross_prefix}${cxx_default}"
nm_default="${cross_prefix}${nm_default}"
pkg_config_default="${cross_prefix}${pkg_config_default}"
Copy the code

When you configure –target-os= Android, FFmpeg’s default compiler is Clang.

Cc_default is the default value of the configuration item cc, and you can see that cc_default is concatenated with cross_prefix. This is why cross_prefix is the cross-compiler prefix.

It looks like this:

cc_defalut=$TOOLCHAIN/bin/arm-linux-androideabi-$cc
Copy the code

Let’s see what the default values are for ar_default, cc_default, cxx_default.

Search cc_default to find the following code

# ffmpeg - 4.2.2 / configure

ar_default="ar"
cc_default="gcc"
cxx_default="g++"
host_cc_default="gcc"
Copy the code

As you can see, the default compiler for FFmpeg is GCC.

Configure enforces cc_default=”clang” when you compile Android libraries:

When you use GCC as a compilation tool, you must either configure the cc option or change the cc_default=”clang” in configure to cc_default=” GCC “;
When you use CLANG as a compilation tool, you do not need to configure the CC option.

If you think about it, why does cc compile properly when configured to the following values?

--cc=$TOOLCHAIN/bin/arm-linux-androideabi-gcc
Copy the code

So cc_defalut is equal to

cc_defalut=$TOOLCHAIN/bin/arm-linux-androideabi-$TOOLCHAIN/bin/arm-linux-androideabi-gcc
Copy the code

This path must be wrong!

This is where cc_default is used.

Initialize a variable

Search set_default arch, and you see the following code, where configure resets the default cc values.

set_default arch cc cxx doxygen pkg_config ranlib strip sysinclude \
    target_exec x86asmexe nvcc
Copy the code

Here we call a function called set_default. Let’s look at the implementation of this function

set_default() {for opt; do
        eval : \${$opt:=\? {opt}_default}
    done
}
Copy the code

The for loop takes all input parameter variables and assigns a value to that variable.

For example, set_default cc, which means cc=cc_default, but one thing to note is the symbol in the middle :=.

This symbol is similar to the ternary operator in Java:

opt ! = null? opt:opt_defalutCopy the code

That is, if the argument is empty, assign xx_default to xx.

That would explain the above question.

When configuring

--cc=$TOOLCHAIN/bin/arm-linux-androideabi-gcc
Copy the code

Set_default cc is useless. Because cc is not empty after the for loop gets the user’s configuration. The value of cc does not change after set_default.

When CC is not configured, FFmpeg sets the concatenated path to CC according to the default concatenation mode.
If cc= GCC, you will not find the compiler correctly.

Why joincorss-prefix-clangThis option

Now you can explain why you need to modify the configure configuration script earlier.

The original configuration looked like this

ar_default="${cross_prefix}${ar_default}"
cc_default="${cross_prefix}${cc_default}"
cxx_default="${cross_prefix}${cxx_default}"
nm_default="${cross_prefix}${nm_default}"
pkg_config_default="${cross_prefix}${pkg_config_default}"
Copy the code

That is, the default cc AR nm path prefix is the same, but the Android NDK path is

See? The ar/nm and CC prefixes are not the same, the former is arm-linux-androideabi-, the latter is armV7a-linux-androideabi16 -.

Therefore, the cc and CXX prefixes need to be modified, and cross_prefix_clange is added for separate configuration.

This is only for NDK R20B, different NDK versions may be different, according to this principle to set.

In summary, this article explains some of the common configuration options for compiling FFmpeg, and explains in principle why it is done this way. Basically, with this in mind, if you want to combine two different versions of FFmpeg and NDK to compile, it will be relatively easy.

Start the compilation

Open CMD terminal, CD to FFmpeg directory

Enter. / build_android_clang. Sh

After compiling, you will get include and lib in ffmpeg/android/armv7-a, which are header files and so library files respectively

Use GCC to compile FFmpeg

Most of the articles on the web use GCC to compile FFmpeg, so let’s see how to configure GCC.

Download Android NDK R17B

GCC has been removed from Google since NDK r17C, so you can only download r17C and older versions of GCC. This article uses R17C to compile GCC.

Select NDK R17C according to your compilation platform

The Mac version selected for this article is Mac OS X.

Ndk-related environment path

The NDK R17C directory is slightly different from the NDK R20B directory.

Cross-compile the environment path

# android library file path - the NDK - r17c/platforms/android - 21 / arch - arm/usr/libCopy the code

Android-ndk-r17c /sysroot/usr/includeCopy the code

GCC tool link path

Android - the NDK - r17c/toolchains/arm - Linux - androideabi - 4.9 / prebuilt/Darwin - x86_64 / binCopy the code

As you can see, Google has separated the header file from the library file, which is also the reason why many novices fail to compile without pairing paths.

Create a compile configuration script

The FFmpeg version still uses ffMPEG-4.2.2, but you don’t need to change configure this time.

With this knowledge, it’s easy to write a compile configuration

Create a new script in the ffmPEG-4.2.2 root directory: build_android_gcc.sh

#! /bin/bash
set -x
API=21
CPU=armv7-a
#so library output directoryThe OUTPUT = / Users/CXP/Desktop/FFmpeg/FFmpeg - 4.2.2 / android /$CPU
# NDK path, set according to your installation location
NDK=/Users/cxp/Desktop/FFmpeg/android-ndk-r17c
# libraries
SYSROOT=$NDK/platforms/android-$API/arch-arm
# header file
ISYSROOT=$NDK/sysroot/usr/include
# assembler header file
ASM=$ISYSROOT/arm-linux-androideabi
TOOLCHAIN=$NDKToolchains/arm - Linux - androideabi - 4.9 / prebuilt Darwin - x86_64function build
{
./configure \
  --prefix=$OUTPUT \
  --target-os=android \
  --arch=arm \
  --cpu=armv7-a \
  --enable-asm \
  --enable-cross-compile \
  --enable-shared \
  --disable-static \
  --disable-doc \
  --disable-ffplay \
  --disable-ffprobe \
  --disable-symver \
  --disable-ffmpeg \
  --sysroot=$SYSROOT \
  --cc=$TOOLCHAIN/bin/arm-linux-androideabi-gcc \
  --cross-prefix=$TOOLCHAIN/bin/arm-linux-androideabi- \
  --extra-cflags="-I$ISYSROOT -I$ASM -fPIC"

make clean all
# here is the definition of compiling with several cpus
make -j12
make
make install
}
build
Copy the code

As you can see, the basic configuration is pretty much the same as compiling with CLANG.

There are the following differences:

Much moreccConfiguration. Because if you don’t configureccThe default isclang(refer to the previous analysis);
Much moreextra-cflagsBecause of the configurationSYSROOTContains onlyThe library files, requires additional configurationThe header fileThe search path of;Assembly header fileThe path is not thereSYSROOTIn, additional configuration is also requiredASM 。

Start the compilation

Open CMD terminal, CD to ffmPEG-4.2.2 directory

Perform. / build_android_gcc. Sh

Six, summarized

Through the analysis of configure, we can clearly understand the meaning of each parameter configuration item and how to use these configurations together. As long as you know the meaning of each configuration, no matter how the version changes, you can quickly write and compile scripts.

However, this article covers only the most basic configuration options. You can also use the more –disable-xxx option to crop up FFmpeg, or –enable-xxx option to enable some advanced features.

Refer to the article

FFmpeg source code simple analysis: configure

Compile FFmpeg clang

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

【Android audio and video development hit strange upgrade: FFmpeg audio and video codec chapter

[sound Ming]

directory

First, Android audio and video hard decoding:

2. Use OpenGL to render video frames

Android FFmpeg audio and video decoding

You can read about it in this article

One, foreword

What is cross-compilation

define

Why cross-compile

What does cross-compilation require

Compile environment

Compilation tool chain

How to cross-compile FFmpeg

What is the FFmpeg

The compilation process

4. Build FFmpeg with CLANG

Download the Android the NDK

The NDK directory

Download FFmpeg source code

The configuration script

Analyze the configure configuration script

Start the compilation

Use GCC to compile FFmpeg

Download Android NDK R17B

Ndk-related environment path

Create a compile configuration script

Start the compilation

Six, summarized

Refer to the article

【Android audio and video development hit strange upgrade: FFmpeg audio and video codec chapter

[sound Ming]

directory

First, Android audio and video hard decoding:

2. Use OpenGL to render video frames

Android FFmpeg audio and video decoding

You can read about it in this article

One, foreword

What is cross-compilation

define

Why cross-compile

What does cross-compilation require

Compile environment

Compilation tool chain

How to cross-compile FFmpeg

What is the FFmpeg

The compilation process

4. Build FFmpeg with CLANG

Download the Android the NDK

The NDK directory

Download FFmpeg source code

The configuration script

Analyze the configure configuration script

Start the compilation

Use GCC to compile FFmpeg

Download Android NDK R17B

Ndk-related environment path

Create a compile configuration script

Start the compilation

Six, summarized

Refer to the article

Related Posts

Android source graphics system request layout

Consider | what Handler

uni-app android