The Android project introduces OpenCV and FFmpeg

Because the graduation project involves the processing of audio and video on Android phones, which requires the use of OpenCV and FFmpeg libraries, I have been looking for how to introduce OpenCV and FFmpeg into Android for some time. There are many relevant tutorials available on the Internet, but many of them have problems because the tutorials have been written for a long time. This article takes OpencV-4.5.1 and FFMPEG-4.4 as examples to record the process of introducing two audio and video libraries into the Android project.

Opencv – 4.5.2

OpenCV Android SDK + OpenCV dynamic library + JNI (can use Java and C++ development)

Development environment: macOS Big SUR-11.2.3 + Android Studio-4.1.2

Download related tools. Click SDK Manager in the toolbar, click SDK Tools, select CMake and NDK(Side by Side), and click Apply to download the required Tools
Create an Android C++ project
Download the OpenCV – Android SDK. Download OpenCV Release
The Android project introduces Opencv-SDK. Click File -> new -> Import Module, select the opencv-Android-sdk/SDK/ Java folder, click OK, the Module will be automatically identified, after the Import is completed, the OpenCV library will be found in the project directory. Settings. gradle files will also change accordingly
Modify the SDK version of the OpenCV library. Click File -> Project structure-Modules, select OpenCV module, modify SDK version to the corresponding SDK version of your Project
APP introduces OpenCV dependency. Open build.gradle in app directory and add OpenCV dependency
```
dependencies {
    ...
    // Add the OpenCV Module name after path
	implementation project(path: ':opencv452')}Copy the code
```
Click File -> Project structure-Dependencies and select app module to add opencV452 Dependencies
Modify build.gradle Settings in Opencv module.
- Apply plugin: ‘com.android.application’ to apply plugin: ‘com.android.library’
- To get rid of
```
    defaultConfig {
        applicationId "org.opencv"
    }
Copy the code
```
Finally, build.gradle in Opencv module is shown below:
Copy all contents in opencv-Android-sdk/SDK/ native/libs directory to project directory /app/ SRC /main/jniLibs (jniLibs directory needs to be created)
Copy the header file. The OpenCV – Android SDK/SDK/native/jni/include directory all content is copied to the project directory/app/libs/include directory

Edit CMakeList. TXT. Open/app/SRC/main/CPP/CMakeLists. TXT file, add the following content

# Makefile set(CMAKE_VERBOSE_MAKEFILE on) # call header directories(.. /.. /.. Add_library (opencv_lib SHARED IMPORTED) IMPORTED_LOCATION is followed by the relative path set_target_properties(opencV_lib PROPERTIES IMPORTED_LOCATION) of jniLibs ${CMAKE_CURRENT_SOURCE_DIR}/.. /jniLibs/${ANDROID_ABI}/libopencv_java4.so)Copy the code

Modify the build. Gradle. Modify build.gradle under app module and add the following content

Android {defaultConfig {cmake {… }}}, see the code below

defaultConfig { applicationId "com.bupt.compose3dvideo" minSdkVersion 25 targetSdkVersion 30 versionCode 1 versionName "1.0" testInstrumentationRunner "androidx. Test. Runner. AndroidJUnitRunner" externalNativeBuild {cmake {cppFlags "" Arguments "-dandroid_stl =c++_shared" // optional 'arm64-v8a', 'armeabi-v7a', 'x86', 'x86_64' abiFilters 'x86', 'armeabi-v7a' } } }Copy the code

The android {… }, see the code below for details

SourceSets {main {jni.srcDirs = ['/ SRC /main/ CPP '] jnilibs. srcDirs = [' SRC /main/jniLibs']}}Copy the code

Initialize OpenCV.
- Create a BaseApplication or initialize it in MainActivity. Call opencVLoader.initDebug () in onCreate() to initialize OpenCV
- If you initialize BaseApplication, you need to add the following code manually. It already exists in MainActivity and does not need to be added manually
```
    companion object {
        // Used to load the 'native-lib' library on application startup.
        init {
            System.loadLibrary("native-lib")}}Copy the code
```

The above is all the steps to introduce Opencv library, because the introduction of opencv-android-sdk and Opencv dynamic library, will greatly increase the size of the installation package, but the advantage is that you can use Java and C++ programming, and use FFmpeg also need to use JNI, At this point, the first introduction of OpenCV dynamic library, convenient for the later introduction of FFmpeg.

FFmpeg – 4.4 –

The FFmpeg library needs to compile itself first and then introduce the Android project. The biggest problem I had was compiling FFmpeg for Android, because the tutorials on the Internet were mainly compiled using the previous version of NDK-20, but there were some differences in the way of compiling before and after NDK-20, so I always failed to compile at first, then I found an article and finally compiled according to the tutorial.

Build FFmpeg for Android

Development environment: macOS Big SUR-11.2.3 + Android Studio-4.1.2 + NK-R22

Install YASM, PKG-config, and GCC using Homebrew
FFmpeg NDK NDK NDK NDK NDK

In the FFmpeg folder, create a build_android.sh file with the following contents

#! / bin/bash # modify the NDK local the NDK directory for you = / Users/XXX/android - the NDK - r22b TOOLCHAIN = $the NDK/toolchains LLVM/prebuilt/Darwin - x86_64 API = 30 Function build_android() {CD ffmpeg-4.4 echo "Compiling ffmpeg for $CPU"./configure \ --prefix=$prefix \ --libdir=$LIB_DIR \ --enable-shared \ --disable-static \ --enable-jni \ --disable-doc \ --disable-symver \ --disable-programs \ --target-os=android \ --arch=$ARCH \ --cpu=$CPU \ --cc=$CC \ --cxx=$CXX \ --enable-cross-compile \ --sysroot=$SYSROOT \ --extra-cflags="-Os -fpic $OPTIMIZE_CFLAGS" \ --extra-ldflags="$ADDI_LDFLAGS" \ --disable-asm \ $COMMON_FF_CFG_FLAGS make clean make -j8 make install echo "The Compilation of FFmpeg for $CPU is completed" echo "Output library : $LIB_DIR" cd .. } # based on the CPU you need, ## # function v8() {# source "config-env.sh" # OUTPUT_FOLDER="arm64-v8a" # ARCH=arm64 # CPU="armv8-a" # TOOL_CPU_NAME=aarch64 # TOOL_PREFIX="$TOOLCHAIN/bin/$TOOL_CPU_NAME-linux-android" # CC="$TOOL_PREFIX$API-clang" # CXX="$TOOL_PREFIX$API-clang++" # SYSROOT="$NDK/toolchains/llvm/prebuilt/darwin-x86_64/sysroot" # PREFIX="${PWD}/android/$OUTPUT_FOLDER" # LIB_DIR="${PWD}/android/libs/$OUTPUT_FOLDER" # OPTIMIZE_CFLAGS="-march=$CPU" # build_android # } # ## armv7-a # function  v7() { # source "config-env.sh" # OUTPUT_FOLDER="armeabi-v7a" # ARCH="arm" # CPU="armv7-a" # TOOL_CPU_NAME=armv7a # TOOL_PREFIX="$TOOLCHAIN/bin/arm-linux-androideabi" # CC="$TOOLCHAIN/bin/armv7a-linux-androideabi$API-clang" # CXX="$TOOLCHAIN/bin/armv7a-linux-androideabi$API-clang++" # SYSROOT="$NDK/toolchains/llvm/prebuilt/darwin-x86_64/sysroot" # PREFIX="${PWD}/android/$OUTPUT_FOLDER" # LIB_DIR="${PWD}/android/libs/$OUTPUT_FOLDER" # OPTIMIZE_CFLAGS="-march=$CPU" # build_android # } # ## x86 # function x86() { # source "config-env.sh" # OUTPUT_FOLDER="x86" # ARCH="x86" # CPU="x86" # TOOL_CPU_NAME="i686" # TOOL_PREFIX="$TOOLCHAIN/bin/${TOOL_CPU_NAME}-linux-android" # CC="$TOOL_PREFIX$API-clang" # CXX="$TOOL_PREFIX$API-clang++" # SYSROOT="$NDK/toolchains/llvm/prebuilt/darwin-x86_64/sysroot" # PREFIX="${PWD}/android/$OUTPUT_FOLDER" # LIB_DIR="${PWD}/android/libs/$OUTPUT_FOLDER" # OPTIMIZE_CFLAGS="-march=i686 -mtune=intel -mssse3 -mfpmath=sse -m32" # build_android # } ## x86_64 function x86_64() { source "config-env.sh" OUTPUT_FOLDER="x86_64" ARCH="x86_64" CPU="x86-64" TOOL_CPU_NAME="x86_64" TOOL_PREFIX="$TOOLCHAIN/bin/${TOOL_CPU_NAME}-linux-android" CC="$TOOL_PREFIX$API-clang" CXX="$TOOL_PREFIX$API-clang++" SYSROOT="$NDK/toolchains/llvm/prebuilt/darwin-x86_64/sysroot" PREFIX="${PWD}/android/$OUTPUT_FOLDER" LIB_DIR="${PWD}/android/libs/$OUTPUT_FOLDER" OPTIMIZE_CFLAGS="-march=$CPU" build_android} # Then echo "Usage:" echo "$0 all" echo" $0 armV8 "echo" all support CPU :armv7 armV8 x86 x86_64" exit -1 fi # Check build all HAVE_ALL=0 for arg in $@; do if [ "$arg" == 'all' ]; then HAVE_ALL=1 fi done if [ $HAVE_ALL -eq 1 ]; Then x86 x86_64 V8 v7 exit 0 fi # do if [ "$arg" == 'x86' ]; then x86 fi if [ "$arg" == 'x86_64' ]; then x86_64 fi if [ "$arg" == 'armv8' ]; then v8 fi if [ "$arg" == 'armv7' ]; Then v7 fi done # Ffmpeg does not need to manually modify the ffmpeg configure file because it does not have a version number # 3. So have a CD process # directory structure # tree-l 1 #. # ├─ android # Export directory # ├─ build_ffmpeg_android.sh # Script # ├─ config-env.sh # option # ├─ /configure -h = configure -h = configure -h = configure -h = configure -h -rwxr-xr-x 1 caijinglong staff 3.5m 7 26 16:28. so # -rwxr-xr-x 1 caijinglong staff 106K 7 26 16:28 libavfilter.so # -rwxr-xr-x 1 caijinglong staff 771K 7 26 16:28 libavformat.so # -rwxr-xr-x 1 caijinglong staff 443K 7 26 16:28 libavutil.so # -rwxr-xr-x 1 caijinglong staff 82K 7 26 16:28 libswresample.so # -rwxr-xr-x 1 caijinglong staff 398K 7 26 16:28 libswscale.so # If you want to merge all the static libraries into one libffmpeg.so library, as ijkplayer does, you need to compile the static libraries into one so library. # gcc-shared-o-wl --whole-archive avcodec.a avfile.a other. a-wl,--no-whole-archiveCopy the code

Create a new config-env.sh file in the FFmpeg folder. This environment is the FFmpeg configuration of ijkPlayer

#! /usr/bin/env bash

# shell/config.sh
# 这个来自ijkplayer的ffmpeg配置, 会在上面的build脚本中被引入

#--------------------
# Standard options:
export COMMON_FF_CFG_FLAGS=
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --prefix=PREFIX"

# Licensing options:
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-gpl"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-version3"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-nonfree"

# Configuration options:
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-static"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-shared"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-small"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-runtime-cpudetect"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-gray"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-swscale-alpha"

# Program options:
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-programs"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-ffmpeg"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-ffplay"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-ffprobe"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-ffserver"

# Documentation options:
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-doc"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-htmlpages"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-manpages"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-podpages"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-txtpages"

# Component options:
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-avdevice"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-avcodec"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-avformat"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-avutil"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-swresample"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-swscale"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-postproc"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-avfilter"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-avresample"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-pthreads"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-w32threads"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-os2threads"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-network"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-dct"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-dwt"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-lsp"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-lzo"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-mdct"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-rdft"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-fft"

# Hardware accelerators:
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-d3d11va"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-dxva2"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-vaapi"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-vda"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-vdpau"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-videotoolbox"

# Individual component options:
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-everything"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-encoders"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-encoder=png"

# ./configure --list-decoders
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-decoders"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=aac"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=aac_latm"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=flv"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=h264"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=mp3*"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=vp6f"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=flac"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=hevc"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=vp8"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=vp9"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=h263"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=h263i"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=h263p"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=mpeg4"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=mjpeg"

export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-hwaccels"

# ./configure --list-muxers
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-muxers"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-muxer=mp4"

# ./configure --list-demuxers
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-demuxers"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=aac"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=concat"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=data"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=flv"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=hls"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=live_flv"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=mov"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=mp3"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=mpegps"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=mpegts"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=mpegvideo"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=flac"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=hevc"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=webm_dash_manifest"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=mpeg4"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=rtsp"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=mjpeg"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-demuxer=avi"

# ./configure --list-parsers
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-parsers"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-parser=aac"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-parser=aac_latm"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-parser=h264"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-parser=flac"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-parser=hevc"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-parser=mpeg4"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-parser=mpeg4video"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-parser=mpegvideo"

# ./configure --list-bsf
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-bsfs"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-bsf=chomp"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-bsf=dca_core"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-bsf=dump_extradata"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-bsf=hevc_mp4toannexb"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-bsf=imx_dump_header"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-bsf=mjpeg2jpeg"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-bsf=mjpega_dump_header"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-bsf=mov2textsub"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-bsf=mp3_header_decompress"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-bsf=mpeg4_unpack_bframes"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-bsf=noise"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-bsf=remove_extradata"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-bsf=text2movsub"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-bsf=vp9_superframe"

# ./configure --list-protocols
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-protocols"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-protocol=async"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-protocol=bluray"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-protocol=concat"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-protocol=crypto"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-protocol=ffrtmpcrypt"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-protocol=ffrtmphttp"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-protocol=gopher"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-protocol=icecast"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-protocol=librtmp*"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-protocol=libssh"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-protocol=md5"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-protocol=mmsh"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-protocol=mmst"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-protocol=rtmp*"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-protocol=rtmp"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-protocol=rtmpt"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-protocol=rtp"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-protocol=sctp"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-protocol=srtp"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-protocol=subfile"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-protocol=unix"

#
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-devices"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-filters"

# External library support:
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-iconv"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-audiotoolbox"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-videotoolbox"

# ...

# Advanced options (experts only):
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --cross-prefix=${FF_CROSS_PREFIX}-"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-cross-compile"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --sysroot=PATH"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --sysinclude=PATH"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --target-os=TAGET_OS"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --target-exec=CMD"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --target-path=DIR"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --toolchain=NAME"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --nm=NM"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --ar=AR"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --as=AS"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --yasmexe=EXE"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --cc=CC"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --cxx=CXX"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --dep-cc=DEPCC"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --ld=LD"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --host-cc=HOSTCC"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --host-cflags=HCFLAGS"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --host-cppflags=HCPPFLAGS"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --host-ld=HOSTLD"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --host-ldflags=HLDFLAGS"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --host-libs=HLIBS"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --host-os=OS"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --extra-cflags=ECFLAGS"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --extra-cxxflags=ECFLAGS"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --extra-ldflags=ELDFLAGS"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --extra-libs=ELIBS"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --extra-version=STRING"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --optflags=OPTFLAGS"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --build-suffix=SUFFIX"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --malloc-prefix=PREFIX"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --progs-suffix=SUFFIX"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --arch=ARCH"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --cpu=CPU"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-pic"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-sram"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-thumb"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-symver"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-hardcoded-tables"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-safe-bitstream-reader"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-memalign-hack"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-lto"

# Optimization options (experts only):
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-asm"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-altivec"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-amd3dnow"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-amd3dnowext"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-mmx"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-mmxext"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-sse"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-sse2"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-sse3"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-ssse3"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-sse4"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-sse42"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-avx"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-fma4"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-armv5te"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-armv6"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-armv6t2"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-vfp"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-neon"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-vis"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-inline-asm"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-yasm"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-mips32r2"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-mipsdspr1"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-mipsdspr2"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-mipsfpu"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-fast-unaligned"

# Developer options (useful when working on FFmpeg itself):
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-coverage"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-debug"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-debug=LEVEL"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-optimizations"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-extra-warnings"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-stripping"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --assert-level=level"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-memory-poisoning"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --valgrind=VALGRIND"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-ftrapv"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --samples=PATH"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-xmm-clobber-test"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-random"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-random"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-random=LIST"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-random=LIST"
# export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --random-seed=VALUE"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-linux-perf"
export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --disable-bzlib"
Copy the code

Access the FFmpeg file directory from the terminal and execute the script
- chmod +x build_android.shExample Modify the script execution permission
- ./build_android.sh allRun the script
- During the compilation process, a message will be displayed indicating that the execution software such as Clang is not secure. Click Allow in Mac Settings – Security & Privacy – General
Once compiled, an Android folder is generated in the FFmpeg folder, where the.so file is the compiled library

Android project introduces FFmpeg

Create a new C++ project or use the OpenCV project above
Copy the. So file of android/libs corresponding to CPU model in FFmpeg folder to android project app/ SRC /main/jniLibs/ corresponding to CPU model folder

Modify the build.gradle file

Android {defaultConfig {cmake {… }}}, see the code below

defaultConfig { applicationId "com.bupt.compose3dvideo" minSdkVersion 25 targetSdkVersion 30 versionCode 1 versionName "1.0" testInstrumentationRunner "androidx. Test. Runner. AndroidJUnitRunner" externalNativeBuild {cmake {cppFlags "" 'arm64-v8A ', 'armeabi-v7a', 'x86', 'x86_64' abiFilters 'x86', 'armeabi-v7a'}}}Copy the code

The android {… }, see the code below for details

SourceSets {main {jni.srcDirs = ['/ SRC /main/ CPP '] jnilibs. srcDirs = [' SRC /main/jniLibs']}}Copy the code

Edit CMakeList. TXT. Open/app/SRC/main/CPP/CMakeLists. TXT file, add the following content

Add_library (avcodec SHARED IMPORTED) IMPORTED_LOCATION IMPORTED_LOCATION Set_target_properties (AVCOdec PROPERTIES IMPORTED_LOCATION ${CMAKE_CURRENT_SOURCE_DIR}/.. /jniLibs/${ANDROID_ABI}/libavcodec.so) add_library(avformat SHARED IMPORTED) set_target_properties(avformat PROPERTIES IMPORTED_LOCATION ${CMAKE_CURRENT_SOURCE_DIR}/.. /jniLibs/${ANDROID_ABI}/libavformat.so) add_library(avutil SHARED IMPORTED) set_target_properties(avutil PROPERTIES IMPORTED_LOCATION ${CMAKE_CURRENT_SOURCE_DIR}/.. /jniLibs/${ANDROID_ABI}/libavutil.so) add_library(swresample SHARED IMPORTED) set_target_properties(swresample PROPERTIES IMPORTED_LOCATION ${CMAKE_CURRENT_SOURCE_DIR}/.. /jniLibs/${ANDROID_ABI}/libswresample.so) add_library(swscale SHARED IMPORTED) set_target_properties(swscale PROPERTIES IMPORTED_LOCATION ${CMAKE_CURRENT_SOURCE_DIR}/.. /jniLibs/${ANDROID_ABI}/libswscale.so) add_library(avfilter SHARED IMPORTED) set_target_properties(avfilter PROPERTIES IMPORTED_LOCATION ${CMAKE_CURRENT_SOURCE_DIR}/.. /jniLibs/${ANDROID_ABI}/libavfilter.so)Copy the code

Create a new JNI folder in the com/bupt/ opencvandFFmpeg directory, create a new FFmpegCmd class, implement the FFmpeg function call, class content is as follows:

object FFmpegCmd {
    private external fun run(cmdLen: Int, cmd: Array<String>): Int
    external fun test(): String?
    fun run(cmd: Array<String>): Int {
        return run(cmd.size, cmd)
    }

    init {
        System.loadLibrary("avutil")
        System.loadLibrary("avcodec")
        System.loadLibrary("swresample")
        System.loadLibrary("avformat")
        System.loadLibrary("swscale")
        System.loadLibrary("avfilter")
    }
}
Copy the code

You can then call the functions in the FFmpegCmd class.

// Use JNI as an example:
// 1. Write native methods in Java
// 2. Use javah-jni to generate a local interface
// 3. Implement the interface in C++ layer
// 4. Call native methods in the Java layer.
Copy the code

conclusion

Although Opencv and FFmpeg libraries have been successfully introduced, there are still many problems. Firstly, the installation package is too large after the introduction of library files, which needs to be optimized in the future. The second problem is the use of android JNI, because I use C++ programming, I am not very familiar with C++, I need to continue learning.

Before the successful introduction of Opencv and FFmpeg libraries, EVERY time I encountered errors, I was very anxious, because I thought I would be successful if I followed the steps of online tutorials. However, the constant errors made me want to hit the mouse. Fortunately, DURING the process of searching for information and learning progress, I accumulated experience and summarized the reasons for previous failures. Finally, I successfully introduced Opencv and FFmpeg libraries, so I wrote this article to record my successful experience. Summary of experience is that we must be patient to find information, and to understand the differences between the compilation or use of these open source library versions, if you encounter compilation error, you can directly copy the error information online to find solutions, step by step to the final success!

References:

AndroidStudio uses OpenCV in three ways

Android records compiling FFMPEG4.2.2 on macOS using NDK20

The Android project introduces OpenCV and FFmpeg

Opencv – 4.5.2

FFmpeg – 4.4 –

Build FFmpeg for Android

Android project introduces FFmpeg

conclusion

Related Posts

Kotlin – Common higher-order functions

Hot fixes and plugins

AOP is faceted programming oriented