Video capture frame + optical flow based on CUDA9+OpenCV3

In the field of computer vision, video algorithm is an important part. Different from images, video requires multiple frames with temporal features, and also includes certain motion information, such as optical flow. During preprocessing, images and light flow in videos need to be extracted. Dense_flow, an open source project, has implemented this function and supports GPU operation.

In CUDA 9 and OpenCV 3, configure dense_flow project, advanced version grey-segment-Networks. Also, recommend the Benchmark project MMAction for video.

Reference:

dense_flow: https://github.com/yjxiong/dense_flow
temporal-segment-networks: https://github.com/yjxiong/temporal-segment-networks
mmaction: https://github.com/open-mmlab/mmaction
Copy the code

Compile the OpenCV

The steps for compiling OpenCV are as follows:

Download OpencV, download opencv_contrib;
Modify 2 cmake files and 1 HPP file;
Modify HDF5, FFMPEG, nonfree(optional);
Make, install OpencV;

OpenCV 3

CUDA 9 does not support Opencv2. x

OpenCV source code

Download the OpenCV source file and unzip it:

wget https://github.com/opencv/opencv/archive/3.1. 0.zip

unzip 3.1. 0.zip

cd opencv3.1. 0
Copy the code

opencv_contrib

In Opencv-3.1.0, download the opencv_contrib file and unpack it:

wget https://github.com/opencv/opencv_contrib/archive/3.1. 0.zip

unzip 3.1. 0.zip
Copy the code

The location is as follows:

The SURF or SIFT algorithm was contrived in opencv_contrib and needed to be compiled. In dense_flow, the SURF algorithm was called.

Error:

undefined reference to `cv::xfeatures2d::SURF::create(double, int, int, bool, bool)'
Copy the code

Modify the cmake file

CMake Error:

CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
CUDA_nppi_LIBRARY (ADVANCED)
Copy the code

The reason is that NPPI is obsolete and needs to be replaced with other CUDAs. Meanwhile, CUDA 2.0 is incompatible with the current version and needs to be deleted.

You need to modify findcuda. cmake and OpencVDETECtuda. cmake in the cmake folder, as well as common.hpp.

Modify findcuda. cmake file, replace 3 places:

replace

find_cuda_helper_libs(nppi)
Copy the code

for

  find_cuda_helper_libs(nppial)
  find_cuda_helper_libs(nppicc)
  find_cuda_helper_libs(nppicom)
  find_cuda_helper_libs(nppidei)
  find_cuda_helper_libs(nppif)
  find_cuda_helper_libs(nppig)
  find_cuda_helper_libs(nppim)
  find_cuda_helper_libs(nppist)
  find_cuda_helper_libs(nppisu)
  find_cuda_helper_libs(nppitc)
Copy the code

replace

set(CUDA_npp_LIBRARY "${CUDA_nppc_LIBRARY}; ${CUDA_nppi_LIBRARY}; ${CUDA_npps_LIBRARY}")
Copy the code

for

set(CUDA_npp_LIBRARY "${CUDA_nppc_LIBRARY}; ${CUDA_nppial_LIBRARY}; ${CUDA_nppicc_LIBRARY}; ${CUDA_nppicom_LIBRARY}; ${CUDA_nppidei_LIBRARY}; ${CUDA_nppif_LIBRARY}; ${CUDA_nppig_LIBRARY}; ${CUDA_nppim_LIBRARY}; ${CUDA_nppist_LIBRARY}; ${CUDA_nppisu_LIBRARY}; ${CUDA_nppitc_LIBRARY}; ${CUDA_npps_LIBRARY}")
Copy the code

replace

unset(CUDA_nppi_LIBRARY CACHE)
Copy the code

for

  unset(CUDA_nppial_LIBRARY CACHE)
  unset(CUDA_nppicc_LIBRARY CACHE)
  unset(CUDA_nppicom_LIBRARY CACHE)
  unset(CUDA_nppidei_LIBRARY CACHE)
  unset(CUDA_nppif_LIBRARY CACHE)
  unset(CUDA_nppig_LIBRARY CACHE)
  unset(CUDA_nppim_LIBRARY CACHE)
  unset(CUDA_nppist_LIBRARY CACHE)
  unset(CUDA_nppisu_LIBRARY CACHE)
  unset(CUDA_nppitc_LIBRARY CACHE)
Copy the code

Modify opencVdetectuda. Cmake file, delete 2:

Annotate “Fermi” and bring forward “Kepler”, i.e. remove the if branch of “Fermi”, mainly to remove CUDA version 2.0 compatibility.

  set(__cuda_arch_ptx "")
  if(CUDA_GENERATION STREQUAL "Fermi")
    set(__cuda_arch_bin "2.0")
  elseif(CUDA_GENERATION STREQUAL "Kepler")
    set(__cuda_arch_bin "3.0 3.5 3.7")
Copy the code

Modified to

  set(__cuda_arch_ptx "")
  if(CUDA_GENERATION STREQUAL "Kepler")
    set(__cuda_arch_bin "3.0 3.5 3.7")
Copy the code

If CUDA version is larger than 6.5, delete the compatibility of VERSION 2.0 as follows:

      elseif(${CUDA_VERSION} VERSION_GREATER "6.5")
        set(__cuda_arch_bin "3.0 3.5")
Copy the code

In opencv – 3.1.0 / modules/cudev/include/opencv2 cudev/common HPP header file, and add:

#include <cuda_fp16.h>
Copy the code

reference

hdf5 error

Error:

hdf5.hpp:40:18: fatal error: hdf5.h: No such file or directory
Copy the code

Modify the opencv – 3.1.0 / modules/python/common cmake file, the file header, add

find_package(HDF5)
include_directories(${HDF5_INCLUDE_DIRS})
Copy the code

reference

nonfree error

Error:

fatal error: opencv2/nonfree/nonfree.hpp: No such file or directory
Copy the code

Install libopencv-nonfree-dev:

sudo apt-get update
sudo add-apt-repository --yes ppa:xqms/opencv-nonfree
sudo apt-get update
sudo apt-get install libopencv-nonfree-dev
Copy the code

If not successful, change the ppa source:

sudo add-apt-repository --remove ppa:xqms/opencv-nonfree
sudo add-apt-repository --yes ppa:jeff250/opencv
sudo apt-get update
sudo apt-get install libopencv-dev
sudo apt-get install libopencv-nonfree-dev
Copy the code

Reference, reference

ffmpeg error

Error:

c->flags |= CODEC_FLAG_GLOBAL_HEADER
Copy the code

In opencv – 3.1.0 / modules/videoio/SRC/cap_ffmpeg_impl HPP, add:

#define AV_CODEC_FLAG_GLOBAL_HEADER (1 << 22)
#define CODEC_FLAG_GLOBAL_HEADER AV_CODEC_FLAG_GLOBAL_HEADER
#define AVFMT_RAWPICTURE 0x0020
Copy the code

reference

make

To execute the make operation, in OPENCV_EXTRA_MODULES_PATH, we need to introduce opencv_contrib:

make -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D OPENCV_EXTRA_MODULES_PATH=/data1/wcl/workspace/opencv3.1. 0/opencv_contrib3.1. 0/modules/ ..
Copy the code

Make 32:

make -j32  
Copy the code

Install, and import OpencV into the system environment.

sudo make install  
sudo /bin/bash -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/opencv.conf'
sudo ldconfig
Copy the code

Compile DenseFlow

reference

Install libzip – dev

apt-get install libzip-dev
Copy the code

Download dense_flow project and switch OpenCV 3.1 branch:

git clonehttp://github.com/yjxiong/dense_flow -- recursive git checkout remotes/origin/opencv 3.1Copy the code

Specify OpenCV_DIR, compile project:

mkdir build && cdBuild OpenCV_DIR = / opencv 3.1.0 / build cmake. J. -DCUDA_USE_STATIC_CUDA_RUNTIME=OFF make -j32Copy the code

After a successful build, in the Build folder:

869956383.mp4 Create TMP folder for test video.
Execute the command, noting that all Spaces are replaced with “=”, reference and reference.
In the TMP folder, generate video frames prefixed with image and x axis optical flow withflow_xPrefix, Y-axis optical flow toflow_yPrefix, and other argumentsreference.

./extract_gpu -f=980044841.mp4 -x=./tmp/flow_x -y=./tmp/flow_y -i=./tmp/image -b=20 -t=1 -d=0 -s=1 -o=dir
Copy the code

Error, indicating that the video cannot be opened. Replace the space with “=”.

FATAL [default] Check failed: [video_stream.isOpened()]
Copy the code

Test video:

Output result:

GPU Usage

OK, that’s all! Enjoy it!

Video capture frame + optical flow based on CUDA9+OpenCV3

Compile the OpenCV

OpenCV 3

OpenCV source code

opencv_contrib

Modify the cmake file

hdf5 error

nonfree error

ffmpeg error

make

Compile DenseFlow

Related Posts

How to start from RNN and understand LSTM step by step

“Voice Recognition Twen-ASR-One Development Notes” chapter 5 using the Twen-ASR-One serial port

Abstract: Bottom-up Abstractive Summarization