Heart of the Machine Editor, participation: Heart of the Machine Editorial Department.

OpenCV 3.0 was released in June 2015. After three and a half years, OpenCV 4.0 has finally been released. So far, OpenCV has gone through nearly 18 years.

OpenCV is Intel’s open source cross-platform computer vision library. That is, it is a library containing a large number of visual apis, from image preprocessing to pre-trained model calls, and can handle mainstream visual tasks such as image recognition, target detection, image segmentation and pedestrian re-recognition. The most notable feature of OpenCV is that it provides a whole set of tools for the process, so we don’t need to know how each model works to build visual tasks from API to API.

OpenCV uses a BSD license and is therefore free for both research and commercial use. It has C++, Python, and Java interfaces and supports Windows, Linux, Mac OS, iOS, and Android. OpenCV aims to improve computing efficiency and focuses on real-time applications. It is written in optimized C/C++ and can take advantage of multi-core processing.

In addition, with the help of OpenCL, OpenCV can take advantage of the hardware acceleration of the underlying heterogeneous computing platform. Its GitHub page has an “Open_model_zoo” repository that contains a number of computer vision pretraining models and provides a way to download them to help speed up the development and production deployment process. OpenCV is widely used, with 47,000 users in the user community and about 14 million downloads.

OpenCV is the must-have toolkit for CV developers and researchers. Mask-rcnn and many other open source projects rely on this toolkit. It has been more than three years since the release of version 3.0, and OpenCV 4.0 final release has been released, which further improves the core interface and adds new features such as qr code detector and ONNX conversion format.

Important Updates:

  • OpenCV 4.0 is now a C++11 library that requires C++11 compatible compilers. You need at least version 3.5.1 of CMake.

  • Removed a large number of C apis in OpenCV 1.x.

  • Persistence in the Core module (for storing and loading structured data in XML, YAML, or JSON format) can be completely re-implemented in C++, so the C API here has been removed.

  • A new module, G-API, has been added as an efficient graph-based image processing process.

  • The DNN module includes an experimental Vulkan back end and supports an ONNX format network.

  • Implemented the popular Kinect Fusion algorithm, and optimized for CPU and GPU (OpenCL).

  • A TWO-DIMENSIONAL code detector and decoder are added to the ObjDetect module.

  • Move the efficient and high quality DIS Dense Optical Flow algorithm from Opencv_contrib to the video module.

In addition, OpenCV 4.0 supports the mask-RCNN model and has improved performance. Image processing operations can achieve a 15-30% speed increase.

OpenCV and deep learning

In the OpenCV 4.0 update, it enhances the DNN module and adds neural networks that support the ONNX interchange format, all of which show that OpenCV is serious about its relationship with deep learning. In fact, since OpenCV 3.1, it has included DNN modules that enable forward propagation of deep networks, which are typically pre-trained by deep learning frameworks such as Caffe. In OpenCV 3.3, the DNN module was moved from Opencv_contrib to the core code base and has been significantly accelerated.

More importantly, the DNN module in OpenCV contains no additional dependencies except libProtobuf, which is now included in OpenCV. Here are some of the frameworks currently supported by OpenCV:

  • Caffe

  • TensorFlow

  • Torch

  • Darknet

  • ONNX interchange format model

OpenCV currently supports deep learning hierarchy functions:

  • AbsVal

  • AveragePooling

  • BatchNormalization

  • Concatenation

  • Convolution (including dilated convolution)

  • Crop

  • Deconvolution, a.k.a. transposed convolution or full convolution

  • DetectionOutput (SSD-specific layer)

  • Dropout

  • Eltwise (+, *, max)

  • Flatten

  • FullyConnected

  • LRN

  • LSTM

  • MaxPooling

  • MaxUnpooling

  • MVN

  • NormalizeBBox (SSD-specific layer)

  • Padding

  • Permute

  • Power

  • PReLU (including ChannelPReLU with channel-specific slopes)

  • PriorBox (SSD-specific layer)

  • ReLU

  • RNN

  • Scale

  • Shift

  • Sigmoid

  • Slice

  • Softmax

  • Split

  • TanH

For high-performance neural network layers, the DNN module includes low-level acceleration libraries such as SSE, AVX, AVX2 and NEON, as well as a Halide backend in continuous optimization.

You can find the latest benchmark results here: github.com/opencv/open…

Median optimal time for forward propagation of a single image (in milliseconds, based on CPU on float32).

The following networks have been tested and proven to work:

  • AlexNet

  • GoogLeNet V1 (also called Inception-5h)

  • ResNet-34/50/…

  • SqueezeNet v1.1

  • Vgg-based FCN (Semantic Segmentation Network)

  • ENet (Lightweight Semantic Segmentation Network)

  • Vgg-based SSD (Target Detection Network)

  • Mobilenet-based SSD (Lightweight Target Detection Network)

OpenCV 4.0 release address: opencv.org/opencv-4-0-…