preface

Although the raspberry PI of this generation has increased memory, the speed of reasoning is not real-time enough due to the limitation of main frequency. In this paper, we introduce the deployment of OpenVINO neural rod on Raspberry PI to further improve the reasoning speed of AI.

Intel’s Neural Compute Stick 2/NCS 2 is still the size of a USB flash drive, measuring 72.5×27×14 mm, and includes the latest Intel Movidius Myriad X VPU vision processor. Integrated with 16 SHAVE computing cores and dedicated deep neural network hardware accelerators, it can perform high-performance vision and AI reasoning operations with very low power consumption, and supports the development frameworks of TensorFlow and Caffe.

According to Intel, the PERFORMANCE of NCS 2 is significantly improved over previous Movidius sticks, including image classification performance is about five times better and object detection performance is about four times better.

Key features of NCS2:

  • Deep learning reasoning at the edge
  • Pre-trained models on Open Model Zoo
  • A library of functions and a pre-optimized kernel for faster launch to market
  • Supports heterogeneous execution across various computer vision accelerators (CPU, GPU, VPU, and FPGA) using a common API
  • Raspberry Pi hardware support

Install the OpenVINO tool suite

NCS2 supports raspberry PI as one of its features, and Intel has an official theme document that is quite user-friendly to deploy.

Docs.openvinotoolkit.org/cn/latest/_…

1. Download the installation package

The version I chose is the April 2020 version:

cd ~/Downloads/
sudo wget https://download.01.org/opencv/2020/openvinotoolkit/2020.4/l_openvino_toolkit_runtime_raspbian_p_2020.4.287.tgz
#Creating an installation folder
sudo mkdir -p /opt/intel/openvino
#Unzip fileSudo tar -xf l_openvino_toolkit_runtime_raspbian_P_2020.4.287. TGZ --strip 1 -c /opt/ Intel /openvinoCopy the code

2. Install external software dependencies

You’ve already installed cmake, so skip this step.

sudo apt install cmake
Copy the code

3. Set environment variables

source /opt/intel/openvino/bin/setupvars.sh
#Permanently set environment variables
echo "source /opt/intel/openvino/bin/setupvars.sh" >> ~/.bashrc
Copy the code

Now every time you open a new command line terminal. The following information will appear:

[setupvars.sh] OpenVINO environment initialized

4. Add USB rules

Add the current Linux user to the Users group: Log out and log in for the Settings to take effect.

sudo usermod -a -G users "$(whoami)"
Copy the code

Installing USB Rules

sh /opt/intel/openvino/install_dependencies/install_NCS_udev_rules.sh
Copy the code

5. Insert the USB into the Intel nerve rod NCS2

Re-insert NCS2 and prepare to run the program.

Build object detection samples

1. Create a compilation directory

mkdir openvino && cd openvino
mkdir build && cd build
Copy the code

2. Build object detection samples

cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-march=armv7-a" /opt/intel/openvino/deployment_tools/inference_engine/samples/cpp
make -j2 object_detection_sample_ssd
Copy the code

3. Download weight files, network topology files, and test pictures

To download the.bin weighted file:

wget --no-check-certificate https://download.01.org/opencv/2020/openvinotoolkit/2020.1/open_model_zoo/models_bin/1/face-detection-adas-0001/FP16/fac e-detection-adas-0001.binCopy the code

To download a file with network topology.xml:

wget --no-check-certificate https://download.01.org/opencv/2020/openvinotoolkit/2020.1/open_model_zoo/models_bin/1/face-detection-adas-0001/FP16/fac e-detection-adas-0001.xmlCopy the code

Search for some images containing human faces as samples to be detected and save them to the ~/Downloads/image directory.

4. Run the program

-m specifies the model topology.xml file. The program automatically looks for the weight file with the same name.

-d MYRIAD stands for neural rod as reasoning device;

-i specifies the path of the image to be tested.

./armv7l/Release/object_detection_sample_ssd -m face-detection-adas-0001.xml -d MYRIAD -i ~/Downloads/image
Copy the code

Build the performance test program

1. Build the test program

make -j2 benchmark_app
Copy the code

2. Build object detection samples

Where -i is the input detected picture;

-m is the input model parameter;

-niter indicates the number of iterations for running inference.

. / armv7l/Release/benchmark_app -i car. PNG - m squeezenet1.1 / FP16 / squeezenet1.1 XML - PC - d MYRIAD niter - 1000Copy the code

Raspberry PI + nerve stick reasoning at 280 FPS. That’s fast enough. Try sticking a nerve stick in your computer to see how it compares.

3. Compare performance

Still around 280 frames, running speed is the same, so it can be seen that the computing bottlenecks are concentrated in NCS2, and there is little difference between the main device using a computer or a raspberry PI. In the case of nerve rods, a PC seems a bit wasteful.

Compare an OpenVino acceleration model running directly on a laptop’s Intel CPU:

340 FPS, the CPU is better.

The development process of raspberry PI

  • Select the pre-training model;
  • Use the model optimizer to transform the model;
  • Finally, reasoning models on raspberry PI.

The normal development approach requires finding a suitable model in open Model Zoo that meets basic needs for most businesses. If you need to run some more cutting-edge models or self-designed neural networks, then all kinds of model conversion methods are necessary skills, and the difficulty will be larger.

Data download

This issue related documents, you can reply: “RPI08” in the background of the public account, get the download link.

Next up

We’ll do some model transformation work and have YoloV5 run on raspberry PI with Openvino… Stay tuned for…