“This is the 25th day of my participation in the First Challenge 2022. For details: First Challenge 2022”

Please follow my official account [Jizhi Vision] for more notes to share

Hi, I’m Jizhi Vision. This article provides a detailed description of OpenVino performance testing.

1. Install OpenVino

For details, see [Model Reasoning] Configuring and Using OpenVino for Ubuntu.

2. Openvino model download and reasoning practice

You can refer to [Model Reasoning] OpenVino Reasoning Practice.

3. Openvino performance test posture

Here alexNet is used as an example.

3.1 Downloading alexnet model

# download alexnet model 
cd open_model_zoo/tools/downloader/ 

python downloader.py --name alexnet
Copy the code

3.2 Model Transformation

## transform model to openvino_IR 
# caffe_model 
#Input_shape changed in alexnet.prototxt 
python openvino/model-optimizer/mo.py --input_model ./alexnet.caffemodel --data_type FP32 --output_dir ./ 

# tf_modelPython openvino/model-optimizer/mo_tf.py --input_model./alexnet.pb --data_type FP32 --input_shape [1,224,224,3] --output_dir ./
# onnx_model 
python openvino/model-optimizer/mo_onnx.py --input_model ./alexnet.onnx --data_type FP32 --output_dir ./
Copy the code

Among them:

Data_type Optional FP32, FP16, half, float.

The following files are generated:

3.3 Performance Test

# benchmark test 
./openvino/bin/intel64/Release/benchmark_app -i image.png -m alexnet.xml -api sync -d CPU -b 1
Copy the code

Among them:

– API Supports sync (synchronous mode) and async (asynchronous mode);

-b indicates batchsize.

-D Target Optional CPU, MULTI:CPU, MULTI:CPU,GPU, MULTI:CPU,MYRIAD, MULTI:CPU,MYRIAD,GPU.

My CPU is two Intel Xeon E5-2630 with 40 CPU logic cores. I have tested model inference FP16 SYNC, FP16 Async, FP32 sync and FP32 async respectively. Sync is synchronous reasoning, with low latency/low throughput characteristics, delay * throughput =1000; Async is asynchronous reasoning with high latency and high throughput.

Here are some numbers:

Summary:

(1) Actually, my purpose is to compare openVino CPU with GPU TRT. In fact, the precision performance of my hardware configuration is more than 15 times worse than fp16.

(2) There are no yOLOv3 and YOLOV4 data attached here. The FP16 I actually tested is about 17 frames, which feels a bit slow


So, the above share openVino performance test situation, I hope my share can help you learn a little.


[Public Account transmission]

[Model Reasoning] OpenVino Performance Test