Welcome to follow my public account [Jizhi Vision], reply 001 to get Google programming specification

O_o >_< O_o O_o ~_~ O_o

Hi, I’m jizhi, and this article introduces the method of deploying yolov3 and yolov4 using deepstream6.0.

Yolo series of target detection algorithms are widely used in engineering, especially starting from YOLOv3, gradually evolving to YOLOV4, YOLOV5, etc., the acceptance of engineering is getting higher and higher. Deepstream is a pipeline application designed by Nvidia to accelerate deep learning. When DeepStream meets YOLO, let’s take a look.

For the deepStream installation tutorial, you can check out my previous articles: installing Ubuntu DeepStream6.0 and Installing Ubuntu DeepStream5.1.

Deepstream6.0 source directory structure

  • apps
    • apps-common
    • audio_apps
    • Sample_apps: routines such as deepstream-app, deepstream-test1…
  • Gst-plugins: Gstreamer plugin
  • Include: the head
  • Library of libs:
  • ObjectDetector_FasterRCNN: The FasterRCNN example
  • ObjectDetector_SSD: SSD example
  • ObjectDetector_Yolo: YOLO example
  • Tools: logs are related

Deepstream6.0 deploys Yolov3

Through the above objectDetector_Yolo project to run yOLOv3, objectDetector_Yolo project is mainly concerned with the following modules:

  • Nvdsinfer_custom_impl_Yolo: YOLOv3 project implementation code;
    • Nvdsinfer_yolo_engine. CPP: parse the model and generate the engine
    • Nvdsparsebbox_yolo. CPP: Parsing function for the output layer, parsing the target detection box
    • Trt_utils.cpp and trt_utils.h: Interfaces and implementations of utility classes that construct TensorRT networks
    • Yolo. CPP and yolo. H: Interface and implementation to generate yOLO engine
    • CPP and yoloplugins. h: interfaces and implementations of YoloLayerV3 and YoloLayerV3PluginCreator
    • Cu: cudA kernel underlying implementation
  • Config_infer_xxx_txt: configuration of the model;
  • Deepstream_app_config_xxx. TXT: configuration file of the Gstreamer nvInfer plug-in;
  • Xxx.cfg, XXX. Weights: model file.

That’s all we need. Let’s start.

1.1 Downloading the model File

There is no yolov3 model file in deepstream6.0 SDK. You need to download it and send it to deepstream6.0 SDK.

Yolov3.cfg:github.com/pjreddie/da…

Yolov3.weights:link.zhihu.com/?target=htt…

If you have yolov3.engine for TensorRT, you don’t need the original model file. If you don’t have.engine, you will create.engine from the original file.

1.2 configuration config_infer_primary_yolov3. TXT

[property] gpu - id = 0 net - scale - factor = 0.0039215697906911373#0=RGB, 1=BGRmodel-color-format=0 custom-network-config=yolov3.cfg model-file=yolov3.weights labelfile-path=labels.txt Int8 - calib file = yolov3 - calibration. Table. Trt7.0## 0=FP32, 1=INT8, 2=FP16 modenetwork-mode=1 num-detected-classes=80 gie-unique-id=1 network-type=0 is-classifier=0 cluster-mode=2 maintain-aspect-ratio=1 parse-bbox-func-name=NvDsInferParseCustomYoloV3 custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so Engine - create - func - name = NvDsInferYoloCudaEngineGet [class - attrs - all] NMS - iou - threshold threshold = 0.3 = 0.7Copy the code

1.3 configuration deepstream_app_config_yolov3. TXT

[application] enable-perf-measurement=1 perf-measurement-interval-sec=5 [tiled-display] enable=1 rows=1 columns=1 width=1280 height=720 gpu-id=0 nvbuf-memory-type=0 [source0] enable=1 type=3 uri=file://.. /.. /samples/streams/sample_1080p_h264.mp4 num-sources=1 gpu-id=0 cudadec-memtype=0 [sink0] enable=1#Type - 1=FakeSink 2=EglSink 3=Filetype=2 sync=0 source-id=0 gpu-id=0 nvbuf-memory-type=0 [osd] enable=1 gpu-id=0 border-width=1 text-size=15 text-color=1; 1; 1; 1; The text - bg - color = 0.3; 0.3; 0.3; 1 font=Serif show-clock=0 clock-x-offset=800 clock-y-offset=820 clock-text-size=12 clock-color=1; 0; 0; 0 nvbuf-memory-type=0 [streammux] gpu-id=0 live-source=0 batch-size=1 batched-push-timeout=40000 width=1920 height=1080 enable-padding=0 nvbuf-memory-type=0 [primary-gie] enable=1 gpu-id=0#model-engine-file=model_b1_gpu0_int8.enginelabelfile-path=labels.txt batch-size=1 bbox-border-color0=1; 0; 0; 1 bbox-border-color1=0; 1; 1; 1 bbox-border-color2=0; 0; 1; 1 bbox-border-color3=0; 1; 0; 1 interval=2 gie-unique-id=1 nvbuf-memory-type=0 config-file=config_infer_primary_yoloV3.txt [tracker] enable=1 tracker-width=640 tracker-height=384 Ll - lib - file = / opt/nvidia/deepstream/deepstream - 6.0 / lib/libnvds_nvmultiobjecttracker. So ll - config file - =.. /.. /samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml gpu-id=0 enable-batch-process=1 enable-past-frame=1 display-tracking-id=1 [tests] file-loop=0Copy the code

1.4 Project Compilation

Into the/opt/nvidia/deepstream/deepstream – 6.0 / sources/objectDetector_Yolo:

CD/opt/nvidia/deepstream/deepstream - 6.0 / sources/objectDetector_YoloCopy the code

Execute the following two commands in sequence to compile and generate the.so file:

Export CUDA_VER=11.4 # Set the SAME CUDA version as the deviceCopy the code

Or in/opt/nvidia/deepstream/deepstream – 6.0 / sources/objectDetector_Yolo/nvdsinfer_custom_impl_Yolo/Makefile changes:

And then you compile it

make -C nvdsinfer_custom_impl_Yolo 
Copy the code

The libnvdsinfer_custom_impl_Yolo. So dynamic library file is generated.

1.5 perform

deepstream-app -c deepstream_app_config_yoloV3.txt  
Copy the code

The deployment of DeepStream6.0 Yolov3 is completed here.

Deepstream6.0 deploys YOLOV4

Here you deploy YoloV4 in a different way, calling TensorRT Engine directly instead of importing it from the original model.

2.1 Generate yoloV4.engine using darknet2onnx2TRT

Download yolov4 Darknet original weight, given Baidu net disk transmission:

https://pan.baidu.com/s/1dAGEW8cm-dqK14TbhhVetA     Extraction code:dm5b
Copy the code

Clone model conversion Project:

git clone https://github.com/Tianxiaomo/pytorch-YOLOv4.git Yolov42TRT
Copy the code

Start model transformation:

cd Yolov42TRT

# darknet2onnx
python demo_darknet2onnx.py ./cfg/yolov4.cfg ./cfg/yolov4.weights ./data/dog.jpg 1

# onnx2trt
trtexec --onnx=./yolov4_1_3_608_608_static.onnx --fp16 --saveEngine=./yolov4.engine --device=0
Copy the code

This generates yoloV4.engine.

2.2 DeepStream YOLOV4 Inference engineering configuration

Clone DeepStream YoloV4

git clone https://github.com/NVIDIA-AI-IOT/yolov4_deepstream.git

cd yolov4_deepstream/deepstream_yolov4
Copy the code

Configuration config_infer_primary_yoloV4. TXT:

[property] gpu - id = 0 net - scale - factor = 0.0039215697906911373#0=RGB, 1=BGR
model-color-format=0
model-engine-file=yolov4.engine
labelfile-path=labels.txt
batch-size=1
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
num-detected-classes=80
gie-unique-id=1
network-type=0
is-classifier=0
## 0=Group Rectangles, 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)cluster-mode=2 maintain-aspect-ratio=1 parse-bbox-func-name=NvDsInferParseCustomYoloV4 The custom - lib - path = nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo. So [class - attrs - all] NMS - iou - threshold = 0.6 The pre - cluster - threshold = 0.4Copy the code

Configuration deepstream_app_config_yoloV4. TXT:

[application] enable-perf-measurement=1 perf-measurement-interval-sec=5 [tiled-display] enable=0 rows=1 columns=1 width=1280 height=720 gpu-id=0 nvbuf-memory-type=0 [source0] enable=1 type=3 Uri = file: / opt/nvidia/deepstream/deepstream - 6.0 / samples/streams/sample_1080p_h264. Mp4 num - sources = 1 gpu - id = 0 cudadec-memtype=0 [sink0] enable=1#Type - 1=FakeSink 2=EglSink 3=Filetype=3 sync=0 source-id=0 gpu-id=0 nvbuf-memory-type=0 container=1 codec=1 output-file=yolov4.mp4 [osd] enable=1 gpu-id=0 border-width=1 text-size=12 text-color=1; 1; 1; 1; The text - bg - color = 0.3; 0.3; 0.3; 1 font=Serif show-clock=0 clock-x-offset=800 clock-y-offset=820 clock-text-size=12 clock-color=1; 0; 0; 0 nvbuf-memory-type=0 [streammux] gpu-id=0 live-source=0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 enable-padding=0 nvbuf-memory-type=0 [primary-gie] enable=1 gpu-id=0 model-engine-file=yolov4.engine labelfile-path=labels.txt batch-size=1 bbox-border-color0=1; 0; 0; 1 bbox-border-color1=0; 1; 1; 1 bbox-border-color2=0; 0; 1; 1 bbox-border-color3=0; 1; 0; 1 interval=0 gie-unique-id=1 nvbuf-memory-type=0 config-file=config_infer_primary_yoloV4.txt [tracker] enable=0 The tracker - width = 512 tracker - height = 320 ll - lib - file = / opt/nvidia/deepstream/deepstream - 5.0 / lib/libnvds_mot_klt. So (tests) file-loop=0Copy the code

Convert 2.1 yolov4 generated. The engine copy to/opt/nvidia/deepstream/deepstream – 6.0 / sources/yolov4_deepstream.

2.3 Project Compilation

Into the/opt/nvidia/deepstream/deepstream – 6.0 / sources/yolov4_deepstream:

CD/opt/nvidia/deepstream/deepstream - 6.0 / sources/yolov4_deepstreamCopy the code

Execute the following two commands in sequence to compile and generate the.so file:

Export CUDA_VER=11.4 # Set the SAME CUDA version as the deviceCopy the code

Or in/opt/nvidia/deepstream/deepstream – 6.0 / sources/yolov4_deepstream/nvdsinfer_custom_impl_Yolo/Makefile changes:

And then you compile it

make -C nvdsinfer_custom_impl_Yolo 
Copy the code

The libnvdsinfer_custom_impl_Yolo. So dynamic library file is generated.

2.4 perform

deepstream-app -c deepstream_app_config_yoloV4.txt  
Copy the code

The deployment of DeepStream6.0 Yolov4 is completed here.

Deepstream6.0 deploys yolov3 and yolov4 in deepstream6.0.

Deepstream6.0 Deploys Yolov3 and YoloV4