“This is the second day of my participation in the First Challenge 2022.

Welcome to my public account [Jizhi Vision]

Hi, I’m Jizhi Vision. This paper introduces the process of resnet50 quantization and off-line inference in Cambrian MLU.

It mainly includes Cambricon PyTorch environment building, RESnet 50 quantization and RESNet 50 offline reasoning. Resnet series is the standard model, and other models can also refer to this process.

1. Cambricon Pytorch environment configuration

Cambricon Pytorch – MLU (pybricon pytorch) pytorch- MLU (Pybricon PyTorch)

According to the process set up good Cambricon Pytorch environment, after confirm the environment is normal, open the ${PYTORCH_HOME} / build/lib/opt/Cambricon Pytorch/SRC/Pytorch/build/lib). Cambricon PyTorch was successfully compiled if libtorch_python.so and libtorch.

2. Resnet50 quantitative reasoning

2.1 Model quantification

cd $HOME/pytorch/tools
./quanti_model.sh
Copy the code

Quanti_model. sh:

Python quantification_tool.py -model_name resnet50 -data_scale 1.0 -mean 0.485 0.456 0.456 - STD 0.229 0.224 0.225 -resize 224 -crop 224 -used_images_num 5 -quantize_mode int8 -save_model_path /opt/cambricon/pytorch/models/int8 -save_name resnet50-19c8e357Copy the code
  • Parameter Description:
    • -model_name: indicates the model name.
    • -data_scale: maximum and minimum scale;
    • -mean: mean of quantization parameters;
    • -std: quantified parameter variance;
    • -resize: adjusts the image size during preprocessing.
    • -crop: crop the image size;
    • -used_images_num: number of images used in quantization.
    • -quantize_mode: quantization accuracy;
    • -save_model_path: save path after model quantization.
    • -save_name: indicates the quantized name of the model.

Take a look at the output of the terminal:

2.2 Switching to the Offline Model

Install python3 – tk

sudo apt install python3-tk
Copy the code

Perform offline model transformation:

cd $HOME/pytorch/tools

python genoff_test_resnet50.py
Copy the code

genoff_test_resnet5.py

import sys
import torch
import torch.nn as nn
from torchvision.models.resnet import resnet50

net = resnet50(pretrained=False, progress=True)
net.load_state_dict(torch.load('/opt/cambricon/pytorch/models/int8/resnet50-19c8e357.pth',map_location='cpu'),strict=False)
net.eval().float().mlu()
net.set_core_number(16)
example_mlu = torch.randn(16.3.224.224,dtype=torch.float).mlu()
net = torch.jit.trace(net,example_mlu, check_trace=False)
net(example_mlu)
net.save('/opt/cambricon/pytorch/models/resnet50_int8_offline'.True)
Copy the code

2.3 Offline Reasoning

Perform offline reasoning:

cd $HOME/pytorch/tools

./inference_resnet50.sh
Copy the code

Inference_resnet50.sh

/opt/cambricon/pytorch/src/pytorch/test/offline_examples/build/clas_offline_multicore/clas_offline_multicore -offlinemodel /opt/cambricon/pytorch/models/resnet50_int8_offline.cambricon -images file_list_for_release -labels synset_words.txt -simple_compile 2 -input_format 0
Copy the code
  • Parameter Description:
    • -offlinemodel: indicates the path of the offlinemodel.
    • -images: image list text file;
    • -labels: labels files.
    • -simple_compile: enables simple compilation. Cambricon Runtime automatically allocates optimal hardware and software resources according to offline model core_number and batCH_size parameters.
    • -input_format: specifies the channel sequence of images to be input during network operation. Images are preprocessed according to this parameter. Support RGB, RGBA and ARGB three kinds of input picture channel order. 0 for RGB, 1 for RGBA, 2 for ARGB;

The final inference result output is as follows:

The process of resNet50 INT8 quantization, conversion to off-line model and off-line inference of Cambricon Pytorch on Cambrian MLU is shared above, which can be used for reference for quantitative inference of other models in Cambrian.

【 model inference 】 quantitative and offline inference process of Cambrian MLU resnet50