Abstract: This practice realizes cat and dog image classification based on mobilenetV2, through data set acquisition and processing, pre-training model fine-tuning and migration, end-to-end deployment and reasoning and other links and knowledge points, experienced the simple development experience of MindSpore and the charm of full scene rapid deployment.

  • StartTime: 00:43:22, January 23, 2021
  • EndTime: January 23, 2021 11:34:44

(including study, sleeping, eating, pit climbing, hydrology… Time)

Learn about the MindSpore open source ecosystem

Discovered a little secret. Github has three more repositories. What are they?

In fact, Mindspaore-AI has 15 warehouses, three more than Gitee:

Mindspore-21-days-tutorials, mail_templates, infrastructure

Look at these three humble warehouses, but they have done a lot in community construction. “Mindore-21-days-tutorials,” for example, were the reference code and tutorials on our previous 21-day camp. What a valuable learning resource. The other two are MindSpore’s open source infrastructure, which contains all the necessary Dockerfiles and YAML files to configure the MindSpore community, Github Action synchronizes Gitee code to Github periodically and automatically.

Training Duration screenshot

Mobile phone identification screenshot

Learn to summarize

Before LEARNING it, I thought I could master it, but it was still rough in practice. All the codes were familiar, which were composed of 26 letters and some symbols. However, when I really tried to understand them, I still found that I didn’t have enough basic skills: first, I didn’t know how to write, and second, I didn’t know why TO write in this way. Nevertheless, I sorted out the whole practice process: training the cat and dog image classification model (cloud) –> mobile inference and application –> Starting from “1” Fine Tune model (local) –> mobile verification. Of course, as a student with poor grades, the whole operation process must have lasted more than an hour. Yes, brain: learning is useless!” Therefore, it is necessary to record the process of “learning failure” :

purpose

This practice realized cat and dog image classification based on mobilenetV2, through data set acquisition and processing, pre-training model fine-tuning and migration, end-to-end deployment and reasoning and other links and knowledge points, and experienced the simple development experience of MindSpore and the charm of full scene rapid deployment.

Project directory

MindSporePetClassification ├ ─ the ADB / / support mobile phones and computers transmit documents tools │ ├ ─ ADB. Exe │ ├ ─ AdbWinApi. DLL │ ├ ─ AdbWinUsbApi. DLL │ └ ─ Fastboot. Exe ├ ─ code / / Fine most cerebral sci-film training code and data set │ ├ ─ the dataset │ │ ├ ─ PetImages │ │ │ ├ ─ eval │ │ │ │ ├ ─ Cat │ │ │ │ └ ─ Dog │ │ │ ├ ─ "train" │ │ │ │ ├ ─ Cat │ │ │ │ └ ─ Dog │ │ ├ ─ MSR - LA - 3467. Docx │ │ └ ─ the readme [1]. TXT │ ├ ─ SRC │ │ ├ ─ __pycache__ │ │ │ ├ ─ the args. Retaining to 37. Pyc │ │ │ ├ ─ config. Retaining - 37. Pyc │ │ │ ├ ─ the dataset, retaining - 37. Pyc │ │ │ ├ ─ │ ├─ ├─ Exercises - ├─ exercises. ├─ ├─ exercises Pyc │ │ ├─ ├─ config.py │ │ ├─ dataset. Py │ │ ├─ Lr_generator ├ ─ models. Py │ │ └ ─ utils. Py │ ├ ─ mobilenetV2. Training models of CKPT / / file │ ├ ─ preprocessing_dataset. Py / / data collection scripts │ preconditioned └ ─ train. Py │ ├─ ├─ LibGlog. DLL │ ├─ LibGlog. DLL │ ├─ LibGlog. DLL │ ├─ LibGlog ├ ─ libmindspore_gvar. DLL │ ├ ─ libssp - 0. DLL │ ├ ─ libstdc++ - 6. DLL │ └ ─ libwinpthread - 1. DLL └ ─ kagglecatsanddogs_3367a. ZipCopy the code

Depend on the installation

This exercise relies on Opencv-Python and Matplotlib, one for processing graphics such as printing images and embedding text, and one for presenting data sets as visual images.

pip install opencv-python -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install matplotlib -i https://pypi.tuna.tsinghua.edu.cn/simple
Copy the code

Data preprocessing

Mainly remove images that do not meet the requirements (such as non-JPEG format) and split the train and eval datasets. Default 9:1:

Python preprocessing_dataset. Py Your path \kagglecatsaData to dragnddogs_3367a.zipCopy the code

Fine tune

Execute the Fine Tune script train.py and generate the model file:

python train.py
Copy the code

At this time, 6 pictures will pop up, because the script will extract 6 pictures from the data set before formal training to load the current model file, which needs to be closed manually to continue. This is because the AI is stupid and identifies all of them as Dog.

Of course, because of the machine, I did not step on the pits, this is not “before the beginning of the body die”, suddenly error to my whole, num_parallel_workers parameter set to 4, continue training!

$ python train.py start cache feature! Traceback (most recent call last): File "train.py", line 52, in <module> data, step_size = extract_features(backbone_net, args_opt.dataset_path, config) File "F:\dosomethings\serverless\ModelArts\MIndSpore\MindSporePetClassification\MindSporePetClassification\code\src\dataset.p y", line 84, in extract_features train_dataset = create_dataset(dataset_path=os.path.join(dataset_path, "train"), do_train=True, config=config) File "F:\dosomethings\serverless\ModelArts\MIndSpore\MindSporePetClassification\MindSporePetClassification\code\src\dataset.p y", line 41, in create_dataset ds = de.ImageFolderDataset(dataset_path, num_parallel_workers=8, shuffle=True) File "C:\Users\huqi\AppData\Local\Programs\Python\Python37\lib\site-packages\mindspore\dataset\engine\validators.py", line 51, in new_method validate_dataset_param_value(nreq_param_int, param_dict, int) File "C:\Users\huqi\AppData\Local\Programs\Python\Python37\lib\site-packages\mindspore\dataset\core\validator_helpers.py", line 352, in validate_dataset_param_value check_num_parallel_workers(param_dict.get(param_name)) File "C:\Users\huqi\AppData\Local\Programs\Python\Python37\lib\site-packages\mindspore\dataset\core\validator_helpers.py", line 340, in check_num_parallel_workers raise ValueError("num_parallel_workers exceeds the boundary between 1 and {}!" .format(cpu_count())) ValueError: num_parallel_workers exceeds the boundary between 1 and 4!Copy the code

This step time depends on the performance of the local machine, my coarse grain slag machine probably ran out of service. During the training, I learned some codes. Py and code\ SRC \config.py, and the loading of data is handled in code\ SRC \dataset. Code \ SRC \ mobilenetv2. py defines the model. Code \ SRC \models.py reads and saves the model and prints the training log.

Once the training is complete, the predict_FROm_net method is called to display the predict_from_net image and tag. When I confidently clicked close, expecting the application to immediately give me mobilenetv2.mindir, I was deceived again and returned a string of error logs!!

early stop! The best epoch is 2 train total cost 4141.2663 s Traceback (most recent call last): File "train.py", line 81, in <module> export_mindir(net, "mobilenetv2") File "F:\dosomethings\serverless\ModelArts\MIndSpore\MindSporePetClassification\MindSporePetClassification\code\src\utils.py" , line 93, in export_mindir export(net, Tensor(input_np), file_name=path, file_format='MINDIR') File "C:\Users\huqi\AppData\Local\Programs\Python\Python37\lib\site-packages\mindspore\train\serialization.py", line 535, in export Validator.check_file_name_by_regular(file_name) File "C:\Users\huqi\AppData\Local\Programs\Python\Python37\lib\site-packages\mindspore\_checkparam.py", line 438, in check_file_name_by_regular target, prim_name, reg, flag)) ValueError: 'F:\dosomethings\serverless\ModelArts\MIndSpore\MindSporePetClassification\MindSporePetClassification\code\mobilenetv2.m indir' is illegal, it should be match regular'^[0-9a-zA-Z\_\-\.\/\\]+$' by flags'256'Copy the code

At first I thought the file hierarchy was too deep, so I moved the file directory to the root directory of the disk and started again! Fortunately, there is no need to load the data set again, or there will be a long wait. Full of confidence results were “pa pa pa” face, after the group to consult again, Wang Hui teacher suggested that I change the path to the file name to try again, sure enough, immediately effective ~

- path = os.path.abspath(f"{name}.mindir")
+ path = name
Copy the code

Finally, code\ Mobilenetv2. mindir here she is!

Mobile inference and application

  • Training model conversion Converts.mindir model files into.ms files, which can be imported into end-to-end devices and trained based on the MindSpore end-to-end framework.

    F:\MindSporePetClassification\converter>call converter_lite –fmk=MINDIR –modelFile=f:\MindSporePetClassification\code\mobilenetv2.mindir –outputFile=pet

We can download the Android APP source code provided by MindSpore: gitee.com/mindspore/m…

Or download package good APP install directly to the mobile: download. Mindspore. Cn/model_zoo/o…

First experience the recognition effect of the pre-training model:

Then we moved to the good models of the mobile phone side/sdcard/PetClassification, used here is the ADB tool: the need to ensure that mobile phone has been open developer mode and open the file transfer

F:\MindSporePetClassification\converter>adb push f:\MindSporePetClassification\converter\pet.ms /sdcard/PetClassification * daemon not running; starting now at tcp:5037 * daemon started successfully f:\MindSporePetClassification\converter\pet.ms: 2. To bounce lightly or lightly over. 2. To bounce lightly or lightly over.Copy the code

Try the recognition effect again:

Interested in this practice APP side code partners can go directly to read the source: gitee.com/mindspore/m…

This article is shared by Huawei cloud community “Copy Attack lion 1 Hour introduction TO AI Development engineer”, originally written by: Hu Qi.

Click to follow, the first time to learn about Huawei cloud fresh technology ~