Extra! Extra!

AIZOO is now open to face detection models and code from PyTorch, TensorFlow, MXNet, Keras and Caffe.

First, attach the Github link to respect.

Github.com/AIZOOTech/F…

Two days ago, AIZOO opened source data and models of nearly 8,000 face masks, which received a lot of positive feedback from friends, and many friends left comments praising our open source spirit. One of his fans said, “You are so selfless to disclose so much data.” In a word, yuanfeng’s pain of labeling data disappeared.

However, I open source Keras and Caffe, and more friends send me messages telling me that I would love to open PyTorch, which is the most popular deep learning framework in the research community today. There are fans friends private message peak to TensorFlow version. In response, Yuanfeng asked the star Fan Wei to answer for me, please watch the following video.

For the basic explanation of face mask detection model and data, because the introduction has been made in the previous article, there is no additional description here. If you need it, please read the article “AIZOO open source face mask detection data + model + code + online web experience, all open source” published by AIZOO on Wednesday. In this paper, we focus on the conversion methods among various deep learning frameworks, especially how the target detection models are transferred to each other.

1

Introduction to model transformation tools \

It’s 2020, and deep learning has been booming for eight years. In the past eight years, deep learning has been the brightest star in the field of artificial intelligence. In the past decade or so, there have been numerous deep learning frameworks — Torch, Theao, Caffe, TensorFlow, Keras, MXNet, PyTorch, CNTK, Chainer, Deepleaning4Java, Baidu’s PaddlePaddle, etc. It can be said that these excellent frameworks have greatly facilitated academic research and industrial implementation by scholars and engineers, and have greatly facilitated the development of the FIELD of AI, for which we owe a big thank you to the authors of these frameworks.

However, after several years of development and precipitation, the head aggregation effect of deep learning framework becomes more and more obvious. Theano and Caffe stopped official updates after their 1.0 release, and Chainer’s parent company announced its switch to PyTorch in December 2019. MXNet, despite its presence in the world of Li And Amazon, has been a lukewarm success. Recently, Baidu’s Paddle is still struggling doggedly and promoting crazily, with a posture of “help me up, I can still fight”. TensorFlow, which had been in the ascendant for a while, was losing popularity in its own right (mainly because the 1.x API was too messy and there was only one conv2D with different parameter names and styles in Slim, tf.layers, tf.contrib, tf.nn). Of course, because TensoFlow has good support for Linux, Windows, Android, iOS, embedded and JavaScript, it is a very widely used framework in the industry. The algorithms in our official website AIZOO.com are all made by tensorflow.js.

There is a growing tendency in academia to use the little fresh PyTorch. This is why, after Yuanfeng opened Keras and Caffe’s models, many academic friends asked me for PyTorch models. One of the main reasons why PyTorch and TensorFlow are competing is that the basic components of deep learning, such as convolution, pooling, BatchNorm, etc., have been established. Each framework provides basically the same functionality. In this case, it is a matter of who has the best code style and usability. Who wins more easily. The basic function of the same component brings a benefit that we can use one framework after training, can be relatively easy to transfer to another framework model. Yuanfeng is a kerAS-trained model, but can be converted to Caffe, TensorFlow, PyTorch, and MXNet models without error.

Model transformation, in fact, is essentially to read topology and weight from the original model, use the target framework to build the same topology, and assign weights to the newly constructed network, so that the model built using the target framework can have the same output value as the original model. \

A great model transformation tool is MMdnn from Microsoft Research Asia, which basically converts all the different frameworks into a topology and weight file called Intermediate and Representation (IR). This intermediate representation is then used to generate a model of the target framework. Its basic principle is shown in the figure below: \

For the common classification and segmentation network, under normal circumstances, using a command, you can use MMdnn can be very convenient model transformation. Here’s an example, which is pretty simple. \

pip install mmdnn Convert TensorFlow to PyTorch mmconvert-sf tensorflow-in imagenet_resnet_v2_152.ckpt. Meta -iw imagenet_resnet_v2_152.ckpt --dstNodeName MMdnn_Output -df pytorch -om tf_resnet_to_pth.pt
Copy the code

2

Experience in target detection model transformation

For the common model, MMdnn can be very convenient conversion, but, if so simple, the author would not have been able to open source five framework model. In this, design comes to a tricky place. \

For TensorFlow and Keras, the default data order is NHWC (batch, height, Width, Channel), but for Caffe, PyTorch, and MXNet, it is NCHW. The channel is in the second dimension. MMdnn certainly knows this, so for ordinary models, it is ok to turn without worry. However, attention should be paid to the model of SSD and Yolo series including Anchor Free.

Familiar with the friends all know that target detection for the target detection of a location classification layer, assuming that the figure is 4 x4 size, total need to identify two classes of words (assuming that need to detect the cats and dogs, the following chart), and there are three groups of anchor on each characteristic figure, so position classification layer classification of part of the convolution output channel number, for 2 x3 = 6 channels, Then, using TensorFlow and Keras, the output matrix of this classification layer is [N, 4, 4, 6], and the front N is batch size. For the other three frameworks, the channel is in the second dimension, that is, [N,6, 4, 4].

For frameworks such as SSD and YoloV3, where there are multiple positioning layers, we typically concatenate them within the model. 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 The new shape is 0 0 Expanding the second and third dimensions into one dimension and splitting the last dimension into groups of two (the number of categories) for example [N, 8, 8, 6] 0 for this pattern [N, 4, 4, 6], 0 is the matrix 0 [N, 48,2]. In this way, two matrices [N, 192, 2] and [N, 48,2] can be spliced into a matrix [N, 240, 2] through the second dimension.

0 0 Using TensorFlow and The Keras framework, for multiple location-based classification tiers, only 0 is needed, and then Concatanate is 0 However, for the other three big frameworks, because they channel in the second dimension, in order to get a consistent result, it’s 0 0 in general that the channel is 0 0, 0, 0, 0, 0, 0, 0, 0

However, MMdnn does not! No! To know! Tao! 0 0 Needs to do a new transpose operation for the 0 0 it’s still using the (N, -1, 2) 0 parameter when it converts the 0 0 model

This is something that needs special attention when using the target detection model with TensorFlow, Keras to PyTorch, Cafffe and other frameworks with channels in front.

Below I write Keras to turn each frame sad process, after watching, friends please help me forward the moments of friends or click to see again. (The public number just opened, the number of attention is very few, sorry I a bitter tears!)

Turn Keras TensorFlow

Since the back end of Keras is generally TensorFlow, Keras conversion to TensorFlow is easy. \

After loading the Keras model, you can fetch the session, then read the graph structure from the session, save the graph structure as TensorFlow, and you are done. The core code is as follows: \

import tensorflow as tffrom keras import backend as Kkeras_model = load_keras_model(keras_model_path)sess = K.get_session()from tensorflow.python.framework import graph_utilfrom tensorflow.python.framework import graph_ioconstant_graph = graph_util.convert_variables_to_constants(sess, sess.graph.as_graph_def(), ["cls_branch_concat"."loc_branch_concat"])graph_io.write_graph(constant_graph, '. ' , graph_output_name, as_text=False)print('saved the constant graph (ready for inference) at: ', osp.join('. ', graph_output_name))
Copy the code

* * turn Keras Caffe * *

Because of the channel mentioned above, MMdnn conversion was not successful, so Yuan Feng used a tool written by himself to transform into Caffe. The principle is to read the topology of Keras model, build Caffe’s layer topology, and then assign weights to the new network. This code, when sorted out, will be open source (mainly because the code is now not elegant, for obsessive-compulsive people can not do open source ugly code).

* * turn Keras PyTorch * *

   \

Using MMdnn, it is possible to generate PyTorch’s structure code files and weights. However, remember the channel problem mentioned earlier, I manually added the transpose operation in the five location classification layers where it was needed.

Take the first group of positioning classification layers as an example, please pay attention to the operation of Permute, which is also referred to as transpose in this paper. Specific can see our open source code, there is a complete network building code.

Cls_0_0 0 = torch. 0 (input = cls_0_conv.permute(0,2,3,1) Shape = (CLS_0_conv. size(0),-1,2) loc_0_0 0 = 0 0 shape = (CLS_0_conv. size(0),-1) Shape = (loc_0_conv. Size (0), 1, 4))Copy the code

* * turn Keras MXNet * *

This is the one that Yuanfeng took the longest, more than a day, because I don’t know much about MXNet either. I have tried various methods to convert to MXNet using Keras, PyTorch, TensorFlow, and Caffe without success. \

In the middle, I wanted to give up countless times, after all, MXNet users are not large. However, because I have always felt that only the collection of five mainstream framework, can summon the Dragon, if there are only four, how can I summon, I am sorry to use MXNet friends ah; In addition, I always think the book hands-on Learning deep Learning written by Li Mu’s team is very good (please pay advertising fee to Yuanfeng privately). So, I forced myself to implement the MXNet model.

How do you do that? In fact, the author simply learned MXNet and implemented the same structure as Caffe’s model using the Gluon interface of MXNet. Then, he wrote a function that assigned weights to the MXNet model. Caffe and the MXNet model produced exactly the same output, thank goodness. \

At this point, Yuanfeng has finally collected the models of five mainstream deep learning frameworks and can summon the Divine Dragon.

But joking aside, it is still very hard to collect the five frames of the model, so Yuan Feng please help forward this article to the AI communication group, or friends circle, or give me a point to see again. After all, just determined to start a business, encountered this epidemic environment, is really down the bad luck. The gate can not go out, then let yuan Feng AIZOO public number more accumulation of some fans.

Finally, Yuan Feng thanks fans again.

3

Open source data acquisition

Finally, it is our open source access to relevant information. \

  • Data download address (reply “mask data set” in the background of the public account is also ok) \

Baidu web disk link: pan.baidu.com/s/1nsQf_Py5…

Extraction code: EYFZ

  • Open source deep learning models and code links

Github.com/AIZOOTech/F…

  • Online Experience Link

    Aizoo.com/face-mask-d…

Being is the END being

“`php

Highlights of past For beginners entry route of artificial intelligence and data download AI based machine learning online manual deep learning online manual download update (PDF to 25 sets) note: WeChat group or qq group to join this site, please reply “add group” to get a sale standing knowledge star coupons, please reply “planet” knowledge like articles, point in watching

Copy the code