This is the sixth day of my participation in the First Challenge 2022. For details: First Challenge 2022.

This article will show you how to quickly reproduce a RepVGG using the MegEngine framework, make sure you know enough about at least one deep learning training framework like PyTorch.

Why should I reproduce the paper

Why reproduce the paper (using other frameworks)?

  1. The questions limit the framework, such as the competition sponsored by the major deep learning framework companies (prize money beckons), which has to be used;
  2. The tutor required the use of other framework to reproduce the paper, had to reproduce;
  3. White whao computing power, the framework of the company in order to promote their own framework will provide some online computing power support, such as Baidu AIStudio, MegStudio, Huawei ModelArts, etc.;
  4. In-depth understanding of the paper, improve programming ability, paper reappearance belt can improve the understanding of the paper and the degree of attention to the details, at the same time, the understanding of the framework source code and the improvement of programming ability also helps.

The preparatory work

When we decide to reproduce a paper quickly, what should we do?

  1. Data set/source code: is the data set/source code used in this paper open source or can be obtained from the author;
  2. Paper: read the paper at least once to know what the work is based on (reference previous work, such as ResNet for trunk, FPN for Neck, etc., which helps us to reproduce the paper more quickly and avoid building wheels), what innovations and contributions, etc.
  3. Source code: have a general look, understand the code structure, what are the recurrence difficulties (for example, the paper uses CUDA operator, which requires you to rewrite the compilation, build into the model, understand the difficulties help us to allocate priority, timely stop loss)
  4. Environment configuration: the environment required by the official code and reproduce the environment required to configure, each framework installation method in the official website have tutorials;
  5. Frameworks: Each framework has a quick start tutorial. Read through it and learn about simple apis like tensor creation, data type declarations (like torch’s use)torch.float, Paddle used'float32'MegEngine usenp.float32), base class of networking (nn.Moudle.nn.Layer.M.moudle), etc.
  6. Training Template: Each framework has official repositories such as ModelHub and ModelZoo, or some development kits such as PyTorch’s MMCV series (MMDetection, MMSegmentation, MMClassification, etc.), Paddledetection for Paddle, Paddleseg, etc., Basecls for Megengine. We need to find training codes for tasks and data sets similar to the regenerated paper (such as image classification and ImageNet) and have them ready for subsequent modification.
  7. API documentation: need to open source framework and target framework apis used by the document, the box is checked, it is important to maintain alignment, “alignment” is the key of the thesis retrieval, including model structure alignment, vector loss alignment, the optimizer alignment, alignment, training strategy alignment, etc, etc, of course, can choose according to their own.

get started

Now that you know what to do, let’s do it.

RepVGG (github.com/DingXiaoH/R…). And its data set ImageNet are open source, paper reading notes in the previous blog has been introduced, unfortunately, RepVGG did not use previous work, such as ResNet, etc. (otherwise you can directly CTRL C V), fortunately the source is relatively simple, the difficulty is not very big.

Training templates are also available in the Official Modles repository provided by MegEngine. This article will use resNet training code and modify it as needed.

Both the official documentation for PyTorch and the official documentation for MegEngine have been opened.

All set, the next blog post will cover the process of replicating the model.