The OpenMMLab framework covers almost all aspects of deep learning visual tasks. For each specific task, we provide a corresponding algorithm library, such as MMClassification for task classification, MMDetection for any task, MMSegmentation for task segmentation and so on.
A subsequent question is, if I want to apply the research method to multiple tasks and evaluate it on multiple tasks, do I have to fork each library? Further, what if I want to open source my code that relies on the OpenMMLab framework?
Here, we introduce how to develop a new backbone network quickly, and use MMCLS, MMDET and MMSEG to easily evaluate the classification, detection and segmentation tasks.
To help you get started, we provide a sample repository: github.com/mzr1996/bac…
In the sample repository, we used ConvNeXt as an example to show the files needed to set up a backbone network and benchmark it. So, let’s use this example repository as an example to learn step by step how to build your own backbone network.
Configure the environment
First, we need to configure Python and the PyTorch environment, which I’m sure those of you who are familiar with PyTorch already have. Note that we require Python >= 3.6, PyTorch >= 1.5.0, and newer versions are of course fine.
In addition, we need the following OpenMMLab repositories:
- MIM: A command-line tool for managing OpenMMLab packages and experiments
- MMCV: OpenMMLab Computer vision foundation library
- MMClassification: OpenMMLab Image classification toolbox and benchmarks. In addition to sorting tasks, it is also used to provide a diverse backbone network
- MMDetection: OpenMMLab instrumentation toolkit and benchmarks
- MMSegmentation: OpenMMLab Semantic segmentation toolkit and benchmarks
Assuming you have the Python and PyTorch environments ready, you can configure the software environment with the following two commands.
pip install openmim mmcls mmdet mmsegmentation
mim install mmcv-full
Copy the code
Data preparation
Suppose we want to benchmark the classification task on ImageNet, the semantic segmentation task on ADE20K, and the detection task on COCO, then we need to organize the data files in the following format.
The data / ├ ─ ─ imagenet │ ├ ─ ─ "train" │ ├ ─ ─ val │ └ ─ ─ meta │ ├ ─ ─ "train". TXT │ └ ─ ─ val. TXT ├ ─ ─ ade │ └ ─ ─ ADEChallengeData2016 │ ├ ─ ─ annotations │ └ ─ ─ images └ ─ ─ coco ├ ─ ─ annotations │ ├ ─ ─ instance_train2017. Json │ └ ─ ─ instance_val2017. Json ├ ─ ─ Train2017 └ ─ ─ val2017Copy the code
Here, we list only the necessary files for training and validation. If you want to benchmark on more data sets or tasks, such as panorama segmentation using MMDetection, you can simply organize the corresponding data sets according to MMDetection’s needs. In addition, the MMSegmentation provides a tutorial on how to organize semantically segmented related data sets.
Implement your backbone network
With everything ready, let’s start implementing and training our backbone network step by step
The first is implementation:
1. Create your trunk network file in the Models folder. For example, models/convnext. Py in the sample repository. In this file, you only need to implement your trunk network using PyTorch, with the following two additional points to note:
- The backbone network and major modules need to be inherited
mmcv.runner.BaseModule
. thisBaseModule
是torch.nn.Module
A subclass of, whose behavior is almost identical, except that it supports additional useinit_cfg
Parameter specifies the initialization method including the pretrained model. - Use the following decorator structure to register your trunk network class to
mmcls.models.BACKBONES
Registry.
from mmcv.runner import BaseModule
from mmcls.models import BACKBONES
@BACKBONES.register_module(force=True)
class MyBackbone(BaseModule):
...
Copy the code
What is a registry? Look at this!
2.[Optional] If you want to add additional modules and components for a particular task, refer to models/det/layer_decay_optimizer_constructor.py to add them to the appropriate folder.
3. Add the trunk network and custom components you added to the models/__init__.py file.
Adding a Configuration File
If you’re not quite sure what configuration files are, this tutorial should help you.
In general, we use several base profiles to organize an experimental profile by inheritance. These base profiles typically include models, data sets, optimization policies, and operational configurations. You can also overwrite the base configuration file in the configuration file, or write it all into one configuration file without using the base configuration file.
In this sample library, we provide a set of common base configuration files. You can find more useful base configuration files by accessing the following links: MMClassification, MMDetection, MMSegmentation.
For each different task, the corresponding configuration files can be placed in different subfolders in the Configs directory. Note that in the Model section of the configuration file, in order to enable MMDetection and MMSegmentation to call the backbone network registered with MMCLs.Models.BACKBONES, We need to specify the location of the backbone network registry in additional configuration files for DET and SEG by adding MMCLS to Type. A prefix, such as:
model = dict( backbone=dict(type='mmcls.MyBackbone', ...) ,...).Copy the code
To learn more about cross-repository invocation modules, see “How to Open configuration files using MMDet Backbone?”
Training and testing
Relying on MIM, the unified experiment management tool provided by OpenMMLab, we can train and test different tasks without writing any Python scripts after we have implemented the model and written the configuration files.
First, we need to add the directory of the current repository to the PYTHONPATH environment variable, so Python can find our model file and run the following command in the repository root:
export PYTHONPATH=`pwd`:$PYTHONPATH
Copy the code
Single machine single GPU
Check MMCLS $CONFIG -- WORK_DIR $WORK_DIR -- check MMCLS $CONFIG -c $CHECKPOINT --metrics accuracy --metric-options "topk=(1, 5)" mim train mmdet $CONFIG --work-dir $WORK_DIR "mim test mmdet $CONFIG -c $CHECKPOINT --eval bbox mim train mmseg $CONFIG -- WORK_DIR $WORK_DIR # test mmseg $config-c $CHECKPOINT --eval mIoUCopy the code
Description of some parameters:
$CONFIG
:configs/
Configuration files in folders.$WORK_DIR
: folder for storing log and weight files$CHECKPOINT
: Weight file path
Single-node multiple Gpus (With four Gpus as an example)
Mim train MMCLS $CONFIG --work-dir $WORK_DIR --launcher Pytorch --gpus $CHECKPOINT --metrics accuracy --metric-options "topk=(1, Mim train mmdet $CONFIG --work-dir $WORK_DIR --launcher Pytorch --gpus 4)" --launcher Pytorch --gpus 4 Test mmdet $CONFIG -c $CHECKPOINT --eval bbox seGM --launcher Pytorch --gpus 4 Mmseg $CONFIG --work-dir $WORK_DIR --launcher Pytorch --gpus 4 # mim test mmseg $CONFIG -c $CHECKPOINT --eval mIoU --launcher pytorch --gpus 4Copy the code
conclusion
OpenMMLab has been dedicated to providing you with a toolbox for visual deep learning tasks. At the same time, the libraries for various tasks are organic and unified. This article introduces a practical way to use multiple libraries together. You are welcome to download and try out our sample repository. In addition, we will also be available at github.com/open-mmlab/… Examples of how to use MIM to accomplish various tasks are constantly updated