The original link: mp.weixin.qq.com/s/VWRiFAQGZ…

Today, I will share two Github projects, both of which use PyTorch to implement deep learning network model, mainly including some commonly used models, such as ResNet, DenseNet, ResNext, SENet, etc., and also give corresponding experimental results. It includes complete data processing and loading, model building, training process building, and test code implementation.

Let’s start with these two projects.

1. PyTorch Image Classification

This code currently has 200+ stars and mainly implements the following networks, conducting experiments onMNIST, CIFAR10, FashionMNIST and other datasets.

The usage method is as follows:

Then the author gives the experimental results of his own training, and the comparison with the experimental results of the original paper, including the differences in training Settings, and the number of iterations and training time are also given respectively.

Later, the author also studied the effects of residual unit, learning rate strategy and data enhancement on classification performance, such as

  • Pyramidnet-like residual units
  • Cosine sliding of learning Rate
  • Cutout
  • Random Erasing
  • Mixup
  • Preset activation of Shortcuts after downsampling

The experimental results show that:

  • A residual unit design similar to pyramid networks is helpful, but not suitable for Preactivation of Shortcuts after downsampling
  • Cosine learning rate decline strategy had a small increase
  • Cutout, random elimination, and mixups, which require more training, all work well

In addition to this experiment, the author continues to do several experiments, including batch size, initial learning rate size, label smoothing and other aspects, and gives the corresponding experimental results.

Finally, the thesis of these network models and different training strategies are given.

In addition to realizing the corresponding network model, this project can give us some inspiration by using different skills or studying the basic batch size and initial learning rate. Some skills can be applied to the network to improve the classification performance.

Links:

Github.com/hysts/pytor…

2. PyTorch Image Models

The code currently has 600+ stars and has been updated in the last few days with more network implementations, including DPN, Xception, InceptionResNetV2 and EfficientNet.

This project has the following features:

  • All models are encapsulated, that is, they have default configuration interfaces and apis, including a unified call classifier interfaceget_classifier,reset_classifierForward computing interface for featuresforward_features
  • All models have consistent pre-training model loaders, that is, it can decide whether to adopt the last layer of the pre-training model or whether the input layer needs to change from 3 channels to 1 channel;
  • The training script can be used in different modes, including distributed, single-machine multi-card, single-machine single-card, or single-CPU
  • Dynamically implement pooling layer operations, including average pooling, Max pooling, average + Max pooling, or connect average and maximum pooling results instead of stacking;
  • Implementation of different training strategies such as cosine learning rate, random elimination, tag smoothing, etc
  • Realize the Mixup
  • Provide a prediction script

The author also gives the experimental results of training, and then there is the use of the method, is also in the designated location to prepare data, can be used.

In addition, the author also provides a ToDo list, which will continue to improve the project.

Links:

Github.com/rwightman/p…

Welcome to follow my wechat official account — the growth of algorithmic ape, or scan the QR code below, we can communicate, learn and progress together!