Residual network
Residual Network (ResNet for short) was proposed in 2015 after Alexnet Googlenet VGG three classic CNN networks and won the first prize in ImageNet competition classification task. Because of its simple and practical advantages, ResNet has been widely used in detection, segmentation, recognition and other fields. ResNet is arguably the most groundbreaking work in the field of computer vision and deep learning in the past few years, effectively addressing the problem that the accuracy of training sets decreases as networks deepen, as shown in the figure below:
Conclusion:
ResNet solves the degradation problem of deep networks through residual learning, allowing us to train deeper networks, which can be called a historical breakthrough of deep networks. There may soon be better ways to train deeper networks, so let’s hope for that! Currently, you can find an example of a 34-layer Residual network (ResNet) implementation based on TensorFlow on the AI modeling platform Mo in the form of CIFAR-10 (CIFAR’s ten classification dataset), which has an accuracy of 90% on the test set and 98% on the verification set. The main program is in resnet_operator. py, the Block structure of the network is in resnet_block. py, and the trained model is saved in the Results folder. Project source code address: momodel.cn/explore/5d1… References: [1] K. He, X. Zhang, S. Ren, And J. Sun. Deep residual learning for image recognition. ArXiv Preprint arXiv:1512.03385,2015. L. Bottou, G. B. Orr, and K. -r. m. Uller. Efficient Backprop.In Neural Networks: Tricks of the Trade, Pages 9 — 50. Springer, 1998. [3] X. Glorot and Y. Bengio. Understanding the difficulty of training deep feedforward neural networks. In AISTATS, 2010. [4] A. M. Saxe, J. L. McClelland, And s. Ganguli. The Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. The arXiv: 1312.6120. 2013. [5] K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectifiers:Surpassing human-level performance on imagenet classification. In ICCV, 2015. [6] S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, 2015.
About us
Mo (momodel.cn) is a Python-enabled online modeling platform for artificial intelligence that helps you quickly develop, train, and deploy models.
Mo Artificial Intelligence Club is initiated by the website’s R&D and product design team, committed to lowering the threshold of artificial intelligence development and use of the club. The team has experience in big data processing and analysis, visualization and data modeling, has undertaken multi-field intelligent projects, and has full design and development capabilities from the bottom to the front end. His research interest is big data management and analysis and artificial intelligence technology, which can promote data-driven scientific research.
At present, the club holds offline technology salon with the theme of machine learning in Hangzhou every Saturday, and carries out paper sharing and academic exchange from time to time. We hope to gather friends from all walks of life who are interested in ARTIFICIAL intelligence, continue to communicate and grow together, and promote the democratization and popularization of artificial intelligence.
