The introduction
Fine-grained images, compared with general/ Generic images, are different and difficult in that their images belong to more fine-grained categories, which is also a popular direction in the field of computer vision at present, including classification, retrieval and image generation.
There are also more and more fine-grained image categories. In the beginning, around 2011, the baseline was flowers, birds (CUB200), and cats and dogs, and the largest dog dataset was around 20,000 images, but now there are more and more categories. It includes clothing, fruits and vegetables, retail goods and so on. With the development of deep learning and the proposal of various classification models, the classification accuracy of these data sets is getting higher and higher.
Today’s recommended website is about fine grained image analysis resources, written by Megvii nanjing research Institute director Wei Xiucan, the website address is as follows
www.weixiushen.com/project/Awe…
This site mainly introduces papers, code and data sets related to fine-grained image analysis. The papers are very new results, basically from 2017 to 2019.
The main contents are as follows:
They are introduced in 9 aspects:
- Introduction to the
- The tutorial
- Research papers
- Base data set
- Fine-grained image recognition
- Fine-grained image retrieval
- Fine grained image generation
- Future development direction
- Classification Accuracy list
1. Introduction
The main content of the site is to present some representative papers, code and data sets on fine-grained images based on deep learning, including classification, retrieval, image generation and so on.
2. The tutorial
Here are two recommended tutorials, both written by the author, respectively 2018 and 2019
- www.weixiushen.com/tutorial/PR…
- www.icme2019.org/conf_tutori…
3. Research papers
Also two copies:
- Deep Learning for Fine-Grained Image Analysis: A Survey. 2019
Address: arxiv.org/abs/1907.03…
- A Survey on Deep Learning-based Fine-Grained Object Classification and Semantic Segmentation. 2017
The paper addresses: link.springer.com/article/10….
4. Benchmark data set
Here 11 data sets are shown, as shown in the figure below, where BBox represents the boundary box information of the object provided by the data set, Part ANno represents the location information of the key parts in the data set, HRCHY represents hierarchical labels, ATR represents attribute labels (such as wing color, etc.), Texts represent text description information of pictures.
The 11 data sets here basically contain different categories of objects, such as birds, flowers, clothing, cats and dogs, fruits and vegetables, retail goods, etc. The most commonly used data set is CUB200, which contains 200 categories of birds and is included in many experimental data sets of papers.
5. Fine-grained image recognition
The methods of fine-grained image recognition are divided into the following three types:
- 163 Fine-grained by localization and classification subnetworks
- 163 Fine-grained by end-to end feature encoding
- 173 Adopt fine-grained recognition with external information, including network data, multi-mode data (text and audio), and manual operations
There are more papers for the first two methods described here, the oldest of which is from 2014, followed by the latest CVPR 2019, and code addresses are provided for those with open source code. Because there are too many papers, I will not list them one by one. I will list the latest paper separately
Based on location-classification network
Weakly Supervised Complementary Parts Models for fine-grained Image Classification From the Bottom Up. CVPR,2019. (In Chinese
The paper addresses: openaccess.thecvf.com/content_CVP…
End-to-end feature coding
173. Destruction and Construction Learning for Fine-grained Image Recognition, CVPR,2019
The paper addresses: openaccess.thecvf.com/content_CVP…
Github：github.com/JDAI-CV/DCL
Use additional information, including network data, multi-mode data (text, audio, etc.), add manual operation
163 Fine-grained Classification using Adversarial Discriminative Neural Networks from Web Data. AAAI,2019
Address: github.com/sxzrt/Learn…
6. Fine-grained image retrieval
Fine-grained search directions are divided into two directions
Unsupervised with Pre-trained Models
6. Selective Convolutional Descriptor Aggregation for fine-grained Image Retrieval.2017
The paper address: www.weixiushen.com/project/SCD…
Supervised with Metric Learning
- Centralized Ranking Loss with Weakly Supervised Localization for Fine-Grained Object Retrieval.IJCAI,2018
The paper address: www.ijcai.org/proceedings…
- Towards Optimal Fine Grained Retrieval via Decorrelated Centralized Loss with Normalize-Scale layer.AAAI,2019
The paper addresses: mac.xmu.edu.cn/rrji/papers…
7. Fine-grained image generation
Also divided into two directions:
Generating images from fine-grained image distribution
- CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training. ICCV,2017
The paper addresses: openaccess.thecvf.com/content_ICC…
Github:github.com/tkazusa/CVA…
- FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discovery. CVPR,2019
The paper addresses: openaccess.thecvf.com/content_CVP…
Github:github.com/kkanshul/fi…
Generating images from text Descriptions
173 AttnGAN: Fine-text to Image Generation with Attentional Generative Adversarial Networks.CVPR,2018
The paper addresses: openaccess.thecvf.com/content_cvp…
Github:github.com/taoxugit/At…
8. Future development direction
Two possible directions are listed:
163 Fine-grained few shot learning
173 173 Learning fine learners for Learning Categories with Few Examples. TIP, 2019
The paper address: www.weixiushen.com/publication…
2. FGIA within More realistic Settings
Here are the three papers presented:
- Fine-grained Recognition in the Wild: A Multi-Task Domain Adaptation Approach. ICCV,2017
Address: ai.stanford.edu/~tgebru/pap…
- The iNaturalist Species Classification and Detection Dataset. CVPR,2018
The paper addresses: openaccess.thecvf.com/content_cvp…
- RPC: A Large-Scale Retail Product Checkout Dataset. 2019
Address: arxiv.org/abs/1901.07…
9. Ranking of classification accuracy
The best current methods and whether they use standard information, additional data, network structure, size Settings of input images, and classification accuracy are listed below:
163 We have A collection of papers, code, and data sets for fine-grained Image Analysis, particularly A recent review paper called Deep Learning for fine-grained Image Analysis: A Survey.
Address: arxiv.org/abs/1907.03…
In addition, Mr. Wei Xiucan also published a review of fine-grained image analysis on Zhihu in 2017. For those who are interested, please check it out:
zhuanlan.zhihu.com/p/24738319
My didi Cloud exclusive AI master code: 9192, purchase Didi Cloud GPU and other AI products enter the master code to enjoy a 10% discount. Click www.didiyun.com to go to the official website of Didi Cloud to buy.
Welcome to follow my wechat official account — the growth of algorithmic ape, or scan the QR code below, we can communicate, learn and progress together!
