Jd.com AI Research Institute recently released a PyTorch based universal Object Reidentification (ReID) open source library, FastReID, which promotes research in related fields and accelerates the application of technical products. FastReID has achieved excellent evaluation results in pedestrian re-recognition and vehicle re-recognition. Within JD.com, this technology has been widely applied in practical projects such as smart parks, smart buildings, smart supply chains and offline retail. The open source library is described in detail in the just published paper FastReID: A Pytorch Toolbox for General Instance Re-Identification.
Links to papers: arxiv.org/pdf/2006.02… GitHub link: github.com/JDAI-CV/fas…
Whether it is the “unmanned supermarket” in the retail scene, the smart park in the supply chain scene, or the mask wearing in the epidemic situation… In the case of “intentional occlusion”, the difficulty of “face recognition” will face no small challenge, and its good recognition rate in “cooperation” mode will be difficult to play a value.
Therefore, the value of universal object reidentification will become more and more important. In order to meet the growing application demand of universal instance reidentification, JD AI Research Institute recently released the Open source library FastReID based on PyTorch. In FastReID, The highly modular and extensible design makes it easy for developers to implement new research ideas. Its release plays a positive role in promoting the research in related fields, and also speeds up the application of technical products.
Before we know what FastReID is, let’s take a look at what pedestrian re-identification is (ReID).
ReID stands for re-identification, which means re-identification. Simple understanding is for a specific object (pedestrians, vehicles, human faces or other specific objects), it is retrieved in the candidate image set. Or instance level retrieval of objects in an image.
ReID algorithm can retrieve pedestrians under the condition of cross-camera through video surveillance system, greatly expanding the utilization depth of camera resources. ReID could compensate for the limitations of face recognition, improve the ability to identify and track specific people, and greatly reduce labor costs.
FastReID is highly modular in design, it has higher flexibility and scalability, can support multi-GPU training, its scalability design allows it to quickly implement many research projects in the case of refactoring code.
Considering that there is often a big gap between academic research and engineering model deployment in practice, the code alignment problem makes it difficult to quickly transform academic research results into products. Therefore, in order to accelerate the development of reidentification technology by academic researchers and industrial engineers, we have launched the FastReID code open source library. FastReID’s highly modular and extensible design makes it easier for researchers and engineers to embed their own modules and implement tasks quickly. In addition, the manageable system configuration makes its application more flexible and easy to accomplish a variety of tasks. FastReID provides a fair comparison between different approaches. The open source library has become the core engine of ReID technology research and development within JD.
FastReID took the overall concept and design philosophy of Detectron2 as reference and designed it into a highly modular and extensible architecture that allows researchers to quickly implement new ideas; More importantly, the framework’s friendly management system configuration and engineering deployment functions allow engineers to deploy it quickly.
1) Highly modular and extensible design, which facilitates researchers to quickly insert newly designed modules into any part of the system, and helps researchers and engineers to quickly experiment with new ideas.
2) System configuration is highly managed, which can uniformly configure model structure, model training, model evaluation and model deployment into YAML files. The model structure can be customized backbone network, training test, loss function, etc.
3) The rich model evaluation system not only realizes the common CMC evaluation indexes, but also adds diversified indexes such as ROC curve, intra-class distribution curve and MINP, which can better reflect the model performance and guide the model deployment.
4) Easy engineering deployment, provides effective knowledge distillation module to obtain accurate lightweight model, and provides Pytorch to Caffe and Pytorch to TensorRT tools to obtain engineering deployment model in one click.
5) Provide leading pre-training models, providing SOTA models and configurations for multiple tasks including pedestrian re-recognition, occlusion/partial pedestrian re-recognition, cross-domain pedestrian re-recognition and vehicle-pedestrian re-recognition.
FastReID’s modules are shown in full, with training and reasoning above and below.
■ Training phase includes modules:
◇ Preprocssing includes rich data augmentation methods, including Flipping, Random erasing, auto-augment, Random patch, Cutout, etc., in order to improve model generalization and robustness.
Backbone includes mainstream Backbone according to requirements. There is also a lot of value-added modules such as non-local and Instance Batch Normalization (IBN).
Aggregation is used to aggregate the feature map generated by the backbone network into a global feature, such as Max pooling, average pooling, GeM pooling, attention pooling, etc.
◇ The Head module generally generates large dimensions of features, which need to be reduced and normalized to facilitate storage.
◇ Training strategy: At present, most networks use ImageNet pre-trained model. Some newly added layers do not have pre-training. Backbone Freeze and Network Warm up are used to pre-train the model in the initial training stage.
Loss functions include the common cross-entropy loss, Triplet loss, Arcface loss, and Circle loss.
■ In the reasoning stage, it contains modules:
◇ Measurement part, in addition to support common cosine and Euclidean distance, also added local matching method deep spatial Reconstruction (DSR).
Metric post-processing refers to the processing of retrieval results, including K- reordering and Query Expansion (QE).
General object recognition is a very important task in computer vision, which can be used in pedestrian/vehicle re-recognition, face recognition, wildlife protection, product tracing, photo shopping, unmanned supermarket and other applications. Including search TV actor of interest in the video, from the surveillance video search market lost child, from the city in search of the suspicious vehicle video monitor system, electronic commerce platform photographed shopping, product traceability anti-counterfeiting, etc., even in the field of ecological environmental protection but also the user the east-northern tiger protection, etc. The application of the protection of the endangered wild animals.
In the new retail scenario, cross-mirror tracking technology allows relational data between “people” and “fields” to be collected and visualized. Businesses have a stronger perception of user profiles and user behavior, enabling them to make more accurate business decisions. At the same time, it also provides the possibility of personalized service and precision marketing for users offline.
In the three data sets of pedestrian re-recognition task, Market1501, DukeMTMC and MSMT17, FastReID has reached the leading level of all evaluation indexes compared with the published papers so far. It also shows the influence of different modules on the performance of the whole system.
Table 1: FastReID results on three major databases
Table 2: Results of FastReID’s comparison of different modules on DukeMTMC
Due to the difference of images collected between different scenes of ReID across the domain, it becomes a very challenging task. FastReID adopts the technology of Hybrid label transmission to integrate pseudo-label prediction and feature learning into an end-to-end network, and adopts the Hybrid Memory Bank technology. Make cross-domain ReID also get the current leading performance.
Table 3: FastReID results across domains, M: Market1501, D: DukeMTMC, MS: MSMT17. BOT is bag-of Tricks, and the unsupervised performance of FastReID is close to supervised learning
Occluded/Partial Person re-identification is used to retrieve partially Occluded or partially incomplete images from candidate images. FastReID uses foreground sensing salient feature learning technology to solve the problem of severe occlusion in dense scenes.
FastReID also achieved the leading level in PartialREID, PartialiLIDs and OccludedReID.
Table 4: Performance of FastReID on occlusion/partial pedestrian re-recognition
FastReID uses adaptive attention model to integrate human head and shoulder to solve the re-recognition problem of pedestrians wearing black clothes or similar clothes under low light conditions. And it can solve the situation that clothes can not distinguish pedestrians in many practical scenes.
Not only has FastReID achieved leading results on pedestrian ReID, but it has also achieved leading levels on the Vehicular ReID, VehicleID, Veri-Wild database.
Table 5: The performance of FastReID on the VeRi, VehicleID and VerI-WILD databases respectively
FastReID has performed well in a variety of tasks. At present, this SOTA ReID Methods and Toolbox has been implemented for both academia and industry.
Within JD, the open source library has become the core engine for research and development of ReID technology within JD. Thus, FastReID is not just a laboratory project, but an open source library of product-level standards. FastReID’s highly modular and extensible design makes it easier for researchers and engineers to embed their own modules and implement tasks quickly. In addition, the manageable system configuration makes its application more flexible and easy to accomplish a variety of tasks. FastReID provides a fair comparison between different approaches.
We warmly welcome individuals and LABS to use FastReID to communicate and collaborate with each other to accelerate ai research and achieve technological breakthroughs.
* Click [read article] to see open source library information now
Why open source?
“We open source FastReID because open source research has been critical to the rapid growth of the FIELD of AI, thanks to the combined efforts of academia and industry. In the future, we will continue to develop and improve FastReID and hope that together we can drive the development of computer vision.” **
