1. Background

As the epidemic is raging, effective isolation is an effective means to defeat the virus as soon as possible. Many local governments have put forward strict regulations on the management of residents’ travel. For example, Yuhang District of Hangzhou issued “ten strict” control measures on February 3:

This brings new challenges to community management. The following problems exist in traditional guard marking:

  1. The situation of people in and out is all manually recorded, and mistakes and omissions can not be timely and effectively dissuaded from frequent residents.
  2. Lack of control and measurement of the overall situation of resident isolation from a global perspective, such as who frequently visits and the total number of residents.

In order to solve the above problems of resident access management, aliyun intelligent database vector retrieval team provides a set of face recognition model under the scenario of wearing masks which is highly compatible for free, and based on the vector retrieval ability of AnalyticDB, builds a set of solution for community staff management, which will be open source to the community.

This program can effectively improve the efficiency of community access management in the current epidemic situation. We also provide AnalyticDB free of charge to users for access management applications related to the current pneumonia outbreak.

The following is the first introduction of our scheme, and then the key technologies of face recognition and AnalyticDB vector retrieval will be introduced in detail to facilitate developers to do secondary development. Finally, we will attach the open source address.

2. Introduction of community personnel management solutions

2.1 Community staff management solution functions

1. Automatic registration of population in the registered community, basic information and face features, the interface is as follows:

2, through the camera automatic face recognition, return to the visitor’s home all access records. It is convenient for community managers to carry out efficient access management. In the current epidemic environment, people commonly wear masks, and removing masks will increase the risk of pneumonia infection. Therefore, this program provides a set of algorithms to support face recognition in the case of wearing masks. The demo effect is as follows:

3. It can retrieve residents’ visit records through any combination of facial photos and structured information, and provide statistical analysis capabilities to provide global measurement data for district managers.

2.2 Overall application Architecture design

The overall architecture of the access management system is shown in the figure below. The front-end interface is realized by HTML and javascript. Its functions include face access control under the scenario of wearing masks, query the visitor’s family members’ access records within 2 days through face recognition, personnel registration, and search for visitor records through the free combination of face and structured information in the background.

The face recognition module converts videos containing faces into human face feature vectors. The face recognition module mainly uses face detection and face tracking module of Seetafce engine and face recognition, eye recognition and mask detection models developed by AnalyticDB team. AnalyticDB is responsible for the whole application of all structured data and face recognition module generated face feature vector storage and query.

3. Introduction of key technologies

3.1 Face recognition algorithm for the epidemic

The algorithm process is shown in the figure below. During the registration process, we extract the overall facial features and eye features of the registrant through face recognition model and eye recognition model respectively, and write the extracted feature vector into AnalyticDB. In the query process, we will first detect whether the visitor wears a mask through the mask detection model. If not, we will use the features of the whole face to search for similar features in AnalyticDB. If the similarity between the features and the visitor’s facial features meets the threshold, the corresponding results will be returned.

If the visitor wears a mask, nose, mouth and other features will be missing, and the overall facial feature extraction model cannot accurately retrieve the correct record. At this time, we will use the eye recognition model to extract the features of the client’s eyes, forehead and other parts that will not be covered by the mask, and then retrieve the previously saved eye features in AnalyticDB.

The face recognition model, eye recognition model and mask detection model used in the system will all be open to the community. After testing, the accuracy of mask detection model is >99.5%. The accuracy of face recognition model and eye recognition model on commonly used data sets in academia is shown in the table below.

It can be seen that AnalyticDB’s model still has more than 99+% recognition accuracy on the LFW data set using only eye features.

AnalyticDB is a pB-level real-time data warehouse with high concurrency and low latency on Ali Cloud. AnalyticDB can perform real-time multidimensional analysis perspective and business exploration for trillions of data in milliseconds.

AnalyticDB for MySQL is fully compatible with MySQL protocol and SQL:2003 syntax standard, AnalyticDB forPostgreSQL supports standard SQL:2003, highly compatible with Oracle syntax ecology. At present, both products contain vector retrieval function, which can support similar query and recommendation system of human face, human body, vehicle, etc.

Currently AnalyticDB in real application scenarios can support 1 billion level of vector data query, 100 milliseconds level of response time. AnalyticDB has been deployed on a large scale in security projects in several cities.

In common applications that include vector retrieval, developers often use a vector retrieval engine (such as Faiss) to store vector data, and then use a relational database to store structured data. It is also necessary to query the two systems alternately during the query, which requires additional development effort and is not optimal performance.

AnalyticDB supports the retrieval of structured data and unstructured data (vector), only using SQL interface can quickly set up a graph search graph or picture + structured data mixed retrieval functions. AnalyticDB’s optimizer will select the optimal execution plan according to the data distribution and query conditions in the mixed retrieval scenario to ensure the recall and obtain the optimal performance.

In our access management system, we realized the function of using photos, gender, age, start time and end time to query access records through AnalyticDB.

Such a graph search graph + structured search function, can be achieved through a SQL:

Note: * Swipe left and right to read

select  name, # the name
        gender, # gender
        age, # age
        time, # Access timeL2_distance (feature, array [0.017, 0.032,... ::real[]) as distanceVector distance
from demo.person a, demo.face_feature b, demo.access_record c
where a.pid = c.pid 
        and a.pid = b.pid 
        and name = 'Joe' # Name condition
        and gender = 'male' and age = 30 # Gender condition
        and time >= '2020-02-07' # start time condition
        and time <= '2020-02-09' End time condition
      order by distance; # Sort by vector similarityCopy the code


The table Demo. person stores the basic information of each person, Demo. face_feature stores face feature vector,demo. access_record stores all visit records, pid is the unique ID of each person.

Mixed retrieval of structured information + unstructured information (picture) is widely used in practical applications. For example, when the face access control system is deployed in multiple communities, we use a table to store the face features of all communities, in face retrieval we only need to retrieve the face features of the current community. In this case, we can easily implement AnalyticDB by adding where cell name =’ XXX ‘to SQL.



Read more: https://yqh.aliyun.com/detail/6366?utm_content=g_1000105429

On the cloud to see yunqi: more cloud information, on the cloud case, best practices, product introduction, visit: https://yqh.aliyun.com/