· Hujiang CCtalk front-end development engineer, Taro framework issue maintenance volunteer, mainly focusing on front-end UI writing and team document construction. This article explains the core logic of my quick mask small program, how to put a mask on the face, that is, to use Tencent cloud face recognition in the facial analysis to achieve.

Basic steps

  1. User select image
  2. Picture cutting, with the help of canvas to achieve
  3. Convert canvas image to applet image
  4. Images are converted to Base64 data
  5. Upload Base64 to Tencent cloud for facial recognition

Let’s start with two quick questions

** Problem 1: The first problem to deal with is to upload directly from the small application side or to the personal server or cloud and then forward? ** Small program side direct upload

  • We need to configure multiple Tencent cloud domain names on wechat public platform, such as iai.tencentCloudapi.com, tiia.tencentCloudapi.com and so on
  • Also depends on the user side of the network speed, and the small program side request is still very prone to failure, will make the small program experience greatly compromised
  • If a function has multiple network requests, such as face recognition, facial analysis, intelligent beauty, the small program side to request, directly consumed a lot of user traffic.

To a personal server or cloud and forward

  • There is no need to configure multiple cloud domain names on wechat public platform
  • Multiple network requests can be combined to achieve faster response based on personal server or cloud stable network environment

** Question 2: How do I choose between personal server and applets cloud development? The latest hot concept is Serverless, which simply means that databases, files, and other resources can be called directly from the server without the help of operation and maintenance. And I did the following Settings in the personal server in this small program development

  1. Purchased a domain name and put it on record, set up the secondary domain name, and set up HTTP SSL certificate for the secondary domain name
  2. I bought a cloud server of Tencent Cloud and installed nginx, NodeJS and other basic software on it
  3. After starting a simple Rest API service written based on NodeJS express.js, requests from secondary domain names are forwarded to the Node site using nginx

First of all, the domain name needs dozens of yuan every year, the server also needs dozens of yuan or hundreds of yuan, the website record also needs all kinds of restrictions, such as in Shanghai, the need for household registration or residence permit, in Jiangsu, the need for mobile phone numbers in Jiangsu. Besides, in operation and maintenance, I not only need to set up the above basic functions, but also consider setting up the test environment, formal environment, and consider the stability of the service. That’s why I chose Applets Cloud Development. Because it’s free, efficient and stable. Cloud development provides several basic capabilities to support:

  • Cloud function – the code running in the cloud, wechat private protocol natural authentication, developers only need to write their own business logic code
  • Database – A JSON database that can be operated on the front end of an applet and read and write in cloud functions
  • Storage – upload/download cloud files directly in front of the small program, and manage them visually in the cloud development console
  • Cloud call — The native wechat service integrates the ability of using applets open interface based on cloud function authentication exemption, including server-side call, access to open data, etc

Free cloud development in the small program side, the quota is as follows:

This can be useful for novice or experimental projects. If you want to upgrade, small program also provides a very complete upgrade program. The details can be viewed at the following link. Developers.weixin.qq.com/miniprogram… Nodejs-based cloud development environments can be highly customizable.

  • wx-server-sdkTo call the applet open interface
  • tcb-admin-nodeIt allows you to use Node.js service to access TCB’s service on the server side (such as Tencent Yunyun function or CVM, etc.).
  • tencentcloud-sdk-nodejsAnd call Tencent cloud services, such as face recognition, facial analysis and so on
  • Even, the above can be modified code, custom function implementation, for example, WHEN I first use facial analysis, need to change the cloud signature method, andtencentcloud-sdk-nodejsThe NPM version is not yet supported, so I will download the code and use it again.

The text start

First edition VS fourth edition

First edition: Personal server version, with images ranging from 1.2 to 2M. Fourth edition: Cloud development version, using cloud storage fileID as the carrier

Image compression using canvas canvasToTempFilePath, JPG format, 0.8 quality, will reduce the image from 1.2-2MB to less than 150KB on An Android phone. The image is also displayed locally. Using a small program called compressImage (0.1 quality) to compress images works well on the iPhone but not so well on Android, but it’s also available here. Even if the quality is low enough for image review, facial analysis. When temporary uploaded images are used as the carrier and fileID is used as the symbol of cloud function calls, the volume of cloud function calls is small and cloud storage uploads and downloads are stable. Image security and facial analysis can be requested at the same time, but using promise.allSettled (). Three problems are solved here:

  • Json timeout. Otherwise, the cloud function call will report an error directly, even if the cloud development environment returns data later.
  • Image size limit 500KB
  • If the base64 data is larger than 1MB, change the signature method toTC3-HMAC-SHA256.

So what’s the effect? The total usage time is about 5 seconds, with the request time being about 3 seconds.

Note: Time is measured in the local development mode of cloud development, cloud call speed is faster. Total use time: from the time of image compression, after calling the cloud function, the cloud function recognizes the five features information, and after returning, the small program processes the five features information and renders the mask effect. Request time: the time when the miniprogram calls the cloud function and the cloud function recognizes the five-sense information and returns to the miniprogram.

Fourth edition VS fifth edition

Fourth edition: Cloud development version, using the cloud storage fileID as the intermediate carrier fifth edition: Cloud development version, using Base64 data directly request

Instead of using cloud storage as the transmission carrier, the base64 data with the size less than 150KB after image compression is used to directly request, which reduces the steps of two asynchronous operations, image uploading on the small program side and image downloading in the cloud development environment. There is also ArrayBuffer data on the applet side, but it was discovered during local development that whenever the size exceeds 50KB (guessed), the cloud function call will directly report an error.

PS: The image recognition of my small program is only a temporary request for data, and there is no need to upload the image to the cloud storage, so that users can see the image again next time.

So what’s the effect? The total usage time is less than 3 seconds, of which the request time is about 0.8-1.2 seconds.

conclusion

  • Local identification, original image required
  • Five senses recognition contour line, image audit with low quality images
  • Image compression, try to compress to the lowest possible
  • The number of asynchronous requests should be reduced and the final data can be passed directly, but the data should be small

Details of five versions of applets

  • Version 1: Personal server version

    • The total usage time was about 13 seconds, and the request took 10 seconds
    • After local development, git upload code, install nodeJS dependency on server, pM2 start, very troublesome.
    • There is no distinction between the test environment and the formal environment. If it is actually used, it must be reconfigured, and it is very troublesome to use each time.
    • Nginx is used for forwarding on personal services and nodeJS is used for receiving
    • Nginx limits the size by default, so change the size
    • Call Tencent cloud service, more than 1M need to change the signature method
    • Canvas is converted to an image, which is then converted to Base64 encoding and is typically 1.2-2m in size
    • Send API requests to personal servers with base64 encoded data
    • Small program
    • Personal server
    • The effect

  • Second edition, cloud development, direct upload base64

    • Total usage time is approximately 11 seconds
    • The request time is long and local development is prone to failure because timeout has to be adjusted to be greater than 10 seconds. I changed it to 20 seconds
    • The development experience is much better than the personal server, basically in wechat developer tools and Visual Code can switch
    • Facial analysis, coded in Base64. If the signature method is larger than 1 MB, change the signature method
    • Canvas is converted to an image, which is then converted to Base64 encoding and is typically 1.2-2m in size
    • The cloud function is called, and the data is base64 encoded
    • Small program
    • Cloud development cloud functions
    • The effect

  • The third version, cloud development based on the switch to cloud storage forwarding

    • Total use time is about 8 seconds
    • Using fileID storage and sending fileID to cloud functions is more secure, because the storage developed by cloud requires Tencent Cloud permission
    • Receive information from all five senses
    • Delete images from cloud storage using fileID
    • Use fileID to download image content and convert it to Base format
    • Facial analysis, coded in Base64. If the signature method is larger than 1 MB, change the signature method
    • The canvas turns into an image,
    • The cloud calls uploadFile to upload cloud storage and returns fileID
    • Call the cloud function with fileID
    • Small program
    • Cloud development cloud functions
    • Small program
    • The effect

  • Fourth edition: greatly compressed picture, cloud storage fileID as the intermediate carrier

    • The total usage time is about 6 seconds, of which the request time is about 4 seconds
    • Add canvas image compression, the effect is very obvious
    • Wechat image compression, android effect is not obvious
    • Set the five senses information, if the picture audit failure, report an error
    • Delete images from cloud storage using fileID
    • Use fileID to download files from cloud storage
    • Promise.allSettled asynchronous full return
    • Image audit – Use Buffer
    • Facial analysis – Base64 format
    • Canvas is converted to an image. Pay attention to image compression. Set the format to JPG and the quality to 0.8
    • Small program, image compression, android effect is not big? But the size of the picture is below 150K
    • Upload the file to the cloud development environment to obtain the fileID
    • Call the cloud function with fileID
    • Small program
    • Cloud development cloud functions
    • Small program
    • The effect

  • The fifth edition

    • The total usage time is about 2-3 seconds, and the request time is under 1.2 seconds, usually 0.8 seconds
    • Compared with the fourth version, it saves the process of uploading and downloading pictures, saving the request time, but correspondingly, the security is not so high
    • The optimization solution is that base64 data can be encrypted.
    • Set the five senses information, if the picture audit failure, report an error
    • Promise.allSettled asynchronous full return
    • Image audit – Use Buffer
    • Facial analysis – Base64 format
    • Canvas is converted to an image. Pay attention to image compression. Set the format to JPG and the quality to 0.8
    • Small program, picture compression quality is 0.1, android effect is not big
    • The image is converted to Base64 format with a size less than 150K
    • Call the cloud function with data in Base64 format
    • Small program
    • Cloud development cloud functions
    • Small program
    • The effect

Article related content:

Cherish life, start from me, quickly put on masks, to introduce you to my open source Taro + Tencent cloud development small program “quickly wear masks” it can be intelligent face recognition, to wear masks as a group. (*￣) ￣)

Taro cross-terminal framework is adopted, Tencent cloud development mode is adopted, and face recognition based on Tencent cloud facial analysis is adopted to realize the function of automatically putting a mask on the profile picture. Source code address: quickly wear mask source code main functions

  • Intelligent face recognition, facial positioning
  • Support for multi-party identification
  • Support to add refueling pictures

Scan code preview wechat search: wear a mask quickly

Applet screenshot

About Cloud Development

CloudBase is a product solution of cloud integration. It adopts serverless architecture, eliminates operation and maintenance transactions such as environment construction, supports one cloud and multiple terminals, and helps to quickly build small programs, Web applications and mobile applications.

  • Technical documentation: www.cloudbase.net/
  • Wechat search: Tencent Cloud Development, get the latest progress of the project