A picture is worth thousands of words, first for you to show the demo video.
It is understood that the Scan Wufu model of Alipay client runs in native engine, while our Wufu model runs on TensorFlowJS (hereinafter referred to as TFJS) platform, which can theoretically run in any terminal (including browser). It can be seen that the model recognition speed is about the same as that of the Alipay client.
Now I will introduce our five Blessings model step by step from training to browser side deployment is how to achieve.
First, a brief overview of our model
volume
The final compressed volume is around 400KB, and the ultimate compression is around 200KB (which loses a bit of model performance).
performance
The accuracy rate is 95+% and the recall rate is 92+%
speed
Performance on the MAC is around 30fps;
The performance of android high-end machine is about 15~ 20FPS;
More detailed performance tests will be provided later;
What is TFJS?
Tensorflow.js is a webGL-accelerated, browser-based machine learning JS framework. With Tensorflow.js, we can develop machine learning, run existing models, or retrain existing models in the browser. Here is the official website
What is TFJS model isomorphism?
The term isomorphism can be compared to SSR isomorphism rendering, that is, server and client rendering use the same framework and code. So, as the name implies, isomorphism of TFJS model means that the training of the model and the inference on the end of the model also use the same set of framework and code. That is, TFJS is used for offline training on the server and TFJS is used to deploy the model on the client.
Why use TFJS to train models?
No why… Just to prove TFJS has the ability to train models, the real production environment is recommended to use Tensorflow for PY training, the reasons will not be repeated.
Principle of model
In fact, the principle of our five blessings model is relatively simple, which is a binary model. “Is the word” or “is not the word”. The convolutional network used is MobileNet. See MobileNet for details.
The training process
I collected 3800 pictures related to the character “fu” from the Internet, among which there were about 1200 pictures of the character “fu”. The six of us spent about half an hour or so. In order to speed up our training speed, I chose to directly take a pre-training model on ImageNet data set for transfer learning
At this point, the size of the model we got was around 4MB, which was unacceptable for a model that needed to be deployed on the end. Okay, check the cause: We found
Our final model looks like this:
The deployment process
The deployment process is relatively simple, upload our model files (model.json, weights. Bin) to CDN, load the model into memory through TFJS on the client side, and we can happily identify the five happiness ~
Who are we?
We are the ai-Team of Artificial Intelligence Department of Ant Financial, committed to developing the best AI service applications and front-end intelligent applications. Welcome to join us and explore the future of front-end intelligence.
If you are interested, please contact me at [email protected]