“Luo: this should be the best 2d browser plug-in in the Eastern hemisphere! (ALMOST twice)”
What is it?
Just tap it and it will recognize all the 2d (bar) codes in your current Tab. When it’s done, it will automatically echo back to the corresponding location! (The GIF below is compressed and the color is wrong, so ignore it.)
In addition to the above automatic recognition, the “right-click recognition” function of peer “may” is also added to cover the scene of unsuccessful recognition:
Two, how to get it?
- 1, use Google plug-in store to obtain, later update more convenient! ❤ ️ recommended!
Google App Store: Handy 2d (bar) code scanner
- 2, if you do not have science online, you can click the link below to download the ZIP installation package, according to the online tutorial installation can be!
Download address: bj-mutou-1301404888.cos.ap-beijing.myqcloud.com/awsome-qr-c…
Install the tutorial: www.cnplugins.com/zhuanti/new…
3. Listen to the (somewhat bumpy) journey to achieve it
The following simple record more than a week of time, the mood of great sorrow, also involves some optimization scheme ideas, is how to step by step optimization to the final effect, we can eat as appropriate ~
1. First line of thoughts
I installed a lot of two-dimensional code recognition plug-in, found that it is not as good as the imagination (some even silly)😶, after work imagine what I want to function is what? How to design is better use?
With that in mind, I silently opened my little notebook one Wednesday night and jotted down the first line of thought:
“Automatic”? B: well… It’s easy to imagine, like: first this, then this, and then…
However, the browser is not a smart person, it does not have awareness, and how to know the current page in the TWO-DIMENSIONAL code where? No… How does it know what a QR code is?
Knowledge has a price
We have learned in the second grade of primary school, surrounded by three generous blocks, the middle of the pixel is a square code is the two-dimensional code, but the computer does not know, it sees is a pixel, so how to let it know what is the two-dimensional code, is placed in front of the first problem.
Let the computer master the ability to understand the two-dimensional code, you need to let it continue to learn, this is lofty and cool “machine learning”.
But… I have been engaged in front end positions since I graduated from university, and I have little understanding of machine learning related concepts, EM… I don’t know anything about it. I’ve only heard one or two terms: nerves, training…
Do not understand what to be afraid of, do not understand to learn 😀, open dreamworks website, surprising discovery really has relevant course, a bit small expensive but also good, buy it!
Ask if you don’t understand
After a night of study, I had a simple knowledge base, and then I consulted my graduate students with relevant questions, and got a roughly accurate direction: “Target detection”.
Successful identification, life milestone!
With a general direction, you have a lot of momentum!
Saturday and Sunday, gave up sleeping in, lunch break, daze, watch TV 🙂, experienced countless adjustment, collapse, transformation, buy members, finally successfully identified the first picture of life:
That’s a good start!
5, slightly boring preparation stage
Then I tried to find a variety of existing classification of two-dimensional code training set, but did not find a more appropriate…
em… What if there is no suitable one? You have to build your own! Save screenshots one by one, and annotate, classify, and collect them.
Finally successfully get the labeled resource set tag, dense…
That’s when I realized that it’s not easy to train models. There’s a threshold… You need to have a computer with a decent GPU. Looking at my laptop I got lost in thought…
The first village
Luckily, Google offers a platform called Colaboratory, which provides a Python environment and… Free GPU, you can train your models here happily!
After repeated failures… An error… After repair, finally start training!
Old father shed a line of tears… Looking at the increasing rate of accuracy, the in the mind is still very happy, as if his good big son began to understand slowly…
After the training, after transformation, optimization, replacement model, export and other operations, finally got a satisfactory weight file, can finally identify the TWO-DIMENSIONAL code in the browser!
Write a demo and test it!
7. A bolt from the blue
Excitedly do a good browser plug-in, load weight file, found a big bolt from the blue!
Tfmodel.executeasync (input) is an asynchronous method that causes the UI page to freeze while loading weights are warming up!
Google did some searching and found no solution, which must block the current thread due to webGL’s problems.
The final phenomenon is that when the plug-in is opened, the page will be stuck for 5-6s, and the result can be rendered in 1-2s, which takes about 8s and is a very bad experience.
I was so disappointed that I almost gave up…
The second village
After I calmed down after lunch, I thought about the optimization scheme and found that I could try using Web worker. Since it would definitely block the main thread, I would do the operation of loading weights in other threads, so that the page would not be stuck.
Since the communication between the Web worker and the main thread cannot transfer non-serializable objects, only serializable data can be transferred, which leads to the need to convert image data on both sides, and a lot of adaptive code is written…
After the final optimization, the effect is as follows. It can be found that the button can also be clicked during loading to prove that the main thread is not stuck:
But have you also found that the smart, loading weight spend some time, no, it is too long, and the plug-in positioning is a one-time tools, almost no open (load) only once, and recognition of many times, so how to reduce the load for the first time (preheating time, is a headache problem…
The third village
Later, during my lunch break on Thursday, I woke up suddenly and found that I could load background.js. And there are two benefits:
- 1, natural other threads, do not affect the main thread
- 2. Background. js will be executed at the moment the plug-in is installed. At this time, the model will be preheated, and the time to identify the image will be greatly reduced. After rough calculation, most of the time from opening the plug-in to the completion of recognition is within 1s, and very few will be within 2-3s, which is completely in line with expectations!
The optimized effect is roughly as follows:
As you can see, the speed is very fast, and finally meets my expectations (an on-demand plug-in scenario).
10, Happy Ending
At this point, from conception to its implementation, landing plugin is roughly after more than a week of time, not only has a record of the above during the period of trial and error process, after other students remind, also added a key copy button, a key support, right-click menu bar code identification, also try to use electron to implement the desktop to the development, the recognition is not limited in the browser, In the end, the package size was too large (100MB+), which did not meet my expectations (less than 10MB), so I reluctantly gave up.
😌 the process is much more painful than expected at the beginning of the plan. There are so many pits to step on that I can’t bear to look back. If I knew at first that I would step on so many pits, I might have given up at that time
But fortunately, the final effect is good, more in line with expectations, but also quite happy! 😎
The end is the beginning
Because only 250 images are used to train the recognition model, some scenes may not be recognized. Please understand and give it a space for slow progress
OK! Going back to the beginning of this article, this is probably the best 2d (bar) code browser plugin in the Eastern hemisphere! 😉 install try!