Chinese Number Gestures Recognition APP (0-10)
Digital Gesture Recognition APP Based on Convolutional Neural Network
1. Project Introduction
This is a digital gesture recognition APP (Android) based on convolutional neural network. The main function is: digital gestures made by mobile phone camera recognition, can recognize the corresponding gestures of the numbers 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10.
Chinese – number – gestures – recognition under the project contains two pieces of code: 1. DigitalGestureRecognition for android APP code; 2. Digital_gesture_recognition is the code for PC side to process data and train model, and the programming language is Python.
Development environment:
PC side: PYTHon3.6, Tensorflow-gpu1.8, Keras2.1.6, PIL, Hardware NVIDIA GTX1070, 16GB Momery
APP side: Android Studio3.1.2, TensorFlow Lite, OpencV-3.4.0-Android-SDK
2. Demo of APP recognition results
2.1 Daytime Scenario
2.2. Night scene
2.3, complex scenes (low accuracy, luck… Just a knowing smile, just curious to try)
3. Data sets
3.1. Original data set
Address: pan.baidu.com/s/1whTtvo6G…
This dataset consists of 215 digital gesture pictures, which were taken by myself, female Piao (Li Na) and my friends: Jiang Wen, Song Yuzhang, Peng Zhongjun, Zhang Meng, Yuan Cheng, Xing Shouyi and Zheng Chao. Thank them for their help. The corresponding label of each image has been marked in the image name format: tag _ number. Such as:
3.2. Data sets with data augmentation
Address: pan.baidu.com/s/1_tLq_HcS…
The original images were translated, stretched, and rotated to produce 100 images per image using digital_gesture_recognition/data_augmentation. Py. Therefore, the generated data set contains 21592 images.
NOTE: The size of this dataset is 11.6GB, about 11GB after compression. If your network bandwidth is slow to download, it can be regenerated yourself using the original data set and digital_gesture_recognition/data_augmentation. Py.
3.3. Compressed data set
Each image is scaled to 64*64 and the data set formed is digital_gesture_recognition/ resized_IMG, a total of 21592 images. The compression algorithm used is area Interpolation in TensorFlow. The digital_geSTUre_recognition /resized_img_split data set is the same as resized_img, but is split into 10 smaller folders.
3.4. H5 data set
Address: pan.baidu.com/s/1COV1UVM3…
This H5 data set is obtained by converting images in Resized_IMG into H5 files. The size of this dataset is: 215926464*3, and you can use the H5PY library to operate on this dataset. For example, you can load a dataset with the following statement:
Data = H5py. File("dataset//data.h5","r") X_data = Np. array(data['X']) #data['X'] is h5py._hl.dataset. Array Y_data = np. Array (data['Y'])Copy the code4. Trained model
In digital_geSTUre_recognition /model_200, is the model obtained by training epoch=200, which is also the model used in APP, while model_2500 is the model trained epoch=2500. The accuracy of the model trained on PC side in the training set and test set is 99.8% and 99.5% respectively. On the APP, the accuracy rate is far less than this figure because the environment is more complicated. Randomly test several photos taken by friends on PC (none appeared in training set or test set), and the test results are satisfactory. The test results are as follows:
5. Configure the Android Studio environment
The trained model migration to Android in the Studio, see my blog this tutorial: blog.csdn.net/u012328159/…
The Android OpenCV environment that is configured in the studio, see my blog: blog.csdn.net/u012328159/…
On the other project introduction, please see my blog: blog.csdn.net/u012328159/…
Note: The camera part of the APP uses the CHNicelee code at github.com/CHNicelee/C… Thank you CHNicelee about this camera code blog: blog.csdn.net/q1242027878…