Written by Vladimir Goncharov, Heart of the Machine.
Author Vladimir Goncharov focuses on two topics: PHP and Server Administration. Over the past six months, the author has spent his free time exploring the combination of PHP and OpenCV and using it to invoke and train excellent machine learning models. This article introduces how to use PHP and OpenCV to build face detection, face recognition, super resolution and object detection systems from a practical point of view, so PHP fans can use OpenCV to explore computer vision.
Like many developers, I often use other people’s work (articles on Medium, code on GitHub, etc.) and am happy to share my work with the community. Writing is not only a way to give back to the community, it also allows you to find like-minded people, get advice from professionals in a narrow field, and further your understanding of your field of study.
In fact, this article is about one of those moments. In this article, I describe my explorations in almost every spare moment I’ve had over the past six months, except for the time I spent watching TV shows and playing games.
Today, “machine learning” is growing fast, there’s a lot of writing about it, including those on Medium blogs, and almost every developer has started using machine learning for work tasks and local projects, but where to start and what to do is always confusing. Most articles aimed at beginners provide a stack of literature that is found to be out of touch with life in reading, or offer some “cheap” courses, etc.
Often a new approach to a particular problem is described in a newly published article, and you can find an implementation of the approach on GitHub. Since the more commonly used programming languages are: C/C ++, Python 2/3, Lua and Matlab, as well as frameworks: Caffe, TensorFlow, Torch. So the sheer number of subdivisions in programming languages and frameworks makes finding what you need and integrating it into your project much more complicated.
A DNN module has been added to OpenCV to reduce this confusion in a way that allows you to work directly with a model trained in the basic framework. I’ll show you how to use this module in PHP.
DNN module: github.com/opencv/open…
Jeremy Howard, founder of free practical courses called Machine Learning for Coders, believes there is a big line between learning machine learning and practical applications.
Howard thinks a year of programming experience is enough to start learning machine learning. I completely agree with his point of view, and I hope my article can help those who are unfamiliar with machine learning, and it is not clear whether is willing to engage in machine learning PHP developers to reduce the use of OpenCV threshold, at the same time, I will do my best I spent a lot of time to get ideas, so you don’t even need a long time can understand it.
I considered writing a PHP-OpencV module using SWIG and spent a lot of time on it, but nothing came of it. Everything is complicated by the fact that I don’t understand C/C++ and haven’t written extension files for PHP 7. Unfortunately, most of the material on the web is a PHP extension based on PHP 5, so I had to gather information bit by bit and solve the problem myself.
Then I found the PHP-OpencV library on GitHub, which is a PHP 7 module for calling OpencV methods. It took me several nights to compile, install, and run the sample. I started experimenting with different features of the module, but the library was missing some methods, so I added them myself and created a pull Request, which the library author accepted. After that, I added more features.
Php-opencv:github.com/hihozhou/ph…
Here’s how the image loads:
$image= CV \ imread (" images/faces. JPG ");Copy the codeIn contrast, under Python images load like this:
Image = cv2. Imread (" images/faces. JPG ")Copy the codeWhen an image is read in PHP (and in C++), the information is stored in the Mat object (matrix). In PHP, what is similar to, but different from, a multidimensional array is that this object can perform a variety of quick operations, such as dividing all elements by a number. In Python, when an image is loaded, a “NumPy” object is returned.
Be careful with the old defaults! It happens that imread (in PHP, C ++, and Python) does not load images in RGB format, but in BGR format. Therefore, in the OpenCV example, you can often see the process of converting BGR to RGB and vice versa.
Face detection
I tried this feature for the first time. For this purpose, there is a “CascadeClassifier” class in OpenCV that loads the pre-trained model in XML format. Until a face is found, the class recommends converting the image to black and white.
$src= imread (" images/faces. JPG ");$gray = cvtColor($src, COLOR_BGR2GRAY);
$faceClassifier = new CascadeClassifier();
$faceClassifier- > load (' models/lbpcascades lbpcascade_frontalface. XML ");$faceClassifier->detectMultiScale($gray.$faces);
Copy the codeComplete test example code: github.com/php-opencv/…
Results:
As you can see from this example, a human face can be found even in a photo of zombie makeup. Feature points do not interfere with face localization.
Face recognition
For face recognition, OpenCV has the “LBPHFaceRecognizer” class and the “train/Predict” method.
If we want to know who is in the photo, first we need to train the model using the train method, which takes two parameters: an array of face images for these images and an array of numeric tags. You can then call predict on the test image (human face) and get the matching value tag.
$faceRecognizer = LBPHFaceRecognizer :: create ();
$faceRecognizer-> train ($myFaces.$myLabels,1,1,1 = [1]); // 4 my faces$faceRecognizer-> update ($angelinaFaces.$angelinaLabels,2,2,2 = [2]); // 4 faces of Angelina$label = $faceRecognizer-> predict ($faceImage.$confidence);
// get label (1 or 2) and confidence
Copy the codeFull example code: github.com/php-opencv/…
Data set:
Results:
When I started calling the LBPHFaceRecognizer class, it couldn’t save/load/update the trained model. In fact, my first pull request added these methods: write/read/update.
Face markers/feature points
As I became familiar with OpenCV, I often saw photos of people with dots marked eyes, noses, lips, etc. I wanted to repeat this experiment myself, but it didn’t work in the Python version of OpenCV. I spent the night adding FacematkLBF support to PHP and returning an object. We load the pre-trained model, input an array of faces, and get an array of each person’s feature points.
$facemark = FacemarkLBF::create();
$facemark- > loadModel (' models/opencv - facemark - LBF/lbfmodel yaml ');$facemark->fit($image.$faces.$landmarks);
Copy the codeFull example code: github.com/php-opencv/…
Results:
As you can see from this example, zombie makeup makes it harder to find features on a person’s face. Feature points can also interfere with face localization. Light also has an effect, and in this case, foreign objects in the mouth (strawberries, cigarettes, etc.) probably won’t interfere.
After my first pull and drop request, I was inspired and started to understand what OpencV could do and came across an article called Deep Learning, Now in OpencV. I immediately decided to use the pre-training model in PHP-OpencV, which is widely available on the Internet. Although I later spent a lot of time learning how to use multidimensional matrices and Caffe/Torch/TensorFlow models without using OpenCV, loading the Caffe model did not prove difficult.
Deep Learning, now in OpenCV: github.com/opencv/open…
DNN model is used for face detection
Therefore, OpenCV allows you to load pretrained models in Caffe using the readNetFromCaffe function. It takes two arguments: the path to the.prototxt and.caffemodel files. The prototxt file has a description of the model, while in Caffemodel there are weights calculated during model training.
Here is an example of the beginning of a Prototxt file:
Input: "data" input_shape {dim: 1 dim: 3 dim: 300 dim: 300}Copy the codeThis file describes the input of a 1x3x300x300 4-dimensional matrix. In the description of the model, it is usually stated what it means to input in this format, but in most cases this means to input an RGB image (3 channels) with a size of 300×300.
By loading a 300×300 RGB image using the imread function, we get a 300x300x3 matrix.
There’s a blobFromImage function in OpenCV that converts a 300x300x3 matrix to a 1x3x300x300 format.
We can then simply apply the BLOB to the network input using the setInput method and call the forward method, which returns the final result to us.
$src= imread (" images/faces. JPG ");$net = \CV\DNN\readNetFromCaffe (' models/SSD/res10_300x300_ssd_deploy prototxt ', 'models/SSD/res10_300x300_ssd_iter_140000 caffemodel');$blob = \CV\DNN\blobFromImage($src.$scalefactor = 1.0, $size = new Size(300, 300), $mean = new Scalar(104, 177, 123), $swapRB = true.$crop = false);
$net->setInput($blob, "");$result = $net->forward();
Copy the codeIn this example, the result is a 1×1×200×7 matrix, or 200 arrays with seven elements per image. In a picture of four faces, the web found 200 candidates. Each object is of the form [,, $confidence, $startX, $startY, $endX, $endY]. The element $confidence represents “confidence,” or how good the predicted probability is, such as 0.75 is good. The following element represents the coordinates of the rectangular box of the face. In this example, only three faces were found with more than 50 percent confidence, while the remaining 197 candidates had less than 15 percent confidence.
Full example code: github.com/php-opencv/…
Results:
As you can see from this example, the neural network does not always produce good results when used “on the forehead.” A fourth face is not found, but if the fourth picture is taken separately and imported into the neural network, the face is found.
Use neural network to improve image quality
A long time ago, I heard about the Waifu2x library, which eliminates noise and increases the size of ICONS/photos. The library is written in Lua, using several models trained in Torch underneath (to increase icon size, eliminate photo noise, etc.). The authors of the library exported these models as Caffe and helped me use them in OpenCV. Thus, an example is written in PHP to increase the resolution of ICONS.
- Waifu2x library: github.com/nagadomi/wa…
- Full code for the example: github.com/php-opencv/…
Image classification
The MobileNet neural network trained on ImageNet can classify images. In all, it can distinguish between 1,000 categories, which is not enough for me.
Full code for the example: github.com/php-opencv/…
Tensorflow target detection API
The Network of MobileNet SSDS (Single Shot MultiBox Detector) trained with Tensorflow on COCO datasets can not only classify images but also return them to target areas, although only 182 categories can be detected.
Full code for the example: github.com/php-opencv/…
Syntax highlighting and code completion
I also added the phpdoc.php file to the repository as an example. Thanks to it, Phpstorm highlights the syntax of functions, classes, and their methods, and can also be used for code completion. This file does not need to be included in your code (otherwise you will get an error), just put it in your project. Personally, it makes programming easier for me. This file describes most of the functions in OpenCV, but not all of them, so you are welcome to send pull and drop requests.
Phpdoc.php:github.com/php-opencv/…
The installation
The “DNN” module is only available in OpenCV 3.4 (for previous releases it was contrib).
The latest version of Ubuntu 18.04 is OpenCV 3.2. It takes about half an hour to build OpenCV from the source code, so I compiled the package under Ubuntu 18.04 (also available for version 17.10, size 25 MB), The PHP-OpencV package was compiled for BOTH PHP 7.2 (Ubuntu 18.04) and PHP 7.1 (Ubuntu 17.10) (size 100 KB). Registered ppa: phP-OpencV, but not finished uploading yet, and found nothing better than uploading packages on GitHub. I also created a request for an account in PECL that went unanswered for several months.
Upload the package on GitHub: github.com/php-opencv/…
So now the installation under Ubuntu 18.04 looks like this:
apt update && apt install -y wget && \ wget https://raw.githubusercontent.com/php-opencv/php-opencv-packages/master/opencv_3.4_amd64.deb && DPKG -i Opencv_3.4_amd64. deb && rm opencv_3.4_amd64.deb && wget https://raw.githubusercontent.com/php-opencv/php-opencv-packages/master/php-opencv_7.2-3.4_amd64.deb && DPKG -i Php-opencv_7.2 -- 3.4_amd64.deb && rm php-opencv_7.2 -- 3.4_amd64.deb && \echo"The extension = opencv. So" > / etc/PHP / 7.2 / cli/conf. D/opencv. IniCopy the codeInstalling this option takes about 1 minute and all installation options are available on Ubuntu: github.com/php-opencv/…
I also compiled a 168 MB Docker image.
Use the sample
Download:
git clone https://github.com/php-opencv/php-opencv-examples.git && cd php-opencv-examples
Copy the codeRun:
php detect_face_by_dnn_ssd.php
Copy the codeOriginal link: medium.com/morozovsk/…