OpenCV Thematic 2 - Face detection + automatic sizing

NDK series:

[-NDK Guide -] Things you should know before NDK development
OpenCV topic 1 – AndroidStudio’s JNI project and references OpenCV
OpenCV Thematic 2 – Face detection + automatic sizing

As the saying goes: any code that isn’t based on requirements is rogue

First, face detection:

1. Prepare materials

First of all, we need to prepare the training data of face, which can be downloaded from the official Github and used here: lBPCascade_frontalface.xml. Then we welcome the world’s most stupid, naive, beautiful and kind-hearted Girl:

2. Java/Kotlin level

I wanted to write it all in Kotlin, but found that Kotlin can’t automatically generate JNI functions…

But I’m too lazy to find an ID, so I’ll just mix it up and use TolyCV to provide native methods.

---->[src/main/java/com/toly1994/toly_cv/TolyCV.java]----
public class TolyCV {
    public static native int faceDetector(Bitmap bitmap, Bitmap.Config argb8888, String path);
}
Copy the code

In Kotlin’s Activity, use faceDetector when clicking on an image to let C++ operate on the image

Because face recognition needs XML model files, here through copyCascadeFile will file into the package

---->[src/main/java/com/toly1994/toly_cv/MainActivity.kt]---- class MainActivity : AppCompatActivity() { private lateinit var mCascadeFile: File private lateinit var mFaceBitmap: Bitmap override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) setContentView(R.layout.activity_main) copyCascadeFile(R.raw.lbpcascade_frontalface,"lbpcascade_frontalface.xml") iv_photo.setOnClickListener { mFaceBitmap = BitmapFactory.decodeResource(resources, R.mipmap.kqq2) val count= TolyCV.faceDetector(mFaceBitmap,Bitmap.Config.ARGB_8888, MCascadeFile. AbsolutePath) title = "$count individual face detected" iv_photo. SetImageBitmap (mFaceBitmap)}} companion object {init { System.loadLibrary("toly_cv") } } private fun copyCascadeFile( id:Int,name:String) { try { val inputStream = resources.openRawResource(id) val cascadeDir = getDir("cascade", Context.MODE_PRIVATE) mCascadeFile = File(cascadeDir, name) if (mCascadeFile.exists()) return val os = FileOutputStream(mCascadeFile) val buffer = ByteArray(4096) var bytesRead: Int = inputStream.read(buffer) while (bytesRead ! = -1) { os.write(buffer, 0, bytesRead) bytesRead = inputStream.read(buffer) } inputStream.close() os.close() } catch (e: IOException) { e.printStackTrace() } } }Copy the code

3.C++ level uses OpenCV for face recognition

A lot of tutorials cram code into JNI’s CPP, and it feels confusing and uncomfortable to watch

Based on the single responsibility principle, we define a FaceDetector class that is specifically designed to recognize the array of incoming images and return the detected result set via the detectorFace method.

---->[src/main/cpp/FaceDetector.h]---- #include <android/bitmap.h> #include <opencv2/opencv.hpp> using namespace cv; #include <vector> using std::vector; Class FaceDetector{public: // Load file static void loadCascade(const char *filename); DetectorFace (Mat & SRC); static vector<Rect> detectorFace(Mat & SRC); };Copy the code

CPP file method implementation, the core is CascadeClassifier#detectMultiScale method

---->[src/main/cpp/FaceDetector.cpp]---- #include "FaceDetector.h" CascadeClassifier cascadeClassifier; / / face detection vector < the Rect > FaceDetector: : detectorFace (Mat & SRC) {vector < the Rect > faces; // Face array Mat temp_mat; CvtColor (SRC, temp_mat, COLOR_BGRA2GRAY); EqualizeHist (temp_mat, temp_mat); / / histogram equalization. / / multi-scale face detection cascadeClassifier detectMultiScale (temp_mat, faces, 1.1, 3, 0, the Size (300300)); return faces; } void FaceDetector::loadCascade(const char *filename) { cascadeClassifier.load(filename); }Copy the code

detectMultiScale

CV_WRAP void detectMultiScale(InputArray image, CV_OUT STD ::vector<Rect>& objects, Double scaleFactor = 1, int minNeighbors = 3, int minNeighbors = 1, double scaleFactor = 1, int minNeighbors = 2, Size minSize = Size(), maxSize = Size()); The maximum size of the targetCopy the code

4. The C++ layer draws squares to identify human faces

In fact, faces have already been recognized and stored in a vector. Now let me draw it on the graph

#include "FaceDetector.h" extern "C" JNIEXPORT jint JNICALL Java_com_toly1994_toly_1cv_TolyCV_faceDetector(JNIEnv *env, jclass clazz, jobject bitmap, jobject argb8888, jstring path_) { const char *path = env->GetStringUTFChars(path_, 0); / / file path FaceDetector: : loadCascade (path); // load file Mat srcMat; // bitmap2Mat(env, bitmap, &srcMat); / / picture source matrix initialization auto faces = FaceDetector: : detectorFace (srcMat); // Rectangle (srcMat, faceRect, Scalar(0, 253, 255), 5); // Draw rectangle mat2Bitmap on srcMat (env, srcMat, bitmap); } env->ReleaseStringUTFChars(path_, path); Return faces.size(); // return size}Copy the code

Depending on the model data, different parts can be detected, such as eyes :haarcascade_eye.xml

Detection will also appear errors, at this time can be filtered through some judgment to screen the results, such as the first detection of face, the other part can be filtered or according to the distance between the two eyes, calculate the impossible rectangle, it is removed, this is also a more interesting place for image recognition

Two, automatic size cutting

Now the requirement is: according to a photo (any size), cut the face and surrounding, and cut into the specified size, such as two inches :413*626, like this:

1. Java/Kotlin layer

A new native method, faceDetectorResize, is defined to perform this function and return a processed image

---->[src/main/java/com/toly1994/toly_cv/TolyCV.java]---- public class TolyCV { public static native int faceDetector(Bitmap bitmap, Bitmap.Config argb8888, String path); public static native Bitmap faceDetectorResize(Bitmap bitmap, Bitmap.Config argb8888 , String path,int width,int height); } ---->[src/main/java/com/toly1994/toly_cv/MainActivity.kt]---- iv_photo.setOnClickListener { mFaceBitmap = BitmapFactory.decodeResource(resources, R.mipmap.kqq) val bitmap= TolyCV.faceDetectorResize(mFaceBitmap,Bitmap.Config.ARGB_8888, Iv_photo mCascadeFile. AbsolutePath, 413626). SetImageBitmap (bitmap)}Copy the code

2. C + + layer

Here only for one face, multiple faces can take the idea of problem differentiation.

The first thing to solve is the regional problem: who is this Rect? If you have questions about an object, debug is the best choice

extern "C" JNIEXPORT jint JNICALL Java_com_toly1994_toly_1cv_TolyCV_faceDetectorResize( JNIEnv *env, jclass clazz, jobject bitmap, jobject argb8888, jstring path_, jint width, jint height) { const char *path = env->GetStringUTFChars(path_, 0); / / file path FaceDetector: : loadCascade (path); // load file Mat srcMat; // bitmap2Mat(env, bitmap, &srcMat); / / picture source matrix initialization auto faces = FaceDetector: : detectorFace (srcMat); Rect faceRect= faces[0]; rectangle(srcMat, faceRect, Scalar(0, 253, 255), 5); Env ->ReleaseStringUTFChars(path_, path); Return createBitmap(env,srcMat,argb8888); // return image}Copy the code

Knowing this information, it’s easy to build the target area (the red area), and all that’s left is to crop the red area

extern "C" JNIEXPORT jint JNICALL Java_com_toly1994_toly_1cv_TolyCV_faceDetectorResize(JNIEnv *env, jclass clazz, jobject bitmap, jobject argb8888, jstring path_, jint width, jint height) { const char *path = env->GetStringUTFChars(path_, 0); / / file path FaceDetector: : loadCascade (path); // load file Mat srcMat; // bitmap2Mat(env, bitmap, &srcMat); / / picture source matrix initialization auto faces = FaceDetector: : detectorFace (srcMat); Rect faceRect= faces[0]; rectangle(srcMat, faceRect, Scalar(0, 253, 255), 5); / / / / the srcMat with rectangle to identify the target area -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- the Rect zone; int a= faceRect.width; // width int b= faceRect. Height; / / high int offSetLeft = a / 4; //x offset int offSetTop=b*0.5; zone.x=faceRect.x-offSetLeft; zone.y=faceRect.y-offSetTop; zone.width= a/4 *2+a; Zone, height = zone width * (* 1.0 height/width); rectangle(srcMat, zone, Scalar(253, 95, 47), 5); Env ->ReleaseStringUTFChars(path_, path); Return createBitmap(env,srcMat,argb8888); // return image}Copy the code

Tailoring is very simple

createBitmap(env,srcMat(zone),argb8888); // Return the imageCopy the code

The Mat class overload () operator can pass in a rectangle, which is implemented by constructing a new Mat

This completes the cutting of the given proportion and ensures that the face is always in the middle and upper part.

---->[mat.hpp#Mat::operator()]----
/** @overload
@param roi Extracted submatrix specified as a rectangle.
*/
Mat operator()( const Rect& roi ) const;

---->[mat.inl.cpp#Mat::operator()]----
inline
Mat Mat::operator()( const Rect& roi ) const
{
    return Mat(*this, roi);
}
Copy the code

Another point to note: when the rectangle scope exceeds Mat, an error will be reported. It should be handled by adding white, Mark it

Finally, the only thing left to do is to resize. Be careful to remove the rectangle you drew, otherwise it will be printed in the result

extern "C" JNIEXPORT jobject JNICALL Java_com_toly1994_toly_1cv_TolyCV_faceDetectorResize(JNIEnv *env, jclass clazz, Jobject bitmap, jobject argb8888, jString path_, Jint width, jint height) {// env->ReleaseStringUTFChars(path_, path); Resize (srcMat(zone),srcMat,Size(width,height)); //<---- resize return createBitmap(env,srcMat,argb8888); // return image}Copy the code

OK, finished work, no longer afraid of sister let me help her set the size of the picture.

For large quantities, various people photos, want to crop neat, a for loop to get done, the program is excellent labor.

So you should feel more about OpenCV, in fact, just tune the existing method

I am Zhang Fengjieteili. if you have anything you want to communicate, please leave a message. You can also add wechat :zdl1994328

OpenCV Thematic 2 – Face detection + automatic sizing