Java version of face detection in detail: coding

Welcome to my GitHub

Github.com/zq2599/blog…

Content: all original article classification summary and supporting source code, involving Java, Docker, Kubernetes, DevOPS, etc.;

This paper gives an overview of

If you’ve seen the three-Minute Experience: Face Detection in Java, or even tried it yourself, you might be interested in the technical details behind it. To develop such an application, we need to do three things:

Prepare the Docker base image
Developing Java applications
Package Java applications into package files and integrate them into the base image to obtain the final Java application image

For the preparation of docker basic image this work, we in the previous “Java version of face detection in detail: Running environment docker image (CentOS+JDK+OpenCV)” has been completed, the next thing to do is to develop Java applications and make it into a docker image

Version information

The version information involved in this Java application is as follows:

Springboot: from 2.4.8
Javacpp: 1.4.3
Javacv: 1.4.3

Download the source code

The full source code for this article can be downloaded at GitHub with the following address and link information (github.com/zq2599/blog…

The name of the	link	note
Project home page	Github.com/zq2599/blog…	The project’s home page on GitHub
Git repository address (HTTPS)	Github.com/zq2599/blog…	The project source warehouse address, HTTPS protocol
Git repository address (SSH)	[email protected]:zq2599/blog_demos.git	The project source warehouse address, SSH protocol

The Git project has multiple folders. The source code for this project is in the Javacv-tutorials folder, as shown in the red box below:

coding

In order to manage the source code and JAR dependencies, the project adopted a maven parent-child structure. The parent project was called Javacv-tutorials, and its POM.xml looked like this.


      
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.bolingcavalry</groupId>
    <artifactId>javacv-tutorials</artifactId>
    <packaging>pom</packaging>
    <version>1.0 the SNAPSHOT</version>
    <modules>
        <module>face-detect-demo</module>
    </modules>

    <properties>
        <java.version>1.8</java.version>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
        <maven-compiler-plugin.version>3.6.1 track</maven-compiler-plugin.version>
        <springboot.version>From 2.4.8</springboot.version>

        <! -- Javacpp current version -->
        <javacpp.version>1.4.3</javacpp.version>
        <! -- OpencV version -->
        <opencv.version>Rule 3.4.3</opencv.version>
        <! -- FfMPEG -->
        <ffmpeg.version>4.0.2</ffmpeg.version>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.projectlombok</groupId>
                <artifactId>lombok</artifactId>
                <version>1.18.18</version>
            </dependency>

            <dependency>
                <groupId>org.bytedeco</groupId>
                <artifactId>javacv-platform</artifactId>
                <version>${javacpp.version}</version>
            </dependency>
            <dependency>
                <groupId>org.bytedeco</groupId>
                <artifactId>javacv</artifactId>
                <version>${javacpp.version}</version>
            </dependency>
            <! -- javacpp -->
            <dependency>
                <groupId>org.bytedeco</groupId>
                <artifactId>javacpp</artifactId>
                <version>${javacpp.version}</version>
            </dependency>
            <! -- ffmpeg -->
            <dependency>
                <groupId>org.bytedeco.javacpp-presets</groupId>
                <artifactId>ffmpeg-platform</artifactId>
                <version>${ffmpeg.version}-${javacpp.version}</version>
            </dependency>
            <dependency>
                <groupId>org.bytedeco.javacpp-presets</groupId>
                <artifactId>ffmpeg</artifactId>
                <version>${ffmpeg.version}-${javacpp.version}</version>
            </dependency>
        </dependencies>

    </dependencyManagement>
</project>
Copy the code

Underneath javacv-tutorials, create a new sub-project called face-detect-demo. Inside this is the application we will develop today. Its POM.xml looks like this:


      
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>javacv-tutorials</artifactId>
        <groupId>com.bolingcavalry</groupId>
        <version>1.0 the SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>face-detect-demo</artifactId>
    <packaging>jar</packaging>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-dependencies</artifactId>
                <version>${springboot.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <! FreeMarker template view dependency -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-freemarker</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>javacv-platform</artifactId>
        </dependency>
        <dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>javacv</artifactId>
        </dependency>
        <! -- javacpp -->
        <dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>javacpp</artifactId>
        </dependency>
        <! -- ffmpeg -->
        <dependency>
            <groupId>org.bytedeco.javacpp-presets</groupId>
            <artifactId>ffmpeg-platform</artifactId>
        </dependency>
        <dependency>
            <groupId>org.bytedeco.javacpp-presets</groupId>
            <artifactId>ffmpeg</artifactId>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <! -- If the parent project is not Springboot, you need to use the plugin in the following way to generate a normal JAR -->
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <mainClass>com.bolingcavalry.facedetect.FaceDetectApplication</mainClass>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>repackage</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>
Copy the code

The configuration files are as follows, focusing on the front segment template, file upload size, model file directory and other configurations:

# # # FreeMarker configuration
spring.freemarker.allow-request-override=false
#Enable template caching.
spring.freemarker.cache=false
spring.freemarker.check-template-location=true
spring.freemarker.charset=UTF-8
spring.freemarker.content-type=text/html
spring.freemarker.expose-request-attributes=false
spring.freemarker.expose-session-attributes=false
spring.freemarker.expose-spring-macro-helpers=false
# Set the panel suffix
spring.freemarker.suffix=.ftl
# set the maximum memory for a single file
spring.servlet.multipart.max-file-size=100MB
# set maximum memory for all files
spring.servlet.multipart.max-request-size=1000MB
# Customize the file upload path
web.upload-path=/app/images
# Model path
opencv.model-path=/app/model/haarcascade_frontalface_default.xml
Copy the code

The front-end page file has only one index. FTL, please forgive Chen’s poor front-end level, there is only one page in the front-end, you can submit the page, but also show the processing results of the page:

<! DOCTYPE html><head>
    <meta charset="UTF-8" />
    <title>Uploading pictures to Demo</title>
</head><body> <h1 > <form action="fileUpload" method="post" encType ="multipart/form-data"> <input type="file" name="fileName"/></p> <input type="number" value="32" name="minneighbors"/></p> <p> <# MSG?? > <span>${msg}</span><br><br> <#else > <span>${msg! }</span><br> </#if> <# > <#--<img src="/show? fileName=${fileName}" style="width: 100px"/>--> <img src="/show? fileName=${fileName}"/> <#else> <#--<img src="/show" style="width: 200px"/>--> </#if> </body> </html>Copy the code

Back to the code behind the scenes, the most common application startup classes are:

package com.bolingcavalry.facedetect;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class FaceDetectApplication {

    public static void main(String[] args) { SpringApplication.run(FaceDetectApplication.class, args); }}Copy the code

After the front end uploads the image, what does the back end do? Without Posting the code, let’s go through the back-end tasks as shown below:

The core business class UploadController.java, in which the Web interface and business logic are handled, is executed in the same order as above. There are several points to note later:

package com.bolingcavalry.facedetect.controller;

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.ResourceLoader;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.multipart.MultipartFile;

import java.io.File;
import java.io.IOException;
import java.util.Map;
import org.opencv.core.*;
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.imgproc.Imgproc;
import org.opencv.objdetect.CascadeClassifier;

import java.util.UUID;

import static org.bytedeco.javacpp.opencv_objdetect.CV_HAAR_DO_CANNY_PRUNING;

@Controller
@Slf4j
public class UploadController {

    static {
        // Load the dynamic link library
        System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
    }

    private final ResourceLoader resourceLoader;

    @Autowired
    public UploadController(ResourceLoader resourceLoader) {
        this.resourceLoader = resourceLoader;
    }

    @Value("${web.upload-path}")
    private String uploadPath;

    @Value("${opencv.model-path}")
    private String modelPath;

    /** * Jump to the file upload page *@return* /
    @RequestMapping("index")
    public String toUpload(a){
        return "index";
    }

    /** * the last time the file was in the specified directory *@paramThe file file *@paramPath File path *@paramFileName indicates the source fileName *@return* /
    private static boolean upload(MultipartFile file, String path, String fileName){
        // Use the original file name
        String realPath = path + "/" + fileName;

        File dest = new File(realPath);

        // Check whether the file parent directory exists
        if(! dest.getParentFile().exists()){ dest.getParentFile().mkdir(); }try {
            // Save the file
            file.transferTo(dest);
            return true;
        } catch (IllegalStateException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
            return false;
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
            return false; }}/ * * * *@paramFile File to be uploaded *@return* /
    @RequestMapping("fileUpload")
    public String upload(@RequestParam("fileName") MultipartFile file, @RequestParam("minneighbors") int minneighbors, Map<String, Object> map){
        log.info("file [{}], size [{}], minneighbors [{}]", file.getOriginalFilename(), file.getSize(), minneighbors);

        String originalFileName = file.getOriginalFilename();
        if(! upload(file, uploadPath, originalFileName)){ map.put("msg"."Upload failed!");
            return "forward:/index";
        }

        String realPath = uploadPath + "/" + originalFileName;

        Mat srcImg = Imgcodecs.imread(realPath);

        // Target gray image
        Mat dstGrayImg = new Mat();
        // Convert gray
        Imgproc.cvtColor(srcImg, dstGrayImg, Imgproc.COLOR_BGR2GRAY);
        // OpenCv face recognition classifier
        CascadeClassifier classifier = new CascadeClassifier(modelPath);
        // Use to store the face rectangle
        MatOfRect faceRect = new MatOfRect();

        // The minimum size of the feature detection point
        Size minSize = new Size(32.32);
        // The image zoom ratio can be understood as the camera's X lens
        double scaleFactor = 1.2;
        // Perform face detection
        classifier.detectMultiScale(dstGrayImg, faceRect, scaleFactor, minneighbors, CV_HAAR_DO_CANNY_PRUNING, minSize);
        // Walk over the rectangle and draw it on top of the original image
        // Define the color to draw
        Scalar color = new Scalar(0.0.255);

        Rect[] rects = faceRect.toArray();

        // Not detected
        if (null==rects || rects.length<1) {
            // Display the image
            map.put("msg"."No faces detected.");
            / / file name
            map.put("fileName", originalFileName);

            return "forward:/index";
        }

        // process one by one
        for(Rect rect: rects) {
            int x = rect.x;
            int y = rect.y;
            int w = rect.width;
            int h = rect.height;
            // Frame each face individually
            Imgproc.rectangle(srcImg, new Point(x, y), new Point(x + w, y + w), color, 2);
        }

        // Add the name of the image after the face frame
        String newFileName = UUID.randomUUID().toString() + ".png";

        / / save
        Imgcodecs.imwrite(uploadPath + "/" + newFileName, srcImg);

        // Display the image
        map.put("msg"."All detected." + rects.length + "Personal face");
        / / file name
        map.put("fileName", newFileName);

        return "forward:/index";
    }
    /** * display a single image *@return* /
    @RequestMapping("show")
    public ResponseEntity showPhotos(String fileName){
        if (null==fileName) {
            return ResponseEntity.notFound().build();
        }

        try {
            // File must be added, path is the path in the application configuration file
            return ResponseEntity.ok(resourceLoader.getResource("file:" + uploadPath + "/" + fileName));
        } catch (Exception e) {
            returnResponseEntity.notFound().build(); }}}Copy the code

Uploadcontroller.java code has the following considerations:

This is where the most common errors are reported during development. Make sure that the local libraries in the path specified by the -djava.library. path parameter are available. So just make sure the -djava.library.path parameter is configured correctly, as mentioned later in the Dockerfile
The public String Upload method is a code entry for face detection, which is executed in the order of the process analyzed above
The new CascadeClassifier(modelPath) is used to instantiate classifiers based on specified models. The model file is downloaded from GitHub, and the address is github.com/opencv/open…
Seemingly magic face detection function, in fact, only one line of code classifier. DetectMultiScale, can get each face rectangle in the original position, then, let’s just according to the position on the original image with rectangular box

Now that the code is written, let’s make it a Docker image

Docker image making

First, write a Dockerfile:

#The base image integrates OpenJDK8 and OpencV3.4.3The FROM bolingcavalry/opencv3.4.3:0.0.3
#Create a directory
RUN mkdir -p /app/images && mkdir -p /app/model

#Specifies the source location for the content of the mirror
ARG DEPENDENCY=target/dependency

#Copy content to mirror
COPY ${DEPENDENCY}/BOOT-INF/lib /app/lib
COPY ${DEPENDENCY}/META-INF /app/META-INF
COPY ${DEPENDENCY}/BOOT-INF/classes /app

#Specifying start commandsENTRYPOINT [" Java ", "- Djava. If the path = / opencv - rule 3.4.3 / build/lib", "- cp", "app: app/lib / *", "com. Bolingcavalry. Facedetect. FaceDetectAppl ication"]Copy the code

Dockerfile (Dockerfile, Dockerfile) -djava.library. path=/opencv-3.4.3/build/lib specifies the location of the local so library from which the system. loadLibrary is loaded. Let’s use the base image is bolingcavalry/opencv3.4.3:0.0.3, is ready to present in the location of opencv all local library
Execute MVN Clean package -u in the parent project directory. This is a pure Maven operation and has nothing to do with Docker
Go to the face-detect-demo directory and run the following command to extract classes, configuration files, and dependency libraries from jar files to the target/dependency directory:

mkdir -p target/dependency && (cd target/dependency; jar -xf .. /*.jar)Copy the code

Finally, in the docker Dockerfile file directory executable commands build -t bolingcavalry/facedetect: 0.0.1. (the end of the command has a point, don’t leak), can complete mirror
If you have a hub.docker.com account, you can also use the Docker push command to push the image to the central repository for more people to use:
Finally, a quick review of the Three Minute Experience: The command to start docker container in Java version face Detection is as follows: The directory of the host machine is mapped to the container by two -v parameters. Therefore, /app/images and /app/model in the container can remain unchanged as long as the correct directory mapping of the host machine can be guaranteed:

docker run \ --rm \ -p 18080:8080 \ -v /root/temp/202107/17/images:/app/images \ -v / root/temp / 202107/17 / model: / app/model/bolingcavalry/facedetect: 0.0.1Copy the code

For more information about SpringBoot’s official recommended Docker image making, please see SpringBoot(2.4) Application Making Docker Images (Gradle version official Solution).

What needs to be paid attention to

XML and JavacV libraries are not compatible with each other. If you want to change them, check the maven central repository to see if the version you need exists.
At this point, “Java version of face detection” from experience to development details are completed, small functions involve a lot of knowledge, but also let us experience the convenience and power of JavACV, with the help of Docker environment configuration and application development separated, It reduces the difficulty of application development and deployment (no longer spending time on JDK and OpencV deployment). If you are looking for an easy-to-use JavacV development and deployment solution, I hope this article will provide you with a reference.

You are not alone, Xinchen original accompany all the way

Java series
Spring series
The Docker series
Kubernetes series
Database + middleware series
The conversation series

Welcome to pay attention to the public number: programmer Xin Chen

Wechat search “programmer Xin Chen”, I am Xin Chen, looking forward to enjoying the Java world with you…

Github.com/zq2599/blog…

Java version of face detection in detail: coding

Welcome to my GitHub

This paper gives an overview of

Version information

Download the source code

coding

Docker image making

What needs to be paid attention to

You are not alone, Xinchen original accompany all the way

Welcome to pay attention to the public number: programmer Xin Chen

Related Posts

GO to God: Deep defer (continued) | GO Theme Month

Use SSH to log in to the server without password on the CLI

The end of 2020: interview preparation (mentality + resume) + interview topics + interview experience!