One, foreword

Matting is a physical job.

The hardest part of matting is dealing with hair.

Ever think that one day, algorithms will automatically matting for you? Meticulous to the kind of hair!

Hair segmentation is not a problem!

Image Matting algorithm, you deserve it.

Same old rule. Today, we’re holding hands.

Algorithm principle, environment construction, effect realization, one-stop service, all in the following!

Second, the Animal Matting

The core of matting problem is to accurately estimate the foreground of the image or video, which is of great significance to image editing and film editing.

The recently published paper end-to-end Animal Image Matting can achieve end-to-end Image Matting with only one Image and no prior knowledge, and the effect is very amazing.

In this paper, a matting model named GFM is proposed, which can generate both global semantic segmentation and local alpha mask.

At the same time, the first natural animal image keening dataset AM-2K is open source, and the RSSN synthesis method based on high resolution background dataset BG-20K is designed.

The GFM network structure is as follows:

Network structure: is the structure of a codec in which the encoder is shared by two parallel decoders.

Shared encoder: ResNET-34 or densenet-121 pre-trained on ImageNet as encoder.

Glance Decoder (GD) : Used to learn high-level semantic information. After the fourth module of the encoder, the pyramid pooling module (PPM) outputs the global context for GD.

Focus Decoder (FD) : Used to extract details from low structural features. After the fourth module of the encoder, the bridge module (BB) is added to converge local contexts in different domains. Combined with U-NET, FD and the corresponding module of the encoder are connected by jumping to train FD.

Finally, the output results of GD and FD are connected with different characterization domains.

Gfm-tt: Class 3 Trimap T of real alpha mask expansion and corrosion is used as GD monitor signal, and alpha mask of unknown transition domain is used as FD monitor signal.

Gfm-ft: Two types of foreground segmentation mask are used as GD supervision signal, and the unknown transition domain alpha mask is used as FD supervision signal.

Gfm-bt: Two types of background segmentation mask were used as GD supervision signal, and the unknown transition domain alpha mask was used as FD supervision signal.

Finally, through collaborative matting (CM), the results of the above three different representation domains were combined to obtain the final alpha prediction.

For more details, please refer to paper:

Address: arxiv.org/pdf/2010.16…

Three, the effect test

Github project address: github.com/JizhiziLi/a…

Step 1: Set up the test environment.

It’s easy to install the dependency libraries according to requirements.txt.

Step 2: Download the trained model weight file.

Download address (need to cross the wall) : click to view

Step 3: In the project directory, run the program.

python ./core/test_samples.py --cuda --arch="e2e_resnet34_2b_gfm_tt" --model_path="models/model_r34_2b_gfm_tt.pth" --pred_choice=3 --hybrid
Copy the code

The Original directory under the Samples directory saves the original images;

The result_alpha directory under the samples directory saves the split mask;

The result_color directory under the samples directory holds the extracted results.

I will be procedures and weight files have been packaged, too much trouble, you can download direct use.

Download address (extraction code: S6uh) :

Pan.baidu.com/s/1xjBbj3ip…

Operation effect:

Fast running speed, running effect display:

Four, the last

The algorithm is only for animals. If you want to matting people, you need to make your own data sets and training models.

This article will be updated continuously. You can find it on our wechat official account by searching [JackCui-ai]. GitHub github.com/Jack-Cheris… Welcome Star.