algorithm

Background Matting: The World is Your Green Screen (2020)

[Background-Matting] [Paper] [Project]

classification

  • Traditional approaches
    • simpling-based techniques
    • propagation-based techniques
  • Learning-based approaches
    • trimap based methods
      • Context Aware matting (CAM)
      • Index Matting (IM)
      • .
    • automatic matting algorithm
      • Late Fusion Matting (LFM)
      • .
  • Matting with known natural background
  • Video Matting

history

  • 2019
    • Disentangled image matting
    • Context-aware image matting for simultaneous foreground and alpha estimation
    • Learning to index for deep image matting
    • A late fusion cnn for digital matting
  • 2018
    • Semantic soft segmentation
    • Encoder-decoder with atrous separable convlution for semantic image segmentation
    • Semantic human matting
    • Alpha-gan: Generative adversarial networks for natural image matting
  • 2017
    • Designing effective inter-pixel information flow for natural image matting
    • Deep image matting
    • Fast deep matting for portrait animation on mobile phone
  • 2016
    • Natural image matting using deep convolutional neural networks
    • Deep automatic portrait matting
  • 2013
    • KNN matting
  • 2011
    • A global sampling method for alpha matting
    • Nonlocal matting
  • 2010
    • Shared sampling for real-time alpha matting
    • Fast matting using large kernel matting laplacian matrics
  • 2008
    • Spectral matting
  • 2007
    • A closed-form solution to natural image matting
  • 2004
    • A bayesian approach to digital matting

The network structure

At the core of our approach is a deep matting network G that extracts foreground color and alpha for a given input frame, augmented with background, soft segmentation, and (optionally nearby video frames), and a discriminator network D that guides the training to generate realistic results.

Loss function

Attention-Guided Hierarchical Structure Aggregation for Image Matting (2020)

[CVPR2020-HAttMatting] [Attention-Guided Hierarchical Structure Aggregation for Image Matting]

A Late Fusion CNN for Digital Matting(2019)

[FusionMatting]

[A Late Fusion CNN for Digital Matting]

[A Late Fusion CNN for Digital Matting]

[CVPR2019][A Late Fusion… Matting]]

[Clarification of misconceptions – Response to CVPR 2019 LFM Paper challenge]

[Alibaba-Zhejiang University Foreword Technology Joint Research Center]

LFM is an end-to-end neural network with inputs containing images of the foreground and outputs as alpha masks of the foreground. Neural network is used to predict three graphs: foreground probability graph, background probability graph and mixed weight graph. According to the mixed weight graph, the foreground probability graph and background probability graph are fused to obtain alpha mask. The networks that need training include segmented network pretraining, converged network pretraining and end-to-end joint training, and the training loss is added to the output alpha mask.

Natural Image Matting via Guided Contextual Attention(2020)

[GCA-Matting] [Natural Image Matting via Guided Contextual Attention]

The network structure

  • GCA

Loss function

Deep Image Matting (2017)

[pytorch-deep-image-matting]

[Deep Image Matting]

[Project]

Deep Image matting(and implementation details)

The data set

[1] Matting is a method for soft segmentation of foreground and background. The goal is to find out the degree of fusion between foreground and background.

Note: Trimaps are generally generated by matte extensions

summary

[Portrait segmentation unreliable summary [1]]

challenge

[Alpha Matting Evaluation Website]

Evalution

  • SAD(sum of absolution difference)
  • MSE(mean square error)

References