Evaluation of two-stage Progressive Image Repair Effect [CVPR 2021]

“This is the second day of my participation in the First Challenge 2022.

🥇 copyright: this article by [Moneo-ai] original, you big guy, one read, please refer to

🎉 statement: as one of the bloggers in AI field, ❤️ lives up to your time and your ❤️

🍖 This blog post is intended to boast about image restoration and is only a brief introduction to the work of this paper

📔 Basic Information

Image Inpainting with External-internal Learning and Monochromic Bottleneck
Image repair with in-out learning and monochromatic bottlenecks
Github.com/Tengfei-Wan…
Arxiv.org/abs/2104.09…

The translation

Although recent repair methods have demonstrated significant improvements in deep neural networks, they still have artifacts such as blunt structures and sudden colors when filling in missing areas. To solve these problems, we propose an external and internal restoration scheme with monochromatic bottleneck, which can help the image restoration model eliminate these artifacts. In the external learning phase, we reconstruct the missing structures and details in the monochrome space to reduce the learning dimension. In the internal learning stage, we propose a novel approach to internal color propagation, using a progressive learning strategy to restore consistent color. A large number of experiments show that our proposed scheme helps the image repair model to produce more structurally preserved and visually compelling results.

The main contributions can be summarized as follows:

To our knowledge, we are the first company to introduce an external-internal learning approach to deep image repair. It learns semantic knowledge externally by training large data sets, while making full use of internal statistics of a single test image.
We designed a progressive internal image shader network that achieved excellent shader performance in our case.
We extend our proposed method to several deep repair models and observe significant improvements in visual quality and model generalization ability on multiple datasets.

Conclusion

In this paper, we propose a generic external-internal learning fix with a monochromatic bottleneck.

It first reconstructs monochrome using semantic knowledge learned from outside a large data set and then restores color from within a single test image. Compared with previous methods, our method can produce more coherent structure and more visually coordinated colors.

A large number of experiments show that our method can be improved qualitatively and quantitatively on several backbone models. The main limitation of our method is speed of reasoning. Our method is slower than the most advanced methods because of the additional stages required for coloring.

In the future, we plan to further accelerate the coloring process and extend the proposed scheme to other low-level visual tasks, such as super-resolution.

📕 Environment Construction

Dependency libraries are neat

Python 3.6
Pytorch 1.6
Numpy

Installed pytorch recommended reference — Linux cuda10.0 pytorch and Torchvision | shorthand

Conda create -n torch16 python=3.6.6 conda activate Torch16# CUDA 10.1Conda Install PyTorch ==1.6.0 TorchVision ==0.7.0 CUDatoolKit =10.1 -D PyTorch PIP install Pillow ==5.2.0 PIP install opencv-python pip install scikit-image pip install scipy pip install thopCopy the code

📗 source code test

For now, the code is very concise and can be run directly from the official readMe

🔵 Stage 1: Colorization

git clone https://github.com/Tengfei-Wang/external-internal-inpainting.git

cd external-internal-inpainting

conda activate torch16
Copy the code

Colorization [coloring method test command]

python main.py  --img_path images/input2.png --gray_path images/gray2.png  --mask_path images/mask2.png  --pyramid_height 3
Copy the code

The output is as follows

starting colorization. Scale 0
starting colorization. Scale 1
starting colorization. Scale 2

Copy the code

The best results are as follows

The source code analysis for this stage is as follows

🔵 Second phase: Reconstruction

Blind guessing: This means that coloring the restored image instead of other backbones’ input will make the restoration better. The official has not made further connection, here is not specific test;

Bit by bit my humble opinion, hope big guy give directions

📘 Rendering of the paper

Interested in detailed classification of image restoration, please refer to the following blog post

Image Inpainting Based on Deep Learning – A Review

🔴 Target removal

🔵 Irregular Mask fix

There is a reference to cross data set evaluation.

Direct understanding: Models trained on Places2 are tested on DTD datasets

🔴 User guide repair

🚀🚀 So far the gold digging platform has created the following classic blog post 🚀🚀

Computer vision field, classic blog post

🍊 Use AI to convert photos of good friends into pencil sketches — 🍊2020 U2Net🍊
🍊 NiceGAN environment to build, style migration (with source) | 【 2020 】 CVPR
🍊 multi-stage progressive image restoration – go rain, denoising, fuzzy – effective tutorial (with source) | 【 2021 】 CVPR
🍊 Image inpainting Based on Deep Learning – A Review
🍊 graduation thesis, academic paper writing basic skills and experience – one read
🍊 LaTeX2021 formula preparation, graphic installation, detailed tutorial, a read

AI learning and deep learning environment construction

🍊 # Ubuntu install CUDa11.2 for current users
🍊 # Linux and Windows setup PIP image source – the most practical machine learning library download acceleration setup
🍊 # Specify the GPU to run and train Python programs, deep learning single card, multi-card training GPU Settings
🍊 # Install Pytorch and Torchvision in Cuda10.0 for Linux

🚀🚀 Mexic AI

🎉 as one of the bloggers with the most dry goods in the field of AI, ❤️ lives up to his time and qing ❤️

❤️ If the article is helpful to you, like, comment encourage bloggers every minute to seriously create