• 🥇 Copyright: this article by [Mohli AI] original, thank you
  • ❤️ if the article is helpful to you, welcome to three links
  • 🍖 This blog post is designed to take you through how to use deep learning for image restoration [rain, noise and blur removal], [❤️ benchmarking, tutorial ❤️]

    • 📔 Basic Information

    A paper on multi-stage progressive image restoration applied to blur, rain and noise removal

    • Multi-Stage Progressive Image Restoration (CVPR 2021)
    • Multi-stage progressive image restoration
    • Arxiv.org/abs/2102.02…
    • Github.com/swz30/MPRNe…

    🔴 Abstract

    Image restoration tasks require a complex balance between spatial details and high-level context information when restoring images. In this paper, we propose a novel collaborative design that best balances these competing goals. Our main proposal is a multi-stage architecture that gradually learns the recovery function of degraded inputs, thereby breaking the entire recovery process down into more manageable steps. Specifically, our model first learns context features using an encoder-decoder architecture and then combines them with high-resolution branching that preserves local information. At each stage, we introduce a novel per-pixel adaptive design that uses in situ supervised attention to re-weight local features. A key element of this multi-phase architecture is the exchange of information between the different phases. To this end, we propose a bidirectional approach in which information is not only exchanged sequentially from early to late, but also there are horizontal connections between feature processing blocks to avoid any information loss. The resulting tightly interconnected multi-phase architecture, called MPRNet, provides powerful performance improvements across ten data sets across a range of tasks including image rain, blur and noise removal.

    🔵 The main contributions

    • A novel multi-stage approach that generates context-rich and spatially accurate output; Due to its multi-stage nature, our framework breaks challenging image restoration tasks into sub-tasks to progressively recover degraded images
    • An efficient supervised attention module that makes full use of the recovered image to refine incoming features at each stage before further propagation
    • Strategies for aggregating multi-scale features across stages
    • We demonstrated the effectiveness of our MPRNet by achieving excellent results on 10 synthetic and real-world datasets for a variety of recovery tasks, including image rain, blur and noise removal, while maintaining low complexity (see Figure 1). In addition, we provide detailed ablation, qualitative results and generalization tests.

    🔴 Paper experimental data set

    Image Deblurring

    • GoPro [53] dataset — download home page
    • Home page details are as follows

    Image Deraining

    • Rain14000 [23] Rain1800 [81] Rain800 [90]
    • Rain100H [81], Rain100L [81], Test100 [90], Test2800 [23], and Test1200 [89]

    Image Denoising

    • SIDD dataset [1] – home page: https://www.eecs.yorku.ca/~kamel/sidd/dataset.php
    • SIDD test data set – home page: https://www.eecs.yorku.ca/~kamel/sidd/benchmark.php
    • DND Benchmark Dataset [60] — Darmstadt Noise Dataset — Home: http://noise.visinf.tu-darmstadt.de/benchmark/

    🔵 Conclusion

    In this work, we propose a multi-stage image recovery architecture that progressively improves degraded inputs by injecting oversight at each stage. We set guidelines for our design, requiring complementary feature processing at multiple stages and flexible information exchange between them. To this end, we propose contextually-enriched and Spatially accurate stages to encode a set of different features uniformly. To ensure synergies between interaction stages, we propose cross-stage feature fusion and attention-guided output exchange from early stage to late stage. Our model achieved significant performance improvements over a number of benchmark data sets. In addition, our model is lightweight in model size and efficient in run time, which makes sense for devices with limited resources.

    📕 Environment Construction

    • Server: Ubuntu1-18.04 Quadro RTX 5000 16G
    • CUDA version V10.0.130

    The basic environment is set up as follows

    Conda create -n Torch11 python=3.7.5 Conda activate Torch11 PIP install Torch ==1.1.0 PIP install TorchVision ==0.3.0Conda is recommended for installation.Conda install Pytorch ==1.1.0 TorchVision ==0.3.0 CUDatoolKit =10.0 -C Pytorch## or (depending on your Cuda version)Conda install Pytorch =1.1 TorchVision =0.3 cudatoolkit=9.0 -c Pytorch PIP install matplotlib scikit-image opencv-python yacs joblib natsort h5py tqdmCopy the code

    Install warmup scheduler

    conda activate torch11

git clone https://github.com/swz30/MPRNet.git

cd MPRNet-main

cd pytorch-gradual-warmup-lr/

python setup.py install

Copy the code

    The output is as follows

    running install running bdist_egg running egg_info ... . Installed/home/moli anaconda3 / envs torch11 / lib/python3.6 / site - packages/warmup_scheduler - 0.3 - py3.6. Egg Processing dependenciesforWarmup - the scheduler = = 0.3 Finished processing dependenciesforWarmup - the scheduler = = 0.3Copy the code

    📗 Pre-training model and test data

    🟧 Pre-training model

    🟨 Set test data

    • Defuzzing: here temporarily use GOPRO_Large\test\GOPR0384_11_00 as the test data, placed in

    The samples/directory

    • Rain: Rain100H /
    • De-blurring: One gaussian noise and one pulse noise image each, placed under the samples/noise/ directory
    • The project directory structure is as follows

    • The test data set used by the blog post is set as follows

    📘 test

    🔴 go fuzzy-testing

    python demo.py --task Deblurring --input_dir ./samples/GOPR0384_11_00/blur --result_dir ./samples/output/
Copy the code

    The output is as follows

    The GPU occupies 10209MiB

Files saved at ./samples/output/

Copy the code

    Contrast effect of input and output

    The deblurred image is compared with the clear image corresponding to the test image in the data set.

    🔵 go rain test

    cmd

    python demo.py --task Deraining --input_dir ./samples/Rain100H/input/ --result_dir ./samples/Rain100H/output/

Copy the code

    The output is as follows

    The GPU occupies 1461MiB

Files saved at ./samples/Rain100H/output/

Copy the code

    Contrast effect of input and output

    🔵 Denoising test

    python demo.py --task Denoising --input_dir ./samples/noise/ --result_dir ./samples/denoise

Copy the code

    The output is as follows

    The GPU occupies 2087MiB

Files saved at ./samples/denoise
Copy the code

    The de-noise effect is as follows

    📙 blog run code + model + test data sharing

    There is a need for the source partners, search attention to the blog of the same name public number, public number background, reply 20211120 can automatically obtain this blog running code + model + test data sharing + corresponding papers

    20211120
Copy the code

    🚀🚀 recommend 🚀🚀

  • 👋 If you feel the article is not satisfied, but also want to go further, then you can come to my other columns to see oh ~
    • ❤️ image style conversion – code environment to build combat tutorial [attention can read]!
    • 💜 image repair – code environment construction – knowledge summary combat tutorial [is said to be ok]
    • 💙 super partition reconstruction – code environment construction – knowledge summary solution to how to make white Moonlight clearer
    • 💛 YOLO column, only actual combat, unreasonable target detection [recommended collection]!
    • 🎻 Cuda series Linux installation tutorial
    • 💜 ubuntu18 cuda11.2 graphic tutorials to the current user install | configuration cuDNN8.1 |
    • 💜 Install CUDA10.0 for the current user on the Linux server
    • 💜 Pytorch and Torchvision are installed in Cuda10.0
    • 💜 Linux can install multiple versions of the Cuda? | give me a new server, what I would arrange Cuda
    • 💜 View the versions of CUDA and cuDNN
    • 🍖 Deep learning, adjusting and moving bricks, developing sharp tools
    • ❤ ️ VSCode Linux server for remote connection development debug | to | | Python remote debugging c + + remote debugging VSCode artifact, you’re worth it
    • ❤ ️ Windows conda pycharm Linux server configuration under the environment of the strongest guide | deep learning model of remote debugging | | SFTP synchronous transmission

    ❤️ Life is too short. Welcome to learn AI 💜 with Murray