Click on “Machine Learning and Generative Adversarial Networks” above to follow the “Stars”
Get fun, fun cutting edge dry goods!
This day, you stretch, yawn, looking out of the window, feeling the time is gone, red cherry green banana…… Suddenly, the mobile phone on the desktop came a wechat vibration sound, you are extremely impatient to go over.
“Pig, I saw a beautiful woman in the supermarket!”
In the face of Lao Tie’s ignorance, you snap your fingers and fly:
“What else will you see? Besides, as far as your taste is concerned…”
“Wait a minute! …”
“What are you doing?”
Soon, a picture came from the other side:
“You secretly take somebody else really good. What about the face? …”
Then the phone lights up again:
“I just cut off the irrelevant part. Let’s cut off the face
“
“????? …”
“It’s a little far away, maybe a little small, like it’s hard to see…”
The body of the quote
SRGAN:The highly challenging task of estimating a highresolution (HR) image from its low resolution (LR) counterpart is referred to as super-resolution (SR).
Image super-resolution, referred to as super-partition SR, generally refers to the amplification resolution, for example, changing 256X256 resolution to 512X512 resolution, when the amplification factor is 2. Obviously, this is a nothing, to fill the pixel ill-posed problem, there is no unique solution. Image super sub, the application scene is extensive naturally. The general method is to take the low-resolution image LR as the input of the method and process it to obtain the high-resolution HR image.
It is worth noting, however, that matching paired datasets are extremely difficult to obtain in real-world scenarios. Nowadays, a considerable number of papers are self-made such LR-HR image pairs as training sets. For example, the original graph HR is first sampled to obtain LR, and then the LR to HR mapping learning is carried out. However, in real practice, is the relationship between LR and HR the relationship of “downsampling” that we think we are? This is probably unknown and difficult to simulate, and artificial downsampling or other artificial methods are just wishful thinking. There may be more caution in medical image SR. \
Today’s collation is the combination of GAN to generate adversarial network image hyperfraction. First of all, this paper summarizes two representative and famous supersplitting GAN namely SRGAN and ESRGAN, and about one paper that uses network to collect small resolution data, and finally gives more than 70 papers that combine GAN to do supersplitting!! I hope to give a reference to students who are interested in exploring and understanding this aspect!
(More than 70 papers have been downloaded and packaged, and the way to obtain them is to enter the background of the public account and reply to [super points GAN].)
1. (2017-05-25) (SRGAN) Photo-realistic Single Image super-resolution Using a Generative Adversarial Network
Arxiv. Xilesou. Top/PDF / 1609.04…
Despite the breakthrough in accuracy and speed of single-image super-resolution using faster and deeper convolutional neural networks, a focus problem remains largely unsolved: how to recover finer texture details when super-resolution images are acquired at larger magnifications? Previous work has focused on mean square deviation reconstruction, using PSNR and so on in the evaluation of results, but it is often lacking in high frequency detail and visually unsatisfactory. In this paper, SRGAN, the first generative adversation network (GAN) for image super resolution (SR), is proposed to be able to deduce 4 times realistic natural images. In order to achieve this goal, a perceptual loss function is proposed, which includes antagonism loss and content loss. Using content loss based on perceived similarity does away with measuring similarity in pixel space. The average opinion score (MOS) demonstrates the excellent performance of the method.
As shown in the figure below, the comparison of the superfraction method with magnification of 4 times. The first one is bicubic interpolation, the second one is convolutional neural network driven by mean-square error loss, the third one is SRGAN, and the last one refers to the original graph.
Optimization:
Loss function:
Generator loss (the author calls the total generator loss perceptual loss: content loss + generator versus loss) :
Content loss:
Generators against losses:
The author has done a lot of ablation research, which is not described here.
Finally, the first result of the experiment. It was the scene of the confrontation between large SSIM and PSNR. SRGAN is inferior to SRResNet in PSNR and SSIM, but it is enough to beat SRGAN in MOS, that is, human eye observation.
2. (2018-09-17) ESRGAN Enhanced Super-Resolution Generative Adversarial Networks
Arxiv. Xilesou. Top/PDF / 1809.00…
SRGAN was groundbreaking work. However, the details are still not satisfactory, so we further study the three key components of SRGAN: network architecture, loss resistance and perceived loss, and improve them into enhanced SRGAN (ESRGAN). In particular, the residue-in-residual Dense Block (RRDB) without BN batch normalization was introduced as the basic network building unit. Moreover, the idea of relative GAN is used to make the discriminator predict relatively true. Finally, by using pre-activation features to perform perceptual loss calculations, the goal is to provide stronger oversight of brightness consistency and texture recovery. Thanks to these improvements, ESRGAN has better visual quality and more realistic natural textures than SRGAN and won the first place in the PIRM2018-SR Challenge.
Improvements in network structure:
As BN has positive effects in coarse-grained task classification, it is not suitable to use batch statistics in tasks with distinctive characteristics of single image, such as style transfer, otherwise it is easy to weaken the inherent details of single image. Therefore, the author tried to remove BN, but this would easily lead to difficulties in network training, and adopted the Dense Block, a structure that is easier to improve network performance.
Improvements in countermeasures:
The design idea of relative GAN is referenced.
Against loss:
A rough derivation:
Original GAN:
Improvement of perceived Loss:
Losses were calculated using the characteristics prior to relU activation. Such features can contain richer and more detailed response information.
Using network interpolation:
GAN is too “loose” and some of the details may not be natural. However, previous convolutional networks based on MSE optimization tend to smooth and fuzzy and lose details. Therefore, network interpolation proposes a method to synthesize the two networks: first, train a conventional superpartite network, then fine-tuning the generator of GAN on the basis of this network, and then add the parameters of the two networks in weight:
As shown in the figure below, a more preferred or balanced intermediate effect can be found by adjusting alpha.
3. (2018-07-30) To learn image super-resolution use a GAN to learn how to do image degradation first
Arxiv. Xilesou. Top/PDF / 1807.11…
As mentioned above, in the training of hypersegmentation, low resolution images are artificially generated by simple bilinear downsampling (in a few cases, fuzzy first and then downsampling), and then hypersegmentation is performed on them. But in real life, this approach doesn’t work very well.
Therefore, a two-stage process is proposed. Firstly, a high-to-low GAN is trained to learn how to downsample high-resolution images. During the training process, only unpaired high-resolution and low-resolution images are required. Once this part is implemented, the output of the network can be used to train a low-to-high GAN to achieve super-resolution reconstruction, this time using paired low-resolution and high-resolution images. Our main result is that this network can effectively improve the quality of low-resolution images in the real world. In this paper, this method is applied to face super-resolution problem and its validity is verified. It may also be applicable to other image object classes.
Experimental results:
001 (2020-03-4) Turbulence Enrichment using Generative Adversarial Networks
Arxiv. Xilesou. Top/PDF / 2003.01…
002 (2020-03-2) MRI Super-Resolution with GAN and 3D Multi-Level DenseNet Smaller Faster and Better
Arxiv. Xilesou. Top/PDF / 2003.01…
003 (2020-02-29) Joint Face Completion and Super-resolution using Multi-scale Feature Relation Learning
Arxiv. Xilesou. Top/PDF / 2003.00…
004 (2020-02-21) Generator From Edges Reconstruction of Facial Images
Arxiv. Xilesou. Top/PDF / 2002.06…
005 (2020-01-22) Optimizing Generative Adversarial Networks for Image Super Resolution via Latent Space Regularization
Arxiv. Xilesou. Top/PDF / 2001.08…
006 (2020-01-21) Adaptive Loss Function for Super Resolution Neural Networks Using Convex Optimization Techniques
Arxiv. Xilesou. Top/PDF / 2001.07…
007 (2020-01-10) Segmentation and Generation of Magnetic Resonance Images by Deep Neural Networks
Arxiv. Xilesou. Top/PDF / 2001.05…
008 (2019-12-15) Image Processing Using Multi-Code GAN Prior
Arxiv. Xilesou. Top/PDF / 1912.07…
009 (2020-02-6) Quality analysis of DCGAN-generated mammography lesions
Arxiv. Xilesou. Top/PDF / 1911.12…
010 (2019-12-19) A deep learning framework for morphologic detail beyond the diffraction limit in infrared spectroscopic imaging
Arxiv. Xilesou. Top/PDF / 1911.04…
011 (2019-11-8) Joint Demosaicing and Super-Resolution (JDSR) Network Design and Perceptual Optimization
Arxiv. Xilesou. Top/PDF / 1911.03…
012 (2019-11-4) FCSR-GAN Joint Face Completion and Super-resolution via Multi-task Learning
Arxiv. Xilesou. Top/PDF / 1911.01…
013 (2019-10-9) Wavelet Domain Style Transfer for an Effective Perception-distortion Tradeoff in Single Image Super-Resolution
Arxiv. Xilesou. Top/PDF / 1910.04…
014 (2020-02-3) Optimal Transport CycleGAN and Penalized LS for Unsupervised Learning in Inverse Problems
Arxiv. Xilesou. Top/PDF / 1909.12…
015 (2019-08-26) RankSRGAN Generative Adversarial Networks with Ranker for Image Super-Resolution
Arxiv. Xilesou. Top/PDF / 1908.06…
016 (2019-07-24) Progressive Perception-Oriented Network for Single Image Super-Resolution
Arxiv. Xilesou. Top/PDF / 1907.10…
017 (2019-07-26) Boosting Resolution and Recovering Texture of micro-CT Images with Deep Learning
Arxiv. Xilesou. Top/PDF / 1907.07…
018 (2019-07-15) Enhanced generative adversarial network for 3D brain MRI super-resolution
Arxiv. Xilesou. Top/PDF / 1907.04…
019 (2019-07-5) MRI Super-Resolution with Ensemble Learning and Complementary Priors
Arxiv. Xilesou. Top/PDF / 1907.03…
020 (2019-11-25) Image-Adaptive GAN based Reconstruction
Arxiv. Xilesou. Top/PDF / 1906.05…
021 (2019-06-13) A Hybrid Approach Between Adversarial Generative Networks and Actor-Critic Policy Gradient for Low Rate High-Resolution Image Compression
Arxiv. Xilesou. Top/PDF / 1906.04…
022 (2019-06-4) A Multi-Pass GAN for Fluid Flow Super-Resolution
Arxiv. Xilesou. Top/PDF / 1906.01…
023 (2019-05-23) Generative Imaging and Image Processing via Generative Encoder
Arxiv. Xilesou. Top/PDF / 1905.13…
024 (2019-05-26) Cross-Resolution Face Recognition via Prior-Aided Face Hallucination and Residual Knowledge Distillation
Arxiv. Xilesou. Top/PDF / 1905.10…
025 (2019-05-9) 3DFaceGAN Adversarial Nets for 3D Face Representation Generation and Translation
Arxiv. Xilesou. Top/PDF / 1905.00…
026 (2019-08-27) Super-Resolved Image Perceptual Quality Improvement via Multi-Feature Discriminators
Arxiv. Xilesou. Top/PDF / 1904.10…
027 (2019-03-28) SRDGAN learning the noise prior for Super Resolution with Dual Generative Adversarial Networks
Arxiv. Xilesou. Top/PDF / 1903.11…
028 (2019-03-21) Bandwidth Extension on Raw Audio via Generative Adversarial Networks
Arxiv. Xilesou. Top/PDF / 1903.09…
029 (2019-03-6) DepthwiseGANs Fast Training Generative Adversarial Networks for Realistic Image Synthesis
Arxiv. Xilesou. Top/PDF / 1903.02…
030 (2019-02-28) A Unified Neural Architecture for Instrumental Audio Tasks
Arxiv. Xilesou. Top/PDF / 1903.00…
031 (2019-02-28) Two-phase Hair Image Synthesis by Self-Enhancing Generative Model
Arxiv. Xilesou. Top/PDF / 1902.11…
032 (2019-10-23) GAN-based Projector for Faster Recovery with Convergence Guarantees in Linear Inverse Problems
Arxiv. Xilesou. Top/PDF / 1902.09…
033 (2019-02-17) Progressive Generative Adversarial Networks for Medical Image Super resolution
Arxiv. Xilesou. Top/PDF / 1902.02…
034 (2019-01-31) Compressing GANs using Knowledge Distillation
Arxiv. Xilesou. Top/PDF / 1902.00…
035 (2019-01-18) Generative Adversarial Classifier for Handwriting Characters Super-Resolution
Arxiv. Xilesou. Top/PDF / 1901.06…
036 (2019-01-10) How Can We Make GAN Perform Better in Single Medical Image Super-Resolution A Lesion Focused Multi-Scale Approach
Arxiv. Xilesou. Top/PDF / 1901.03…
037 (2019-01-9) Detecting Overfitting of Deep Generative Networks via Latent Recovery
Arxiv. Xilesou. Top/PDF / 1901.03…
038 (2018-12-29) Brain MRI super-resolution using 3D generative adversarial networks
Arxiv. Xilesou. Top/PDF / 1812.11…
039 (2019-01-13) Efficient Super Resolution For Large-Scale Images Using Attentional GAN
Arxiv. Xilesou. Top/PDF / 1812.04…
040 (2019-12-24) Learning Temporal Coherence via Self-Supervision for GAN-based Video Generation
Arxiv. Xilesou. Top/PDF / 1811.09…
041 (2018-11-20) Adversarial Feedback Loop
Arxiv. Xilesou. Top/PDF / 1811.08…
042 (2018-11-1) Bi-GANs-ST for Perceptual Image Super-resolution
Arxiv. Xilesou. Top/PDF / 1811.00…
043 (2018-10-15) Lesion Focused Super-Resolution
Arxiv. Xilesou. Top/PDF / 1810.06…
044 (2018-10-15) Deep learning-based super-resolution in coherent imaging systems
Arxiv. Xilesou. Top/PDF / 1810.06…
045 (2018-10-10) Image Super-Resolution Using VDSR-ResNeXt and SRCGAN
Arxiv. Xilesou. Top/PDF / 1810.05…
046 (2019-01-28) Multi-Scale Recursive and Perception-Distortion Controllable Image Super-Resolution
Arxiv. Xilesou. Top/PDF / 1809.10…
047 (2018-09-2) Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networks
Arxiv. Xilesou. Top/PDF / 1809.00…
048 (2018-09-17) ESRGAN Enhanced Super-Resolution Generative Adversarial Networks
Arxiv. Xilesou. Top/PDF / 1809.00…
049 (2018-09-6) CT Super-resolution GAN Constrained by the Identical Residual and Cycle Learning Ensemble(GAN-CIRCLE)
Arxiv. Xilesou. Top/PDF / 1808.04…
050 (2018-07-30) To learn image super-resolution use a GAN to learn how to do image degradation first
Arxiv. Xilesou. Top/PDF / 1807.11…
051 (2018-07-1) Performance Comparison of Convolutional AutoEncoders Generative Adversarial Networks and Super-Resolution for Image Compression
Arxiv. Xilesou. Top/PDF / 1807.00…
052 (2018-12-19) Generative Adversarial Networks and Perceptual Losses for Video Super-Resolution
Arxiv. Xilesou. Top/PDF / 1806.05…
053 (2018-08-22) cellSTORM – Cost-effective Super-Resolution on a Cellphone using dSTORM
Arxiv. Xilesou. Top/PDF / 1804.06…
054 (2018-04-10) A Fully Progressive Approach to Single-Image Super-Resolution
Arxiv. Xilesou. Top/PDF / 1804.02…
055 (2018-07-18) Maintaining Natural Image Statistics with the Contextual Loss
Arxiv. Xilesou. Top/PDF / 1803.04…
056 (2018-06-9) Efficient and Accurate MRI Super-Resolution using a Generative Adversarial Network and 3D Multi-Level Densely Connected Network
Arxiv. Xilesou. Top/PDF / 1803.01…
057 (2018-05-28) tempoGAN A Temporally Coherent Volumetric GAN for Super-resolution Fluid Flow
Arxiv. Xilesou. Top/PDF / 1801.09…
058 (2018-10-3) High-throughput high-resolution registration-free generated adversarial network microscopy
Arxiv. Xilesou. Top/PDF / 1801.07…
059 (2017-11-28) Super-Resolution for Overhead Imagery Using DenseNets and Adversarial Learning
Arxiv. Xilesou. Top/PDF / 1711.10…
060 (2019-10-3) The Perception-Distortion Tradeoff
Arxiv. Xilesou. Top/PDF / 1711.06…
061 (2017-11-7) Tensor-Generative Adversarial Network with Two-dimensional Sparse Coding Application to Real-time Indoor Localization
Arxiv. Xilesou. Top/PDF / 1711.02…
062 (2017-11-7) ZipNet-GAN Inferring Fine-grained Mobile Traffic Patterns via a Generative Adversarial Neural Network
Arxiv. Xilesou. Top/PDF / 1711.02…
063 (2017-10-19) Generative Adversarial Networks An Overview
Arxiv. Xilesou. Top/PDF / 1710.07…
064 (2018-05-21) Retinal Vasculature Segmentation Using Local Saliency Maps and Generative Adversarial Networks For Image Super Resolution
Arxiv. Xilesou. Top/PDF / 1710.04…
065 (2018-11-28) Simultaneously Color-Depth Super-Resolution with Conditional Generative Adversarial Network
Arxiv. Xilesou. Top/PDF / 1708.09…
066 (2017-06-20) Perceptual Generative Adversarial Networks for Small Object Detection
Arxiv. Xilesou. Top/PDF / 1706.05…
067 (2017-05-7) A Design Methodology for Efficient Implementation of Deconvolutional Neural Networks on an FPGA
Arxiv. Xilesou. Top/PDF / 1705.02…
068 (2017-05-5) Face Super-Resolution Through Wasserstein GANs
Arxiv. Xilesou. Top/PDF / 1705.02…
069 (2017-10-12) CVAE-GAN Fine-Grained Image Generation through Asymmetric Training
Arxiv. Xilesou. Top/PDF / 1703.10…
070 (2017-02-21) Amortised MAP Inference for Image Super-resolution
Arxiv. Xilesou. Top/PDF / 1610.04…
071 (2017-05-25) Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Arxiv. Xilesou. Top/PDF / 1609.04…
Public account recently recommended reading: \
GAN is 6 years old! It’s time to stroke him!
Newbie guide reviews | GAN model too much, don’t know where to choose? \
Hundreds of GAN papers have been downloaded! With a recent overview of generative adversarial networks! \
A bit exaggerated, a bit twisted! See how these Gans exaggerate and caricature human faces! \
It rained from heaven, but nothing for me! How about GAN for rain? \
Face to face! GAN can make the side face killer, peppa pig true face no escape? \
Losing face! GAN to predict? \
Strong data is hard! How about SSL (semi-supervised Learning) with GAN? \
Weak water three thousand, only take your mark! How about AL (Active Learning) and GAN? \
Anomaly detection, GAN how GAN?
Virtual change of clothes! Quick overview of these latest papers how to do! \
Facial makeup migration! Take a quick look at a few papers using GAN
[1] How about the generation of GAN in medical images?
01-GAN formula concise principle of ironclad small treasure
GAN&CV communication Group, no matter small white or big guy, sincerely invite you to join! \
Discuss and exchange! Long press note to join:
Share more, long click to follow our public account: \
