This is the second day of my participation in the November Gwen Challenge. Check out the details: the last Gwen Challenge 2021

Rethinking and Improving the Robustness of Image Style Transfer

Motivation

Through a lot of experiments, the author found that VGG was better than ResNet in the style transfer task. The reason was that the entropy of feature graph generated by residual connection was low, which was not conducive to style conversion. Therefore, the author proposed a method based on SoftMax to enhance entropy. It is verified that this method is still valid for randomly initialized weights, which suggests that the network structure used for feature extraction is more important than the learned weights.

Introduction

As shown in the figure above, the Vgg/ResNet model with random initialization or pre-training on ImageNet is used for style conversion. It can be found that the effect of pre-training is better than that of random initialization, and the effect of Vgg is better than that of Res, which is also the starting point of the author. It is to explore why ResNet as backbone is better than Vgg in classification and segmentation tasks, but it is the opposite in style transformation.

To confirm the hypothesis, the authors added the following experiments:

(f) is the Res after eliminating residual connection, and (g) is

Method