This is the 24th day of my participation in Gwen Challenge

In the analysis of Alexnet by Professor Li Hongyi, some patterns and colors were identified in the figure. In fact, we input images in the first layer. If the filter in the second layer is the filter in the first layer as the input, it is input to the feature layer.

from keras.applications import VGG16
from keras import backend as K
import matplotlib.pyplot as plt
Copy the code

Keras provided classical prediction models such as VGG16, and most of our neural networks were trained based on these classical neural networks.

For a layer convolution filter of Block 3, 150 x 150 go, we find what image can block conv1 number 0 feature reflection, we need to define the function, feature sum average of all pixels maximum to indicate that feature graph is activated.

VGG16 The first filter of block3_conv1 layer (number 1), 150×150150 \times150150×150 So this filter is the most sensitive to this image, so how do we mathematically describe and measure this, so the average of all the values calculated by the image and the filter is the largest, which means that the filter is the most sensitive to this image. So that’s our target function, and what we’re going to do is we’re going to take an image and put it in and maximize the mean of that filter. Let’s use x for the picture we’re looking for, Maxxf(x)Max_x f(x)Maxxf(x) df(x)dx\frac{df(x)}{dx} DXDF (x)w We use gradient rise instead of differential process, Let’s enter the picture here so x is 150×150×3150 \times 150 \times 3150×150×3 matrix, and we take the derivative of each of the components of this matrix

model = VGG16(weights='imagenet',include_top=False)
layer_name = 'block3_conv1'
filter_index = 0
Copy the code

Gradient rise

W1 ← W0 +η∂L∂ WW_1 \leftarrow w_0 + \eta \frac{\partial L}{\partial w}w1← \partial L}{\partial w}w1← \partial L}{\partial w}w1←w0+η∂ W ∂L I’m going to do a regularization of L2 for the gradient vector.

layer_output = model.get_layer(layer_name).output
loss = K.mean(layer_output[:,:,:,filter_index])
Copy the code
layer_output
Copy the code
<tf.Tensor 'block1_conv1/Relu:0' shape=(? ,? ,? , 64) dtype=float32>Copy the code
print(loss)
Copy the code
Tensor("Mean_1170:0", shape=(), dtype=float32)
Copy the code
grads = K.gradients(loss,model.input) [0]
Copy the code
grads
Copy the code
<tf.Tensor 'gradients_585/block1_conv1/convolution_grad/Conv2DBackpropInput:0' shape=(? ,? ,? , 3) dtype=float32>Copy the code
grads /= (K.sqrt(K.mean(K.square(grads))) + 1e-5)
Copy the code
iterate = K.function([model.input],[loss,grads])
import numpy as np
loss_value,grads_value = iterate([np.zeros((1.150.150.3)))Copy the code
input_img_data = np.random.random((1.150.150.3)) * 20 + 128.
step = 1.
for i in range(40):
    loss_value,grads_value = iterate([input_img_data])
    input_img_data += grads_value * step
Copy the code

This function is used to compress the image. What we need to do is to compress the image

def depress_image(x) :
    x -= x.mean()
    x /= (x.std() + 1e-5)
    x *= 0.1
    
    x += 0.5
    x = np.clip(x,0.1)
    
    x *= 255
    x = np.clip(x,0.255).astype('uint8')
    return x
Copy the code

This function

def generate_pattern(layer_name, filter_index, size=150) :
    layer_output = model.get_layer(layer_name).output
    loss = K.mean(layer_output[:,:,:,filter_index])
    
    grads = K.gradients(loss,model.input) [0]
    grads /= (K.sqrt(K.mean(K.square(grads))) + 1e-5)
    
    iterate = K.function([model.input],[loss,grads])

    input_img_data = np.random.random((1,size,size,3)) * 20 + 128.
    step = 1.
    for i in range(40):
        loss_value,grads_value = iterate([input_img_data])
        input_img_data += grads_value * step
    img = input_img_data[0]
    return depress_image(img)
Copy the code
plt.imshow(generate_pattern('block1_conv1'.0))
plt.show()
Copy the code

plt.imshow(generate_pattern('block3_conv1'.0))
plt.show()
Copy the code

plt.imshow(generate_pattern('block4_conv1'.1))
plt.show()
Copy the code

for layer_name in ['block1_conv1'.'block2_conv1'.'block3_conv1'.'block4_conv1']:
    size = 64
    margin = 5
    results = np.zeros(( 8 * size + 7 * margin, 8 * size + 7 * margin,3))
    for i in range(8) :for j in range(8):
            
            filter_img = generate_pattern(layer_name, i + (j*8),size=size) horizontal_start = i * size + i * margin horizontal_end = horizontal_start + size vertical_start = j * size  + j * margin vertical_end = vertical_start + size results[horizontal_start:horizontal_end,vertical_start:vertical_end,:] = filter_img plt.figure(figsize=(20.20))
    plt.imshow(results)
    plt.show()
Copy the code
model.summary()
Copy the code
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         (None, None, None, 3)     0         
_________________________________________________________________
block1_conv1 (Conv2D)        (None, None, None, 64)    1792      
_________________________________________________________________
block1_conv2 (Conv2D)        (None, None, None, 64)    36928     
_________________________________________________________________
block1_pool (MaxPooling2D)   (None, None, None, 64)    0         
_________________________________________________________________
block2_conv1 (Conv2D)        (None, None, None, 128)   73856     
_________________________________________________________________
block2_conv2 (Conv2D)        (None, None, None, 128)   147584    
_________________________________________________________________
block2_pool (MaxPooling2D)   (None, None, None, 128)   0         
_________________________________________________________________
block3_conv1 (Conv2D)        (None, None, None, 256)   295168    
_________________________________________________________________
block3_conv2 (Conv2D)        (None, None, None, 256)   590080    
_________________________________________________________________
block3_conv3 (Conv2D)        (None, None, None, 256)   590080    
_________________________________________________________________
block3_pool (MaxPooling2D)   (None, None, None, 256)   0         
_________________________________________________________________
block4_conv1 (Conv2D)        (None, None, None, 512)   1180160   
_________________________________________________________________
block4_conv2 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block4_conv3 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block4_pool (MaxPooling2D)   (None, None, None, 512)   0         
_________________________________________________________________
block5_conv1 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block5_conv2 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block5_conv3 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block5_pool (MaxPooling2D)   (None, None, None, 512)   0         
=================================================================
Total params: 14,714,688
Trainable params: 14,714,688
Non-trainable params: 0
_________________________________________________________________
Copy the code