Interesting | 18 two dimensional image generated deep

Introduction to the

Similar to CelebA, ACGAN is used to generate 2d avatars and is controlled by multiple conditions

data

Getchu.com/, a Japanese meta-game site, contains a large number of game character renderings, with a total of 31,970 renderings

Head interception

The dLIB described earlier can be used to extract faces, but not 2d avatars

Use OpenCV to capture the avatar portion of each image using the following project, github.com/nagadomi/lb…

The detection results should be properly enlarged to include more character details

# -*- coding: utf-8 -*-

import cv2

cascade = cv2.CascadeClassifier('lbpcascade_animeface.xml')

image = cv2.imread('IMGS / 2d head sample.jpg') gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) gray = cv2.equalizeHist(gray) faces = cascade.detectMultiScale(gray, ScaleFactor = 1.1, minNeighbors = 5, minSize = (64, 64))for i, (x, y, w, h) inenumerate(faces): Cx = x + w // 2 cy = y + h // 2 x0 = cx-int (0.75 * w) x1 = cx + int(0.75 * w) y0 = cy-int (0.75 * h) y1 = cy + Int (0.75 * h)if x0 < 0:
        x0 = 0
    if y0 < 0:
        y0 = 0
    if x1 >= image.shape[1]:
        x1 = image.shape[1] - 1
    if y1 >= image.shape[0]:
        y1 = image.shape[0] - 1
    w = x1 - x0
    h = y1 - y0
    if w > h:
        x0 = x0 + w // 2 - h // 2
        x1 = x1 - w // 2 + h // 2
        w = h
    else:
        y0 = y0 + h // 2 - w // 2
        y1 = y1 - h // 2 + w // 2
        h = w
    face = image[y0: y0 + h, x0: x0 + w, :]
    face = cv2.resize(face, (128, 128))
    cv2.imwrite('face_%d.jpg' % i, face)
Copy the code

Label extract

Extract rich tags from a quadratic image using Illustration2Vec, github.com/rezoo/illus…

Illustration2Vec uses the Deep learning framework Chainer, as well as several other libraries, if not installed

pip install chainer Pillow scikit-image
Copy the code

Illustration2Vec does three things:

• Each image is represented as a 4096-dimensional vector
• Specify a threshold and extract labels whose probability is higher than the threshold
• Specify some labels and return the corresponding probabilities

For example, extract all possible labels with a threshold of 0.5

# -*- coding: utf-8 -*-

import i2v
from imageio import imread

illust2vec = i2v.make_i2v_with_chainer('illust2vec_tag_ver200.caffemodel'.'tag_list.json')

img = imread('IMGS / 2d head sample.jpg')
tags = illust2vec.estimate_plausible_tags([img], threshold=0.5)
print(tags)
tags = illust2vec.estimate_specific_tags([img], ['blue eyes'.'red hair'])
print(tags)
Copy the code

You can also specify the label and get the corresponding probability

[{'blue eyes': 0.9488178491592407.'red hair': 0.0025324225425720215}]
Copy the code

pretreatment

All images are processed on the server, that is, head images are captured and labels are extracted

Load the library

# -*- coding: utf-8 -*-import i2v import cv2 import glob import os from imageio import imread from tqdm import tqdm import matplotlib.pyplot as  plt %matplotlib inline import numpy as np import pickleCopy the code

Read image path

images = glob.glob('characters/*.jpg')
print(len(images))
Copy the code

Loading two models

illust2vec = i2v.make_i2v_with_chainer('illust2vec_tag_ver200.caffemodel'.'tag_list.json')
cascade = cv2.CascadeClassifier('lbpcascade_animeface.xml')
OUTPUT_DIR = 'faces/'
if not os.path.exists(OUTPUT_DIR):
    os.mkdir(OUTPUT_DIR)
Copy the code

All the heads were extracted and 27,772 were detected

num = 0
for x intqdm(range(len(images))): img_path = images[x] image = cv2.imread(img_path) gray = cv2.cvtColor(image, Cv2.color_bgr2gray) Gray = cv2.equalizehist (gray) Faces = cascade.DetectMultiScale (gray, scaleFactor=1.1, minNeighbors=5, minSize=(64, 64))for (x, y, w, h) infaces: Cx = x + w // 2 cy = y + h // 2 x0 = cx-int (0.75 * w) x1 = cx + int(0.75 * w) y0 = cy-int (0.75 * h) y1 = cy + Int (0.75 * h)if x0 < 0:
            x0 = 0
        if y0 < 0:
            y0 = 0
        if x1 >= image.shape[1]:
            x1 = image.shape[1] - 1
        if y1 >= image.shape[0]:
            y1 = image.shape[0] - 1
        w = x1 - x0
        h = y1 - y0
        if w > h:
            x0 = x0 + w // 2 - h // 2
            x1 = x1 - w // 2 + h // 2
            w = h
        else:
            y0 = y0 + h // 2 - w // 2
            y1 = y1 - h // 2 + w // 2
            h = w
        
        face = image[y0: y0 + h, x0: x0 + w, :]
        face = cv2.resize(face, (128, 128))
        cv2.imwrite(os.path.join(OUTPUT_DIR, '%d.jpg' % num), face)
        num += 1
print(num)
Copy the code

The labels of interest include the following 34:

• 13 Hair Colors: blonde hair, brown hair, black hair, blue hair, pink hair, purple hair, green hair, red hair, silver hair, white hair, orange hair, aqua hair, grey hair
• 5 styles: Long hair, short hair, Twintails, Drill hair, ponytail
• 10 Eye Colors: blue eyes, red eyes, brown eyes, green eyes, purple eyes, yellow eyes, pink eyes, aqua eyes, black eyes, orange eyes
• 6 other attributes: blush, smile, Open mouth, Hat, ribbon, glasses

Hair color, hairstyle and eye color are selected with the highest probability, and other attributes with probability higher than 0.25 are treated as existence

fw = open('face_tags.txt'.'w')
tags = ['blonde hair'.'brown hair'.'black hair'.'blue hair'.'pink hair'.'purple hair'.'green hair'.'red hair'.'silver hair'.'white hair'.'orange hair'.'aqua hair'.'grey hair'.'long hair'.'short hair'.'twintails'.'drill hair'.'ponytail'.'blue eyes'.'red eyes'.'brown eyes'.'green eyes'.'purple eyes'.'yellow eyes'.'pink eyes'.'aqua eyes'.'black eyes'.'orange eyes'.'blush'.'smile'.'open mouth'.'hat'.'ribbon'.'glasses']
fw.write('id,' + ', '.join(tags) + '\n')

images = glob.glob(os.path.join(OUTPUT_DIR, '*.jpg'))
for x in tqdm(range(len(images))):
    img_path = images[x]
    image = imread(img_path)
    result = illust2vec.estimate_specific_tags([image], tags)[0]
    
    hair_colors = [[h, result[h]] for h in tags[0:13]]
    hair_colors.sort(key=lambda x:x[1], reverse=True)
    for h in tags[0:13]:
        if h == hair_colors[0][0]:
            result[h] = 1
        else:
            result[h] = 0
    
    hair_styles = [[h, result[h]] for h in tags[13:18]]
    hair_styles.sort(key=lambda x:x[1], reverse=True)
    for h in tags[13:18]:
        if h == hair_styles[0][0]:
            result[h] = 1
        else:
            result[h] = 0
    
    eye_colors = [[h, result[h]] for h in tags[18:28]]
    eye_colors.sort(key=lambda x:x[1], reverse=True)
    for h in tags[18:28]:
        if h == eye_colors[0][0]:
            result[h] = 1
        else:
            result[h] = 0
    
    for h in tags[28:]:
        ifResult [h] > 0.25: result[h] = 1else:
            result[h] = 0
        
    fw.write(img_path + ', ' + ', '.join([str(result[t]) for t in tags]) + '\n')
    
fw.close()
Copy the code

This gives you 27,772 quadratic heads and 34 tag values for each head

Get 4096 dimensional vector representation of each head

illust2vec = i2v.make_i2v_with_chainer("illust2vec_ver200.caffemodel")
img_all = []
vec_all = []
for x in tqdm(range(len(images))):
    img_path = images[x]
    image = imread(img_path)
    vector = illust2vec.extract_feature([image])[0]
    img_all.append(image / 255.)
    vec_all.append(vector)
img_all = np.array(img_all)
vec_all = np.array(vec_all)
Copy the code

2000 heads were randomly selected for tSNE dimensionality reduction visualization

from sklearn.manifold import TSNE
from imageio import imsave
data_index = np.arange(img_all.shape[0])
np.random.shuffle(data_index)
data_index = data_index[:2000]

tsne = TSNE(perplexity=30, n_components=2, init='pca', n_iter=5000)
two_d_vectors = tsne.fit_transform(vec_all[data_index, :])
puzzles = np.ones((6400, 6400, 3))
xmin = np.min(two_d_vectors[:, 0])
xmax = np.max(two_d_vectors[:, 0])
ymin = np.min(two_d_vectors[:, 1])
ymax = np.max(two_d_vectors[:, 1])

for i, vector in enumerate(two_d_vectors):
    x, y = two_d_vectors[i, :]
    x = int((x - xmin) / (xmax - xmin) * (6400 - 128) + 64)
    y = int((y - ymin) / (ymax - ymin) * (6400 - 128) + 64)
    puzzles[y - 64: y + 64, x - 64: x + 64, :] = img_all[data_index[i]]
imsave('Two-dimensional image reduction visualization. PNG', puzzles)
Copy the code

The visualization shows that similar heads are indeed clustered together

model

Use ACGAN structure, but unlike DCGAN used in CelebA, this time use deeper and more complex network to implement G and D, see SRGAN, arxiv.org/abs/1609.04…

The generator structure is as follows:

• Use 16 residual blocks, the shortcut idea in ResNet
• Use sub-pixel CNN instead of deconvolution, arxiv.org/abs/1609.05…

The principle of sub-Pixel CNN is as follows: multiple layers are splicing into one layer, so as to increase the height and width and reduce the depth

The discriminator structure is as follows, using 10 residual blocks, including two output terminals, to complete the discrimination and classification tasks respectively

implementation

Load the library

# -*- coding: utf-8 -*-import tensorflow as tf import numpy as np import pandas as pd import matplotlib.pyplot as plt %matplotlib inline import  os from imageio import imread, imsave, mimsave import glob from tqdm import tqdmCopy the code

Loading pictures

images = glob.glob('faces/*.jpg')
print(len(images))
Copy the code

Load the label

tags = pd.read_csv('face_tags.txt')
tags.index = tags['id']
tags.head()
Copy the code

Define some constants, network tensor, auxiliary functions, and let’s say that the batch size is a good power of two, 64 here, for learning rate decay

Batch_size = 64 Z_DIM = 128 WIDTH = 128 HEIGHT = 128 LABEL = 34 LAMBDA = 0.05 BETA = 3 OUTPUT_DIR ='samples'
if not os.path.exists(OUTPUT_DIR):
    os.mkdir(OUTPUT_DIR)

X = tf.placeholder(dtype=tf.float32, shape=[batch_size, HEIGHT, WIDTH, 3], name='X')
X_perturb = tf.placeholder(dtype=tf.float32, shape=[batch_size, HEIGHT, WIDTH, 3], name='X_perturb')
Y = tf.placeholder(dtype=tf.float32, shape=[batch_size, LABEL], name='Y')
noise = tf.placeholder(dtype=tf.float32, shape=[batch_size, z_dim], name='noise')
noise_y = tf.placeholder(dtype=tf.float32, shape=[batch_size, LABEL], name='noise_y')
is_training = tf.placeholder(dtype=tf.bool, name='is_training') global_step = tf.Variable(0, Trainable =False) add_global = global_step. Assign_add (1) initial_learning_rate = 0.0002 Learning_rate = tf.train.exponential_decay(initial_learning_rate, global_step=global_step, decay_steps=20000, Decay_rate = 0.5) def lrelu (x, leak = 0.2) :return tf.maximum(x, leak * x)

def sigmoid_cross_entropy_with_logits(x, y):
    return tf.nn.sigmoid_cross_entropy_with_logits(logits=x, labels=y)

def conv2d(inputs, kernel_size, filters, strides, padding='same', use_bias=True):
    returntf.layers.conv2d(inputs=inputs, kernel_size=kernel_size, filters=filters, strides=strides, padding=padding, Use_bias =use_bias) def batch_norm(inputs, is_training=is_training, decay=0.9):return tf.contrib.layers.batch_norm(inputs, is_training=is_training, decay=decay)
Copy the code

Discriminator part

def d_block(inputs, filters):
    h0 = lrelu(conv2d(inputs, 3, filters, 1))
    h0 = conv2d(h0, 3, filters, 1)
    h0 = lrelu(tf.add(h0, inputs))
    return h0

def discriminator(image, reuse=None):
    with tf.variable_scope('discriminator', reuse=reuse):
        h0 = image
        
        f = 32
        for i in range(5):
            if i < 3:
                h0 = lrelu(conv2d(h0, 4, f, 2))
            else:
                h0 = lrelu(conv2d(h0, 3, f, 2))
            h0 = d_block(h0, f)
            h0 = d_block(h0, f)
            f = f * 2
        
        h0 = lrelu(conv2d(h0, 3, f, 2))
        h0 = tf.contrib.layers.flatten(h0)
        Y_ = tf.layers.dense(h0, units=LABEL)
        h0 = tf.layers.dense(h0, units=1)
        return h0, Y_
Copy the code

Generator section

def g_block(inputs):
    h0 = tf.nn.relu(batch_norm(conv2d(inputs, 3, 64, 1, use_bias=False)))
    h0 = batch_norm(conv2d(h0, 3, 64, 1, use_bias=False))
    h0 = tf.add(h0, inputs)
    return h0

def generator(z, label):
    with tf.variable_scope('generator', reuse=None):
        d = 16
        z = tf.concat([z, label], axis=1)
        h0 = tf.layers.dense(z, units=d * d * 64)
        h0 = tf.reshape(h0, shape=[-1, d, d, 64])
        h0 = tf.nn.relu(batch_norm(h0))
        shortcut = h0
        
        for i in range(16):
            h0 = g_block(h0)
            
        h0 = tf.nn.relu(batch_norm(h0))
        h0 = tf.add(h0, shortcut)
        
        for i in range(3):
            h0 = conv2d(h0, 3, 256, 1, use_bias=False)
            h0 = tf.depth_to_space(h0, 2)
            h0 = tf.nn.relu(batch_norm(h0))
        
        h0 = tf.layers.conv2d(h0, kernel_size=9, filters=3, strides=1, padding='same', activation=tf.nn.tanh, name='g', use_bias=True)
        return h0
Copy the code

Loss function, here gp term from DRAGAN, arxiv.org/abs/1705.07… , WGAN uses the interpolation of the real sample and the synthetic sample, while DRAGAN uses the interpolation of the real sample and the interference sample

g = generator(noise, noise_y)
d_real, y_real = discriminator(X)
d_fake, y_fake = discriminator(g, reuse=True)

loss_d_real = tf.reduce_mean(sigmoid_cross_entropy_with_logits(d_real, tf.ones_like(d_real)))
loss_d_fake = tf.reduce_mean(sigmoid_cross_entropy_with_logits(d_fake, tf.zeros_like(d_fake)))
loss_g_fake = tf.reduce_mean(sigmoid_cross_entropy_with_logits(d_fake, tf.ones_like(d_fake)))

loss_c_real = tf.reduce_mean(sigmoid_cross_entropy_with_logits(y_real, Y))
loss_c_fake = tf.reduce_mean(sigmoid_cross_entropy_with_logits(y_fake, noise_y))

loss_d = loss_d_real + loss_d_fake + BETA * loss_c_real
loss_g = loss_g_fake + BETA * loss_c_fake

alpha = tf.random_uniform(shape=[batch_size, 1, 1, 1], minval=0., maxval=1.)
interpolates = alpha * X + (1 - alpha) * X_perturb
grad = tf.gradients(discriminator(interpolates, reuse=True)[0], [interpolates])[0]
slop = tf.sqrt(tf.reduce_sum(tf.square(grad), axis=[1]))
gp = tf.reduce_mean((slop - 1.) ** 2)
loss_d += LAMBDA * gp

vars_g = [var for var in tf.trainable_variables() if var.name.startswith('generator')]
vars_d = [var for var in tf.trainable_variables() if var.name.startswith('discriminator')]
Copy the code

Defining the optimizer

update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS) with tf.control_dependencies(update_ops): Optimizer_d = tf. Train. AdamOptimizer (learning_rate = learning_rate, beta1 = 0.5). Minimize (loss_d, Var_list =vars_d) optimizer_g = tf.train.AdamOptimizer(learning_rate=learning_rate, beta1=0.5). Minimize (loss_g, var_list=vars_g)Copy the code

The function that synthesizes the picture

def montage(images):    
    if isinstance(images, list):
        images = np.array(images)
    img_h = images.shape[1]
    img_w = images.shape[2]
    n_plots = int(np.ceil(np.sqrt(images.shape[0])))
    iflen(images.shape) == 4 and images.shape[3] == 3: M = np.ones((images.shape[1] * n_plots + n_plots + 1, images.shape[2] * n_plots + n_plots + 1, 3)) * 0.5eliflen(images.shape) == 4 and images.shape[3] == 1: M = np.ones((images.shape[1] * n_plots + n_plots + 1, images.shape[2] * n_plots + n_plots + 1, 1)) * 0.5eliflen(images.shape) == 3: M = np.ones((images.shape[1] * n_plots + n_plots + 1, images.shape[2] * n_plots + n_plots + 1)) * 0.5else:
        raise ValueError('Could not parse image shape of {}'.format(images.shape))
    for i in range(n_plots):
        for j in range(n_plots):
            this_filter = i * n_plots + j
            if this_filter < images.shape[0]:
                this_img = images[this_filter]
                m[1 + i + i * img_h:1 + i + (i + 1) * img_h,
                  1 + j + j * img_w:1 + j + (j + 1) * img_w] = this_img
    return m
Copy the code

Sorting data

X_all = []
Y_all = []
for i intqdm(range(len(images))): Image = imread(images[I]) image = (image / 255.-0.5) * 2 x_all.appEnd (image) y = list(tags. Loc [images[I]]) Y_all.append(y[1:]) X_all = np.array(X_all) Y_all = np.array(Y_all)print(X_all.shape, Y_all.shape)
Copy the code

Define a function that randomly generates labels. Labels are not evenly distributed in the original data, but we want G to learn all kinds of labels, so we evenly generate all kinds of labels

def get_random_tags(): Y = Np.random. Uniform (0.0, 1.0, [BATCH_size, LABEL]).astype(NP.float32) Y [y > 0.75] = 1 y[y <= 0.75] = 0for i in range(batch_size):
        hc = np.random.randint(0, 13)
        hs = np.random.randint(13, 18)
        ec = np.random.randint(18, 28)
        y[i, :28] = 0
        y[i, hc] = 1 # hair color
        y[i, hs] = 1 # hair style
        y[i, ec] = 1 # eye color
    return y
Copy the code

Training model: male and female ratio in CelebA was balanced, so a batch of data was randomly selected for training in each iteration. However, since the original data is not evenly distributed, the data needs to be iterated completely

Sess = tf.session () sess.run(tf.global_variables_initializer()) z_samples = Np.random. Uniform (-1.0, 1.0, [batch_size, z_dim]).astype(np.float32) y_samples = get_random_tags()for i in range(batch_size):
    y_samples[i, :28] = 0
    y_samples[i, i // 8 % 13] = 1 # hair color
    y_samples[i, i // 8 % 5 + 13] = 1 # hair style
    y_samples[i, i // 8 % 10 + 18] = 1 # eye color
samples = []
loss = {'d': [].'g': []}

offset = 0
for i in tqdm(range(60000)):
    if offset + batch_size > X_all.shape[0]:
        offset = 0
    ifoffset == 0: data_index = np.arange(X_all.shape[0]) np.random.shuffle(data_index) X_all = X_all[data_index, :, :, :] Y_all = Y_all[data_index, :] X_batch = X_all[offset: offset + batch_size, :, :, :] Y_batch = Y_all[offset: offset + batch_size, :] X_batch_perturb = X_batch + 0.5 * X_batch.std() * NP.random. Random (x_batch.shape) offset += batch_size n = Np.random. Uniform (-1.0, 1.0, [BATCH_SIZE, z_DIM]).astype(NP.float32) ny = get_random_tags() _, d_ls = sess.run([optimizer_d, loss_d], feed_dict={X: X_batch, X_perturb: X_batch_perturb, Y: Y_batch, noise: n, noise_y: ny, is_training: True}) n = Np.random. Uniform (-1.0, 1.0, [BATCH_SIZE, z_DIM]).astype(NP.float32) ny = get_random_tags() _, g_ls = sess.run([optimizer_g, loss_g], feed_dict={noise: n, noise_y: ny, is_training: True}) loss['d'].append(d_ls)
    loss['g'].append(g_ls)
    
    _, lr = sess.run([add_global, learning_rate])
    
    if i % 500 == 0:
        print(i, d_ls, g_ls, lr)
        gen_imgs = sess.run(g, feed_dict={noise: z_samples, noise_y: y_samples, is_training: False})
        gen_imgs = (gen_imgs + 1) / 2
        imgs = [img[:, :, :] for img in gen_imgs]
        gen_imgs = montage(imgs)
        plt.axis('off')
        plt.imshow(gen_imgs)
        imsave(os.path.join(OUTPUT_DIR, 'sample_%d.jpg' % i), gen_imgs)
        plt.show()
        samples.append(gen_imgs)

plt.plot(loss['d'], label='Discriminator')
plt.plot(loss['g'], label='Generator')
plt.legend(loc='upper right')
plt.savefig('Loss.png')
plt.show()
mimsave(os.path.join(OUTPUT_DIR, 'samples.gif'), samples, fps=10)
Copy the code

The generated quadratic avatar is as follows, each row corresponds to the same hair color, hairstyle, eye color, other attributes random. A few don’t turn out well, probably because of some noise or conditions

Save the model

saver = tf.train.Saver()
saver.save(sess, './anime_acgan', global_step=60000)
Copy the code

Load the model on a single machine and carry out the following three attempts:

• Generate samples randomly according to the original label distribution
• Generates a sample of the specified tag
• Fixed noise, generate sample according to original label distribution

# -*- coding: utf-8 -*-

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
from imageio import imsave

batch_size = 64
z_dim = 128
LABEL = 34

def montage(images):    
    if isinstance(images, list):
        images = np.array(images)
    img_h = images.shape[1]
    img_w = images.shape[2]
    n_plots = int(np.ceil(np.sqrt(images.shape[0])))
    iflen(images.shape) == 4 and images.shape[3] == 3: M = np.ones((images.shape[1] * n_plots + n_plots + 1, images.shape[2] * n_plots + n_plots + 1, 3)) * 0.5eliflen(images.shape) == 4 and images.shape[3] == 1: M = np.ones((images.shape[1] * n_plots + n_plots + 1, images.shape[2] * n_plots + n_plots + 1, 1)) * 0.5eliflen(images.shape) == 3: M = np.ones((images.shape[1] * n_plots + n_plots + 1, images.shape[2] * n_plots + n_plots + 1)) * 0.5else:
        raise ValueError('Could not parse image shape of {}'.format(images.shape))
    for i in range(n_plots):
        for j in range(n_plots):
            this_filter = i * n_plots + j
            if this_filter < images.shape[0]:
                this_img = images[this_filter]
                m[1 + i + i * img_h:1 + i + (i + 1) * img_h,
                  1 + j + j * img_w:1 + j + (j + 1) * img_w] = this_img
    returnm def get_random_tags(): Y = Np.random. Uniform (0.0, 1.0, [BATCH_size, LABEL]). Astype (NP.float32) p_other = [0.6, 0.6, 0.25, 0.04488882, 0.3, y = NP.random. Uniform (0.0, 1.0, [BATCH_size, LABEL]). 0.05384738]for i in range(batch_size):
        for j in range(len(p_other)):
            if y[i, j + 28] < p_other[j]:
                y[i, j + 28] = 1
            else: Y [I, j + 28] = 0 PHC = [0.15968645, 0.21305391, 0.15491921, 0.10523116, 0.07953927, 0.09508879, 0.03567429, 0.07733163, PHS = [0.52989922, 0.01833307, 0.02236442, 0.00537514, 0.00182371] 0.00847864] pec = [0.28350664, 0.15760678, 0.17862742, 0.13412254, 0.14212126, 0.0543913, 0.01020637, 0.00617501, 0.28350664, 0.15760678, 0.17862742, 0.13412254, 0.14212126, 0.0543913, 0.01020637, 0.00617501, 0.03167493, 0.00156775]for i in range(batch_size):
        y[i, :28] = 0

        hc = np.random.random()
        for j in range(len(phc)):
            if np.sum(phc[:j]) < hc < np.sum(phc[:j + 1]):
                y[i, j] = 1
                break

        hs = np.random.random()
        for j in range(len(phs)):
            if np.sum(phs[:j]) < hs < np.sum(phs[:j + 1]):
                y[i, j + 13] = 1
                break

        ec = np.random.random()
        for j in range(len(pec)):
            if np.sum(pec[:j]) < ec < np.sum(pec[:j + 1]):
                y[i, j + 18] = 1
                break
    return y

sess = tf.Session()
sess.run(tf.global_variables_initializer())

saver = tf.train.import_meta_graph('./anime_acgan-60000.meta')
saver.restore(sess, tf.train.latest_checkpoint('/'))

graph = tf.get_default_graph()
g = graph.get_tensor_by_name('generator/g/Tanh:0')
noise = graph.get_tensor_by_name('noise:0')
noise_y = graph.get_tensor_by_name('noise_y:0')
is_training = graph.get_tensor_by_name('is_training:0')

# Generate samples randomlyZ_samples = Np.random. Uniform (-1.0, 1.0, [batch_size, z_dim]).astype(np.float32) y_samples = get_random_tags() gen_imgs = sess.run(g, feed_dict={noise: z_samples, noise_y: y_samples, is_training: False}) gen_imgs = (gen_imgs + 1) / 2 imgs = [img[:, :, :]for img in gen_imgs]
gen_imgs = montage(imgs)
gen_imgs = np.clip(gen_imgs, 0, 1)
imsave('1_ quadratic head generated randomly. JPG ', gen_imgs)

Generate a sample of the specified tag
all_tags = ['blonde hair'.'brown hair'.'black hair'.'blue hair'.'pink hair'.'purple hair'.'green hair'.'red hair'.'silver hair'.'white hair'.'orange hair'.'aqua hair'.'grey hair'.'long hair'.'short hair'.'twintails'.'drill hair'.'ponytail'.'blue eyes'.'red eyes'.'brown eyes'.'green eyes'.'purple eyes'.'yellow eyes'.'pink eyes'.'aqua eyes'.'black eyes'.'orange eyes'.'blush'.'smile'.'open mouth'.'hat'.'ribbon'.'glasses']
for i, tags in enumerate([['blonde hair'.'twintails'.'blush'.'smile'.'ribbon'.'red eyes'], ['silver hair'.'long hair'.'blush'.'smile'.'open mouth'.'blue eyes']): z_samples = Np.random. Uniform (-1.0, 1.0, [BATch_size, z_DIM]).astype(NP.float32) y_samples = NP.zeros ([1, LABEL]).for tag in tags:
        y_samples[0, all_tags.index(tag)] = 1
    y_samples = np.repeat(y_samples, batch_size, 0)
    gen_imgs = sess.run(g, feed_dict={noise: z_samples, noise_y: y_samples, is_training: False})
    gen_imgs = (gen_imgs + 1) / 2
    imgs = [img[:, :, :] for img in gen_imgs]
    gen_imgs = montage(imgs)
    gen_imgs = np.clip(gen_imgs, 0, 1)
    imsave('%d_ second element head specified tag.jpg ' % (i + 2), gen_imgs)

# Fixed noise random tagZ_samples = Np.random. Uniform (-1.0, 1.0, [1, z_DIM]).astype(NP.float32) Z_samples = Np.repeat (z_samples, batch_size, 0) y_samples = get_random_tags() gen_imgs = sess.run(g, feed_dict={noise: z_samples, noise_y: y_samples, is_training: False}) gen_imgs = (gen_imgs + 1) / 2 imgs = [img[:, :, :]for img in gen_imgs]
gen_imgs = montage(imgs)
gen_imgs = np.clip(gen_imgs, 0, 1)
imsave('4_ quadratic head fixed noise.jpg ', gen_imgs)
Copy the code

Generate samples randomly according to the original label distribution

Create blonde hair, ponytail, blush, smile, ribbon, red eyes

Create an avatar with silver hair, long hair, red face, smile, open mouth and blue eyes

Fixed random noise labels so that the avatar body is roughly the same but the details are different

With that in mind, you can also train ACGAN models controlled by 40 01 attributes on CelebA, and it’s a little easier than the quadratic avatar

reference

• Towards the Automatic Anime Characters Creation with Generative Adversarial Networks: arxiv.org/abs/1708.05…
• Cartoon head detection: github.com/nagadomi/lb…
• Illustration2Vec:github.com/rezoo/illus…

Video lecture course

Deep and interesting (1)