Directory:
What is face_recognition
Two, how to install
Third, the principle of
Four, presentations,
Five, handwriting simple neural network
What is face_recognition
1. Face_recognition is a powerful, simple and easy to use open source face recognition project, and equipped with complete development documents and application cases. 2. Based on the deep learning model in dlib, the industry's leading C++ open source library, the test was carried out with Labeled Faces in the Wild dataset with an accuracy of 99.38%.Copy the codeTwo, how to install
Configure face_recognition in Linux
Specific details refer to my blog: www.cnblogs.com/UniqueColor…
1. If you already have PYTHon2.7 on Linux, you need to update Python 2.7 to python2.x
Sudo add-apt-repository PPA :fkrull/deadsnakes-python2.7 sudo apt-get update sudo apt-get upgradeCopy the code2. Deployment steps
Install Boost, Boost.Python sudo apt-get install build-essential cmake
sudo apt-get install libgtk- 3-dev
sudo apt-get install libboost-all-dev
Copy the codeInstallation of Cmake :(it tooks a while to install ~1.5 min)
sudo wget https:/ / cmake.org/files/v3.9/cmake-3.9.0-rc5.tar.gz -o cmake. Tar. Gz
sudo tar -xvf cmake.tar.gz
cd cmake3.9. 0-rc5/ sudo chmod +x bootstrap sudo./bootstrap sudo make sudo make install Note: After installing cmake, run the cmake -version command to check whether the cmake version is successfully installed.Copy the codepip installation
$ wget https://bootstrap.pypa.io/get-pip.py
$ sudo python get-pip.py Note: After the installation is complete, run PIP -v to check whether the PIP version is successfully installed. Note: If python3.x is used, the last command python is changed to python3Copy the codeInstall the DLIB by manually compiling the dLIB
git clone https://github.com/davisking/dlib.git //Clone the code from githubcd dlib mkdir build cd build cmake ..                                            // Compile dlib in the default way (the SSE41 directive)cmake --build . cd .. Sudo Python setup.py install Note: The last step will take some time. If python3.x is used, the last command python is changed to python3Copy the codeAfter the installation is complete, run Python and type import dlib to execute successfully.
Install face_recognition
sudo pip install face_recognition
Copy the codeAfter the installation is complete, run Python and type import face_recognition.
Install opencv – python
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple opencv-python
Copy the codeTest: Run Python and type import cv2.
Face_recognition is configured on the Mac
Specific details refer to my blog www.cnblogs.com/UniqueColor…
Install dependency libraries:
1. Install cmake (a cross-platform installation tool)
brew install cmake
Copy the codeInstall boost, boost-python (C++ library)
brew install boost
brew install boost-python --with-python27.
Copy the code3. Compile dlib
git clone https://github.com/davisking/dlib.git cd dlib mkdir build cd build cmake ..                                            // Compile dlib in the default way (the SSE41 directive)
cmake --build .
cd ..
sudo python setup.py install
Copy the code- Install the Python library for face recognition
pip install face_recognition
Copy the code5. Install Opencv-Python
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple opencv-python
Copy the codeError:
Cannot uninstall 'numpy'. It is a distutils installed project and thus we cannot accurate
Copy the codeSolution: Forcibly upgrade
pip install numpy --ignore-installed numpy
Copy the codeThree, principle:
1. HOG
What is HOG?
HOG stands for histogram of directional gradient. Feature is a feature descriptor used for object detection in computer vision and image processing. HOG feature is formed by calculating and counting the histogram of gradient direction in local area of image.
Use the HOG algorithm to encode the image to create a simplified version of the image. Using this simplified image, find the part that looks most like the universal HOG face encoding.
Steps:
1.1 Convert the image to black and white. For dimensionality reduction, color is not considered
1.2 View every pixel in the picture. For a single pixel, look at the other pixels around it. The goal is to find and compare the depth of the current pixel with the pixel directly surrounding it, forming the arrow direction. By repeating this process for each pixel in the image, each pixel is eventually replaced by an arrow. These arrows are called gradients, and they show the flow of light to darkness on an image. (Gradients are used instead of pixels because the same person has different pixels in two photos with different shades, but if you analyze gradients only, two photos with different shades will get the same result).
1.3 It is too tedious to calculate each pixel, so the image is divided into some small squares of 16×16 pixels. In each square, we will calculate how many gradients there are in each main direction (how many point up, up right, right, etc.). Then we’ll replace the square with an arrow in the most directional direction.
Specific text algorithm steps:
1. Read the detection image, namely the input image
2. Grayscale the image (that is, r, G and B values of the input color image are converted to grayscale values by specific formulas)
3. Use Gamma correction method to standardize the color space of the input image (normalization)
4. Calculate the gradient (including size and direction) of each pixel of the image and capture the contour information. If we analyze pixels directly, two photos of the same person with different shades of light will have completely different pixel values. But if only the direction of brightness change is considered, the light and dark image will have the same result
5. Calculate the gradient histogram (number of different gradients) of each cell, and form each cell descriptor
6. Each several cells (such as 16*16 mentioned above) form a block, and the features of all cells in a block are connected in series to obtain the HOG feature descriptor of the block
7. Connect the HOG feature Descriptor of all blocks in image in series to obtain the HOG feature descriptor of the image (detection target), and obtain the feature vector of the final classification
Attached is the HOG implementation code:
# coding=utf- 8 -
import cv2
import numpy as np
import math
import matplotlib.pyplot as plt
# 1, read the detection image, namely the input image #2, gray image (that is, r, G, B values of the input color image are converted to gray value by a specific formula) #3The color space of the input image was normalized (normalized) # by Gamma correction method4, calculate the gradient (including size and direction) of each pixel of the image, and capture the contour information. If we analyze pixels directly, two photos of the same person with different shades of light will have completely different pixel values. But if only the direction of brightness change is considered, the light and dark image will have the same result5, calculates the gradient histogram (number of different gradients) of each cell, and forms the descriptor # of each cell6, will each several cells (as mentioned above16*16) to form a block, and the features of all cells in a block are concatenated to get the HOG feature descriptor # of the block7, connect the HOG feature Descriptor of all blocks in image in series to obtain the HOG feature descriptor of the image (detection target), and obtain the feature vector of the final classificationclass Hog_descriptor():
def __init__(self, img, cell_size=16, bin_size=8) : Self.img = np.sqrt(img/np.max(img)) self.img = img *255
self.cell_size = cell_size
self.bin_size = bin_size
self.angle_unit = 360 / self.bin_size
def extract(self):
height, width = self.img.shape
# 1Gradient_angle = self.global_gradient() gradient_magnitude = ABS (gradient_magnitude) #2Cell_gradient_vector = np.zeros((height/self.cell_size, width/self.cell_size, Self.bin_size)) # initialize8The first two parameters represent the length and width of each histogram #3, and filled all cell cells circularly in the unit of histogramfor i in range(cell_gradient_vector.shape[0) :for j in range(cell_gradient_vector.shape[1]):
cell_magnitude = gradient_magnitude[i * self.cell_size:(i + 1) * self.cell_size,
j * self.cell_size:(j + 1) * self.cell_size]
cell_angle = gradient_angle[i * self.cell_size:(i + 1) * self.cell_size,
j * self.cell_size:(j + 1) * self.cell_size]
cell_gradient_vector[i][j] = self.cell_gradient(cell_magnitude, cell_angle)
# 4Then, hog image hog_image = self.render_gradient(Np.zeros ([height, width]), cell_gradient_vector) hog_vector = []for i in range(cell_gradient_vector.shape[0] - 1) :for j in range(cell_gradient_vector.shape[1] - 1):
block_vector = []
block_vector.extend(cell_gradient_vector[i][j])
block_vector.extend(cell_gradient_vector[i][j + 1])
block_vector.extend(cell_gradient_vector[i + 1][j])
block_vector.extend(cell_gradient_vector[i + 1][j + 1])
mag = lambda vector: math.sqrt(sum(i ** 2 for i in vector))
magnitude = mag(block_vector)
ifmagnitude ! =0:
normalize = lambda block_vector, magnitude: [element / magnitude for element in block_vector]
block_vector = normalize(block_vector, magnitude)
hog_vector.append(block_vector)
returnDef global_gradient(self): gradient_values_x = cv2.sobel (self.img, cv2.cv_64f,1.0, ksize=5Gradient_values_y = cv2.sobel (self.img, cv2.cv_64f,0.1, ksize=5Gradient_magnitudes = cv2.addweighted (gradient_values_x, gradient_magnitudes = cv2.addweighted (gradient_values_x, gradient_magnitudes = cv2.addweighted (gradient_values_x, gradient_magnitudes = cv2.addweighted (gradient_values_x, gradient_magnitudes = cv2.addweighted)0.5, gradient_values_y, 0.5.0) # Is weighted in the X and Y directions, both are0.5The weight, Gradient_angle = cv2. Phase (gradient_values_x, gradient_values_y, gradient_angle = cv2. AngleInDegrees =True) # print(gradient_magnitude, gradient_angle)returnGradient_magnitude, gradient_angle # Def cell_gradient(self, cell_magnitude, cell_angle): orientation_centers = [0] * self.bin_size
for i in range(cell_magnitude.shape[0) :for j in range(cell_magnitude.shape[1) : gradient_strength = cell_magnitude[i][j] gradient_angle = cell_angle[i][j] min_angle, max_angle, mod = self.get_closest_bins(gradient_angle) orientation_centers[min_angle] += (gradient_strength * (1 - (mod / self.angle_unit)))
orientation_centers[max_angle] += (gradient_strength * (mod / self.angle_unit))
return orientation_centers
def get_closest_bins(self, gradient_angle):
idx = int(gradient_angle / self.angle_unit)
mod = gradient_angle % self.angle_unit
return idx, (idx + 1Def render_gradient(self, image, cell_gradient): cell_width = self.cell_size /2
max_mag = np.array(cell_gradient).max()
for x in range(cell_gradient.shape[0) :for y in range(cell_gradient.shape[1]):
cell_grad = cell_gradient[x][y]
cell_grad /= max_mag
angle = 0
angle_gap = self.angle_unit
for magnitude incell_grad: X1 = int(x * self.cell_size + magnitude * cell_width * math.cos(angle_radian)) y1 = int(y * self.cell_size + magnitude * cell_width * math.sin(angle_radian)) x2 = int(x * self.cell_size - magnitude * cell_width * math.cos(angle_radian)) y2 = int(y * self.cell_size - magnitude * cell_width * math.sin(angle_radian)) cv2.line(image, (y1, x1), (y2, x2), int(255 * math.sqrt(magnitude)))
angle += angle_gap
return image
img = cv2.imread('Stephen_Curry.jpg', cv2.IMREAD_GRAYSCALE)
hog = Hog_descriptor(img, cell_size=8, bin_size=8)
vector, image = hog.extract()
print np.array(vector).shape
plt.imshow(image, cmap=plt.cm.gray)
plt.show()
Copy the codeRunning results:
2. Image torsion
Scenario: To a computer, the same face facing in different directions is a different thing. So step two requires a face twist.
Find the posture of the face by finding 68 key features on the face. Once we find these features, we use them to distort the image so that the eyes and mouth are centered.
This problem can be solved using an algorithm called Face Landmark Estimation. The basic idea is: The basic idea is to find specific points (called landmarks) that are common on 68 people’s faces — including the top of the chin, the outer outline of each eye, the inner outline of each eyebrow, and so on. We then trained a machine learning algorithm to find those 68 specific points on any face.
Now that you know where the eyes and mouth are, rotate, scale, and misslice the image so that the eyes and mouth are as close to the center as possible. I’m not going to do any fancy 3d distortion here, because that would distort the image. Only basic image transformations such as rotation and scaling (called affine transformations) that keep the image relatively parallel are used:
Neural networks
Now comes the key part, which is how do you distinguish between different faces?
The solution was to train a deep convolutional neural network, but instead of recognizing objects in pictures, this time we trained it to generate 128 measurements for faces.
Each training session looked at three different images of faces:
Load a face training image of a known person. Load another photo of the same person three. Load another person's photoCopy the codeThe algorithm then looks at its own measurements generated for the three images. Then, the neural network is tweaked slightly to ensure that the first and second measurements are close, while the second and third produce slightly different measurements. Put the face image from the previous step into a neural network that knew how to find 128 feature measurements. Save the 128 measurements.
After repeating this step millions of times for millions of images of thousands of people, the neural network learned how to reliably generate 128 measurements for each person. It should give roughly the same measurement for any ten different photos of the same person. 128 test values for us, we don’t need to know what they are or how they are calculated, it’s the deep learning that makes sense to the computer.
Classification of 4.
Look at all the faces we’ve measured in the past and figure out which one is closest to the face we’re measuring. This step is relatively simple, just use various classification algorithms, the following two classification algorithms are recommended.
K – means: k-means clustering algorithm (k means clustering algorithm) reference my blog www.cnblogs.com/UniqueColor…
KNN: K nearest neighbor (K – NearestNeighbor) reference my blog www.cnblogs.com/UniqueColor…
4. Demonstration (using Face Recognition)
1. Recognize all the faces in the photo
# -*- coding: utf- 8 --*- # Detect faceimport face_recognition
importImg = face_recognition. Load_image_file ("scl.jpg"Face_locations (img) print(face_locations) # img = cv2.imread("scl.jpg")
# cv2.namedWindow("origin")
# cv2.imshow("origin", img) # add faceNum = len(face_locations)for i in range(0, faceNum):
top = face_locations[i][0]
right = face_locations[i][1]
bottom = face_locations[i][2]
left = face_locations[i][3]
start = (left, top)
end = (right, bottom)
color = (55.255.155)
thickness = 3Rectangle (img, start, end, color, thickness) #"recognition")
cv2.imshow("recognition", img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Copy the codeEffect:
2. Recognize faces in photos and determine who they are
# -*- coding: utf- 8 -- * -import face_recognition
from PIL import Image, ImageDraw
import numpy asKlay_image = face_recognition. Load_image_file ("Klay_Thompson.jpg")
klay_face_encoding = face_recognition.face_encodings(klay_image)[0Curry_image = face_recognition. Load_image_file ()"Stephen_Curry.jpg")
curry_face_encoding = face_recognition.face_encodings(curry_image)[0Known_face_encodings = [klay_face_encoding, curry_face_encoding] known_face_names = [klay_face_encoding, curry_face_encoding]"Klay Thompson"."Stepthen Curry"Unknown_image = face_recognition. Load_image_file (unknown_image = face_recognition."scl.jpg"Face_locations = face_recognition. Face_unknown_image face_encodings = Face_encodings (unknown_image, face_locations) # Convert the image to PIL format, Pil_image = image.fromarray (Unknown_image) draw = ImageDraw.Draw(pil_image) # loop each face in the Image to be recognizedfor (top, right, bottom, left), face_encoding inzip(face_locations, face_encodings): # see if this face matches a known face matches matches = face_recognition.com pare_FACES (known_face_encodings, face_encoding) name ="Unknown"# Compare the faces to be identified with known faces, Face_distance. face_distance(known_face_encodings, face_encoding) best_match_index = np.argmin(face_distances)ifmatches[best_match_index]: Name = known_face_names [best_match_index] # draw a box around the face. The draw a rectangle (((left, top), (right, bottom)), the outline = (0.0.255Text_width, text_height = draw.textSize (name) draw.rectangle((((left, bottom-text_height -)10), (right, bottom)), fill=(0.0.255), outline=(0.0.255))
draw.text((left + 6, bottom - text_height - 5), name, fill=(255.255.255.255Pil_image.show ()Copy the codeImage effect:
3, video real-time face recognition and determine which person
# -*- coding: utf- 8 -- * -import face_recognition
importCv2 # open video input_movie = cv2.videocapture ("hamilton_clip.mp4") length = int(input_movie.get(cv2.cap_prop_frame_count)) # Create output video fourcc = cv2.videoWriter_fourcc (*'XVID')
output_movie = cv2.VideoWriter('output.mp4', fourcc, 29.97, (640.360Lmm_image = face_recognition. Load_image_file ("lin-manuel-miranda.png")
lmm_face_encoding = face_recognition.face_encodings(lmm_image)[0Curry_image = face_recognition. Load_image_file ()"Stephen_Curry.jpg")
curry_face_encoding = face_recognition.face_encodings(curry_image)[0Known_face_encodings = [lmm_face_encoding, curry_face_encoding ] face_locations = [] face_encodings = [] face_names = [] frame_number =0
whileTrue: # Capture video frame ret, frame = input_movie.read() frame_number +=1# fetch endif not ret:
break# Convert image from BGR color (used by OpencV) to RGB color (used by face recognition)- 1Face_locations = face_recognition. Face_locations (RGB_frame) face_encodings = face_recognition.face_encodings(rgb_frame, face_locations) face_names = []for face_encoding inFace_encodings: # see if this face matches a known face match = face_recognition.compare_faces(known_face_encodings, face_encoding, tolerance=0.50Name = Noneif match[0]:
name = "Lin-Manuel Miranda"
elif match[1]:
name = "Stepthen Curry"Face_names.append (name) # Display recognition resultsfor (top, right, bottom, left), name in zip(face_locations, face_names):
if not name:
continueRectangle (frame, (left, top), (right, bottom), (0.0.255), 2Rectangle (frame, (left, bottom -25), (right, bottom), (0.0.255), cv2.FILLED)
font = cv2.FONT_HERSHEY_DUPLEX
cv2.putText(frame, name, (left + 6, bottom - 6), font, 0.5, (255.255.255), 1Print () # print()"Writing frame {} / {}".format(frame_number, length))
output_movie.write(frame)
input_movie.release()
cv2.destroyAllWindows()
Copy the codeBecause the video is relatively large, I will not do a demonstration, you can try yourself interested.
Handwriting neural network
So what is a neural network? Let’s explore neural networks with a simple example.
1. The neurons
As shown in the figure, the biological neuron includes the cell body, dendrites, axons and other parts.
Dendrites are used to receive input information, which is processed by synapses. When certain conditions are reached, the neuron is activated. On the contrary, if the corresponding conditions are not reached, the neurons are in the inhibitory state.
Inspired by biological neurons, the neural network model was developed. Neurons are the basic components of neural networks. They receive input and then perform some mathematical operations to produce output.
As shown below, transform the neurons into mathematical shapes
In this neuron, let x1 and x2 be inputs, w1 and w2 be weights, and b be bias,
The math is as follows:
x1w1+x2w2+b
Finally, the activation function is processed to obtain:
Y = f(x1 × w1 + x2 × w2 + b)Copy the codeConsider: Why use activation functions?
If you don’t use the excitation function (equivalent to the activation function f(x) = x), in which case the input of each layer node is a linear function of the output of the upper layer, it can be easily verified. No matter how many layers you have in your neural network, the output is a linear combination of the input, which is equivalent to having no hidden layer. This situation is the most primitive Perceptron, so the approximation ability of neural network is quite limited.
Because of this, the nonlinear function is introduced as the excitation function, so that the expression ability of deep neural network is more powerful (it is no longer a linear combination of inputs, but can approximate almost any function).
An activation function converts an unrestricted input into an output of predictable form between 0 and 1, that is, squeezing numbers in the range (−∞,+∞) down to (0, 1). The larger the positive value is, the closer the output is to 1. The larger the negative value is, the closer the output is to 0.
2. Build a neural network
A neural network is actually composed of a number of neurons, as follows:
The network has two inputs, a hidden layer (H1 and H2) containing two neurons, and an output layer (O1) containing one neuron.
The hidden layer is the part sandwiched between the input layer and the output layer. A neural network can have multiple hidden layers.
The process of passing input from a neuron forward for output is called feedforward.
That is:
H1 =f(x1 × w1 + x2 × w2 + b1) H2 =f(x1 × w3 + x2 × w4 + b2) o1=f(h1 × w5 + h2 × w6 + b3) o1=f(x1 × w5 + h2 × w6 + b3)Copy the code3. Train the neural network
Training neural network is actually a process of iterative optimization, which can be understood as finding the optimal solution.
The next goal is to predict a person’s sex from their height and weight. There are the following data:
In order to simplify the calculation, the height and weight of each person are subtracted a fixed value (weight -100, height -170), and the gender of male is defined as 0 and that of female as 1.
Before we train a neural network, we need to have a standard definition of whether it’s good or not, so that we can improve it, and that’s the loss.
If loss is defined by ** mean square error ** :
Formula (1)
N is the number of samples, 4 in the data set above;
Y represents the gender, male is 0, female is 1;
Y_real is the true value of the variable and y_expect is the predicted value of the variable.
The mean square error is the average of all data variances (the loss function MSE). The better the prediction, the lower the loss, and training the neural network is to minimize the loss.
For example, the output of the network above is always y_real=0, which predicts that everyone will be female, then the loss is:
MSE= 1/4 [(10) ^2+ (00) ^2+ (00) ^2+ (10) ^2] =0.5
Copy the code4. Reduce neural network losses
The above results are not good enough at first sight, so we need to constantly optimize the neural network to reduce losses. Therefore, the output of the neural network can be affected by changing the weight (w) and bias (b) values. For the sake of calculation, I’m going to reduce the data set to just one person.
Thus, the calculation of the above MSE can be simplified as:
Formula (2)
The predicted value is calculated from a series of network weights and biases:
So the loss function is actually a multivariate function with multiple weights and biases:
M(w1, w2, w3, w4, w5, w6, b1, b2, b3)
Copy the codeHow does MSE change if YOU adjust W1? Here requires a partial derivative partial L/partial w1 is negative to verify this problem.
The next step is a series of calculations, using the chain rule:
Formula (3)
Then, according to the variance calculation formula ① and ②,
Formula (4)
(4) substitution
Formula (5)
And according to the neuron operation rules:
The formula 6.
From a single neuron, h1 and W1 are closely related, so the chain rule is used again to obtain:
Formula 7 * * * *
Will 6 substitution
The formula was * * * *
Run the neuron algorithm again to get:
Formula pet-name ruby
So let’s plug in the above equation
Formula for attending
Finally, the formula of activation function
Formula ⑪
Derivation of activation function is:
Formula ⑫
To sum up, from ③⑦ :
Formula ⑬
By ⑬ (5) end up attending
Assuming x1=-2, x2=-1, w1=w2=w3=w4=w5=w6=1,b1=b2=b3=0, substitute into the neuron calculation formula:
Therefore, the value o1=y(expect)=0.524 is not a strong predictor of gender, suggesting that it is not sufficiently accurate.
∂M/∂w1=0.0214, w1 is proportional to M, and M increases with the increase of W1.
5. Iterative optimization
It is not enough to optimize the display from above, so we need to go through the iterative optimization of stochastic gradient descent (SGD).
** Random gradient descent (SGD) ** is the updating of parameters using one sample per iteration. Make the training speed up. The ordinary BGD algorithm runs through all the samples in each iteration and updates the gradient once for each training group of samples. The SGD algorithm randomly selects a group of samples, updates it according to the gradient after training, and then selects another group and updates it again. In the case of extremely large sample size, it is possible to obtain a model with an acceptable loss value without training all samples.
T is the variable,
Code Demo (note that the neural network on the project is not so simple) :
Import numpy as NP def sigmoID (x): # Sigmoid activation function: f(x) = 1 / (1 + e^(-x))) converts unrestricted input to predictable output between 0 and 1. # is to compress the range of (−∞,+∞) into (0, 1). The larger the positive value is, the closer the output is to 1. The larger the negative value is, the closer the output is to 0. Return 1 / (1 + np.exp(-x)) def deriv_sigmoid(x): f'(x) = f(x) * (1 - f(x)) fx = sigmoid(x) return fx * (1 - fx) def mse_loss(y_real, y_expect): # Calculate the loss function, which is actually the average of all data variances (mean square error). The better the prediction result, the lower the loss will be. Return ((y_real-y_expect) ** 2). Mean () class NeuralNetworkDemo: A neural network consists of: - 2 inputs - 1 hidden layer with 2 neurons (H1, H2) - 1 output layer (o1) "def __init__(self): Normal () self.w1 = np.random.normal() self.w2 = np.random.normal() self.w3 = np.random.normal() self.w4 = Np.random.normal () self.w5 = np.random.normal() self.w6 = np.random.normal() self.b2 = np.random.normal() self.b3 = np.random.normal() def feedforward(self, x): # x includes x[0], x[1], calculate feedforward: H1 = sigmoid(self.w1 * x[0] + self.w2 * x[1] + self.b1) h2 = sigmoid(self.w3 * x[0] + self.w4 * x[1] + Self.b2) o1 = sigmoid(self.w5 * h1 + self.w6 * h2 + self.b3) o1 = sigmoid(self.w5 * h1 + self.w6 * h2) # 2. Calculate the partial derivative of loss function with respect to ownership weight and bias; # 3. Update each weight and bias using the update formula; # 4. Go back to Step 1. def train(self, data, all_y_reals): ''' - data is a (n x 2) numpy array, n = # of samples in the dataset. - all_y_reals is a numpy array with n elements. Elements in all_y_reals correspond to "Learn_rate = 0.1 # Iterations = 1000 for Iteration in range(iterations): for x, y_real in zip(data, all_y_reals): Sum_h1 = self.w1 * x[0] + self.w2 * x[1] + self.b1 h1 = sigmoid(sum_h1) # Sum_h2 = self.w3 * x[0] + Self.w4 * x[1] + self.b2 h2 = sigmoid(sum_h2) # Calculate the feedforward of o1 sum_o1 = self.w5 * h1 + self.w6 * h2 + self.b3 o1 = Sigmoid (sum_o1) y_expect = o1 # -- calculate partial partial derivative # -- d_L_d_w1 means "partial derivative L/partial derivative w1" d_L_d_ypred = -2 * (y_real-y_expect) # O1 d_YpreD_W5 = H1 * deriv_sigmoid(sum_o1) d_Ypred_W6 = H2 * deriv_sigmoid(sum_o1) d_ypred_B3 = deriv_sigmoid(sum_o1) d_ypred_d_h1 = self.w5 * deriv_sigmoid(sum_o1) d_ypred_d_h2 = self.w6 * deriv_sigmoid(sum_o1) # Neuron H1 d_H1_D_W1 = x[0] * deriv_sigmoid(sum_H1) d_H1_W2 = x[1] * deriv_sigmoid(sum_H1) d_H1_B1 = D_h2_d_w3 = x[0] * deriv_sigmoid(sum_H2) d_H2_D_W4 = x[1] * deriv_sigmoid(sum_H2) D_h2_d_b2 = deriv_sigmoid(sum_H2) # -- Update weights and bias values # neuron h1 self.w1 -= learn_rate * d_L_d_ypred * d_ypred_H1 * d_h1_d_w1 self.w2 -= learn_rate * d_L_d_ypred * d_ypred_d_h1 * d_h1_d_w2 self.b1 -= learn_rate * d_L_d_ypred * W3 -= learn_rate * d_L_d_ypred * d_ypred_h2 * d_H2_d_w3 self. W4 -= learn_rate * D_l_ypred * d_ypred_h2 * d_h2_d_w4 self.b2 -= learn_rate * d_l_ypred * d_ypred_h2 * d_h2_d_b2 # neuron o1 self.w5 -= learn_rate * d_L_d_ypred * d_ypred_d_w5 self.w6 -= learn_rate * d_L_d_ypred * d_ypred_d_w6 self.b3 -= learn_rate * Calculate total loss at the end of each iteration - Calculate total loss at the end of each iteration if Iteration % 10 == 0: y_expects = np.apply_along_axis(self.feedforward, 1, data) loss = mse_loss(all_y_reals, y_expects) print("Iteration %d Mes loss: %, 3 f "% (iteration, loss)) # define data set data = np, array ([[2, 1], [25, 6], [17, 4], [15, - 6]. ]) all_y_reals = np.array([1, 0, 0, 1,]) # network = NeuralNetworkDemo() network.Copy the codeRunning results:
Iteration 0 Mes loss: 0.335
Iteration 10 Mes loss: 0.205
Iteration 20 Mes loss: 0.128
Iteration 30 Mes loss: 0.087
Iteration 40 Mes loss: 0.063
Iteration 50 Mes loss: 0.049
Iteration 60 Mes loss: 0.039
Iteration 70 Mes loss: 0.032
Iteration 80 Mes loss: 0.028
Iteration 90 Mes loss: 0.024
Iteration 100 Mes loss: 0.021
Iteration 110 Mes loss: 0.019
Iteration 120 Mes loss: 0.017
Iteration 130 Mes loss: 0.015
Iteration 140 Mes loss: 0.014
Iteration 150 Mes loss: 0.013
Iteration 160 Mes loss: 0.012
Iteration 170 Mes loss: 0.011
Iteration 180 Mes loss: 0.010
Iteration 190 Mes loss: 0.010
Iteration 200 Mes loss: 0.009
Iteration 210 Mes loss: 0.009
Iteration 220 Mes loss: 0.008
Iteration 230 Mes loss: 0.008
Iteration 240 Mes loss: 0.007
Iteration 250 Mes loss: 0.007
Iteration 260 Mes loss: 0.007
Iteration 270 Mes loss: 0.006
Iteration 280 Mes loss: 0.006
Iteration 290 Mes loss: 0.006
Iteration 300 Mes loss: 0.006
Iteration 310 Mes loss: 0.005
Iteration 320 Mes loss: 0.005
Iteration 330 Mes loss: 0.005
Iteration 340 Mes loss: 0.005
Iteration 350 Mes loss: 0.005
Iteration 360 Mes loss: 0.005
Iteration 370 Mes loss: 0.004
Iteration 380 Mes loss: 0.004
Iteration 390 Mes loss: 0.004
Iteration 400 Mes loss: 0.004
Iteration 410 Mes loss: 0.004
Iteration 420 Mes loss: 0.004
Iteration 430 Mes loss: 0.004
Iteration 440 Mes loss: 0.004
Iteration 450 Mes loss: 0.004
Iteration 460 Mes loss: 0.003
Iteration 470 Mes loss: 0.003
Iteration 480 Mes loss: 0.003
Iteration 490 Mes loss: 0.003
Iteration 500 Mes loss: 0.003
Iteration 510 Mes loss: 0.003
Iteration 520 Mes loss: 0.003
Iteration 530 Mes loss: 0.003
Iteration 540 Mes loss: 0.003
Iteration 550 Mes loss: 0.003
Iteration 560 Mes loss: 0.003
Iteration 570 Mes loss: 0.003
Iteration 580 Mes loss: 0.003
Iteration 590 Mes loss: 0.003
Iteration 600 Mes loss: 0.003
Iteration 610 Mes loss: 0.003
Iteration 620 Mes loss: 0.002
Iteration 630 Mes loss: 0.002
Iteration 640 Mes loss: 0.002
Iteration 650 Mes loss: 0.002
Iteration 660 Mes loss: 0.002
Iteration 670 Mes loss: 0.002
Iteration 680 Mes loss: 0.002
Iteration 690 Mes loss: 0.002
Iteration 700 Mes loss: 0.002
Iteration 710 Mes loss: 0.002
Iteration 720 Mes loss: 0.002
Iteration 730 Mes loss: 0.002
Iteration 740 Mes loss: 0.002
Iteration 750 Mes loss: 0.002
Iteration 760 Mes loss: 0.002
Iteration 770 Mes loss: 0.002
Iteration 780 Mes loss: 0.002
Iteration 790 Mes loss: 0.002
Iteration 800 Mes loss: 0.002
Iteration 810 Mes loss: 0.002
Iteration 820 Mes loss: 0.002
Iteration 830 Mes loss: 0.002
Iteration 840 Mes loss: 0.002
Iteration 850 Mes loss: 0.002
Iteration 860 Mes loss: 0.002
Iteration 870 Mes loss: 0.002
Iteration 880 Mes loss: 0.002
Iteration 890 Mes loss: 0.002
Iteration 900 Mes loss: 0.002
Iteration 910 Mes loss: 0.002
Iteration 920 Mes loss: 0.002
Iteration 930 Mes loss: 0.002
Iteration 940 Mes loss: 0.002
Iteration 950 Mes loss: 0.002
Iteration 960 Mes loss: 0.002
Iteration 970 Mes loss: 0.002
Iteration 980 Mes loss: 0.002
Iteration 990 Mes loss: 0.001
Copy the codeIt can be seen that with the continuous iteration, the loss is getting smaller and smaller.
Next, you can use custom input to predict gender
import numpy as np
def sigmoid(x):
..........
def deriv_sigmoid(x):
..........
def mse_loss(y_real, y_expect):
..........
class NeuralNetworkDemo:
..........
def testPredict(self):
you = np.array([- 10.- 10#]) weight90The height160. For the x1 =90- 100.=- 10, x2 =160- 170.=- 1017. print("your output is ", network.feedforward(you)) # Define data set.......... # Training neural network.......... network.testPredict()Copy the codeTake a girl of 90 weight and 160 height to test, and the output result is:
your output is 0.9651728303105593
Copy the codeThe output is going to be close to 1 for as many times as you do this, so you can predict that this is a girl.
There are other types of neural network (CNN, RNN, DBN, GAN, etc.) that are extended and widely used, which can be discussed if you are interested.
This post is from The Toothfish community of Choerodon.