♚
Author: Yishui Hancheng, CSDN blog expert, personal research interests: machine learning, deep learning, NLP, CV
Blog: yishuihancheng.blog.csdn.net
I learned the perceptron algorithm when I first got into machine learning in school. Recently, I just used this model, so I just took it out for review. Here I summarize several key steps of perceptron construction and calculation, and the specific algorithm principle is as follows:
Perceptron algorithm is the most simple and most basic neural network model, because it is only a layer of network structure, the computing power and ability are limited, the limited here is not to say that its computing power is very weak, for linear calculation problem: theoretically perceptron can calculation model of the linear fitting out any form; For practical problems or nonlinear problems: since most calculations in real life are nonlinear expression relations, it is still difficult for the perceptron model to accurately capture the characteristics of data. The emergence of neural network solves this problem well, so I will not expand it here. Perceptron is a machine model that simulates biological neurons. The schematic diagram of a simple perceptron model is shown in the figure below:
Given an n-dimensional input, where W and B are parameters, w is the weight, each input corresponds to a weight, and B is the bias term, which needs to be trained from the data. Activation functions have more choices, the more common are sigmoID function and step function. I will not explain the calculation of the formula here. In order to see the similarities and differences of the two activation functions more clearly, I have carried out visual processing here, and give the specific code implementation first, as shown below:
1.def jieyueFunc(x):
2. ' '' ''3. Define perceptron step function 4.'' '
5. return np.where(x<0.0.1)
6.
7.
8.def jieyueFuncDerivative(x):
9. ' '' ''10. Define the perceptron step function derivative 11.'' '
12. return np.where(x<0.0.0)
13.
14.
15.def sigmoid(x):
16. ' '' 'Define sigmoid function 18.' '
19. return 1.0/ (1.0+np.exp(-x))
20.
21.
22.def sigmoidDerivative(x):
23. ' '' ''24. Define sigmoid derivative 25.'' '
26. return sigmoid(x)*(1-sigmoid(x))
Copy the codeThe above code defines the SigmoID function and step function and their corresponding derivative function, the following function curve visualization, implementation is as follows:
1.x=np.arange(-10.10.0.1)
2.y1=sigmoid(x)
3.y2=jieyueFunc(x)
4.y11=sigmoidDerivative(x)
5.y22=jieyueFuncDerivative(x)
6.plt.clf()
7.plt.plot(x,y1,label='sigmoid')
8.plt.plot(x,y2,label='jieyue')
9.plt.plot(x,y11,label='sigmoidDerivative')
10.plt.plot(x,y22,label='jieyueDerivative')
11.plt.title('ActivateFunction Compare')
12.plt.legend()
13.plt.show()
Copy the codeThe results are shown below:
The above comparison is clear. From the value range of the original function, there is no big difference between the two. Sigmoid function is a gradual trend, and step function is a sudden trend. From the point of view of derivative function: Sigmoid derivative function curve presents the characteristic of “normal distribution”, and the derivative function of step function is always 0.
On the perceptron algorithm further theoretical introduction of this paper is not much involved, there are also a lot of online learning materials for reference, today is mainly combined with the actual application needs to practice the perceptron algorithm model.
Here, we randomly constructed data sets of 5 sample points (only for demonstration and can be modified according to the actual situation), as shown below:
Among them, the data set is divided into two categories: 0 and 1, which contain 2 and 3 sample data respectively. The first column No indicates that Id serial number is not of practical use, and the last column represents the label of sample data.
Before explaining the specific implementation, the visualized data of model classification results is shown as the figure below:
Where, the blue dot represents the sample data labeled 0, and the red dot represents the sample data labeled 1. It can be seen that the model has correctly realized classification.
Let’s look at the specific code implementation, as follows:
1.def demoModel(datasets,x_total,W,n,iters,pic_path='demo_res.png') :2. ' '' ''3. Perceptron model 4.'' '
5. if not os.path.exists('my/') :6. os.mkdir('my/')
7. x1,x2,t=datasets[:, 1], datasets[:, 2], datasets[:, 3]
8. label_list=[x_total[i][-1] for i in range(len(x_total))]
9. print('label_list: ',label_list)
10. index,right=0.0
11. for i in range(iters):
12. ifindex> =5:
13. index,right=0.0
14. node=inputNetCal(x_total[index][:3],W)
15. bias=biasCal(label_list[index],node,1)
16. if x_total[index][3] = =1and node< =0:
17. W=netWadjust(W,n,x_total[index][:3].1)
18. elif x_total[index][3] = =0and node> =0:
19. W=netWadjust(W,n,x_total[index][:3],-1)
20. plt.cla()
21. for j in range(len(x1)):
22. if t[j]==1:
23. plt.plot(x1[j],x2[j],'ro')
24. else:
25. plt.plot(x1[j],x2[j],'bo')
26. plt.xlim(-1.5.1.5)
27. plt.ylim(-3.3)
28. plt.grid(c='g')
29. plt.title('Dynamic Ploter')
30. x_list=np.arange(-2.2.0.01)
31. plt.plot(x_list,-((W[1]/W[2])*x_list)-(W[0]/W[2]))
32. plt.savefig('my/'+str(i)+'.png')
33. plt.pause(0.1)
34. if x_total[index][3] = =1 and node>0:
35. right+=1
36. elif x_total[index][3] = =0 and node<0:
37. right+=1
38. index+=1
39. print('index: {0}, right: {1}'.format(index,right))
40. if right==5:
41. print('Best W: ',W)
42. plt.savefig('demoModel.png')
43. break
44. return W.tolist(),bias
Copy the codeThe above code implements the perceptron algorithm model and classifies the sample data in the data set provided by us.
The implementation of the code is very simple, and I won’t explain it here because it is based on the theory of perceptrons, but a few key points will be explained.
Adaptive adjustment of network weight and bias calculation:
1.def netWadjust(W,n,X,flag):
2. ' '' ''3. Adaptive adjustment of network weight 4.'' '
5. return W+((n*X)*flag)
6.
7.
8.def biasCal(label,output,n):
9. ' '' ''10. Calculate the offset item 11.'' '
12. bias=label-output
13. bias+=n*bias
14. return bias
Copy the codeSingle neuron calculation:
1.def inputNetCal(X,W):
2. ' '' ''3. Calculate 4.'' '
5. print('W====> ',W)
6. res=np.dot(X,W)
7. return res
Copy the codeExcel data set loading processing:
1.def write2Excel(data_list,targetfile='res.xls') :2. ' '' ''3. Transpose the contents of TXT to Excel 4.'' '
5. j=0
6. book=xlwt.Workbook(encoding='utf-8')
7. table=book.add_sheet('w_b')
8. for one_list in data_list:
9. for i in range(len(one_list)):
10. table.write(i,j,one_list[i])
11. j+=1
12. book.save(targetfile)
13.
14.
15.def readOneExcelData(data='data/Homework4.xlsx',row=1,col=1) :16. ' '' ''17. Row: start row index 18. Col: start column index 19.'' '
20. workbook=xlrd.open_workbook(data,encoding_override="utf-8")
21. table=workbook.sheets()[0]
22. row_num,col_num=table.nrows,table.ncols
23. data_list=[]
24. for i in range(row,row_num):
25. one_list=[]
26. for j in range(col,col_num):
27. one_list.append(table.cell_value(i,j))
28. data_list.append(one_list)
29. return np.array(data_list),[np.array(one) for one in data_list]
Copy the codeAt this point, the complete implementation process is introduced. When the above code is actually run, the classification situation in each round can be dynamically drawn, and the dynamic visualization of the model output can be realized. At the same time, THE classification results of each time and the final classification result images are saved.
If you need to use dynamic visualization in the process of PPT or explanation, you can synthesize it based on the saved single classification result data, and synthesize GIF code as follows:
1.def createGif(picDir='pic/',save_path='a.gif',duration=0.35) :2. ' '' ''3. Create GIF image 4.'' '
5. pic_list=os.listdir(picDir)
6. pic_list.sort()
7. image_list=[picDir+one for one in pic_list]
8. frames=[]
9. for image_name in image_list:
10. frames.append(imageio.imread(image_name))
11. imageio.mimsave(save_path,frames,'GIF',duration=duration)
12. print('Finished!!!! ')
Copy the codeIn just a few lines of code, static image data composite GIF is realized, where PicDir represents the storage path of the original static image data, save_path represents the storage path of the generated GIF image, duration represents the time interval between two adjacent images when the image is created.
Above, we have implemented a simple perceptron algorithm model ourselves. Spark provides a package of multi-layer perceptron model, which makes it easy to perform call, instance and experimental analysis. Here we briefly explain it, if you are interested, you can check out the series of articles in my blog.
MLP is a forward structured artificial neural network that maps a set of input vectors to a set of output vectors. An MLP can be thought of as a directed graph consisting of multiple node layers, each of which is fully connected to the next. Except for the input node, each node is a neuron (or processing unit) with a nonlinear activation function. A supervised learning method called a backpropagation algorithm is often used to train MLPS. MLP is a generalization of the perceptron, which overcomes the shortcoming that the perceptron can’t recognize linear non-fractional data. Multi-layer Perceptron (MLP), also called Artificial Neural Network (ANN), can have multiple hidden layers in addition to the input and output layers. The simplest MLP requires a hidden layer, namely an input layer, a hidden layer, and an output layer, to be called a simple neural network.
If you are interested, please check out the pySpark series of practices in my CSDN blog. Here is a brief introduction to use.
A simple SCHEMATIC of the MLP result is shown below:
The above is the MLP neural network model with only one hidden layer, and the following is the schematic diagram of the neural network with two hidden layers:
As the name implies, PySpark is a combination of Python and Spark. Spark provides a Python_Shell, or PySpark, so that Spark programs can be written in Python in an interactive manner. The basic architecture of Spark and the configuration of the PySpark environment will not be covered in this blog post, but there are plenty of good ones on the web.
Next, the MLP classification model based on PySpark is implemented as follows:
1.#! usr/bin/env python2.#encoding:utf-8
3.from __future__ import division
4.
5.
6.' '' 'Function: PySpark Machine Learning Practice [Multi-layer perceptron classifier model] 9.'' '
10.
11.
12.
13.import findspark
14.findspark.init()
15.import pyspark
16.from pyspark import SparkConf
17.from pyspark.ml import Pipeline
18.from pyspark.context import SparkContext
19.from pyspark.sql.session import SparkSession
20.from pyspark.ml.classification import DecisionTreeClassifier
21.from pyspark.ml.evaluation import MulticlassClassificationEvaluator
22.from pyspark.ml.feature import StringIndexer, VectorIndexer,IndexToString
23.
24.
25.
26.conf=SparkConf().setAppName('MLDemo')
27.sc = SparkContext('local')
28.spark = SparkSession(sc)
29.
30.
31.def MLPClassifier(data="mllib/sample_multiclass_classification_data.txt") :32. ' '' 'The multi-layer Perceptron classifier (MLPC) is a classifier based on feedforward artificial neural network. MLPC consists of several layers of nodes. Each layer is fully connected to the next layer in the network. Nodes in the input layer represent input data. All other nodes map inputs to outputs by linear combinations of inputs with node weights W and bias B, and apply activation functions. Multilayer perceptron classifier 35. '' '
36.Load the data set in LIBSVM format37. data = spark.read.format("libsvm").load(data)
38.# Division of training set and test set39. splits = data.randomSplit([0.6.0.4].1234)
40. train = splits[0]
41. test = splits[1]
42.# specify the layer of the neural network43.# input layer4The number of neurons in the two layers is as follows:5,4And the output layer3A neuron44. layers = [4.5.4.3]
45. trainer = MultilayerPerceptronClassifier(maxIter=100, layers=layers, blockSize=128, seed=1234)
46. model = trainer.fit(train)
47. result = model.transform(test)
48. predictionAndLabels = result.select("prediction"."label")
49. evaluator = MulticlassClassificationEvaluator(metricName="accuracy")
50. print("Test set accuracy = " + str(evaluator.evaluate(predictionAndLabels)))
51.
52.
53.
54.if __name__=='__main__':
55. MLPClassifier(data="mllib/sample_multiclass_classification_data.txt")
Copy the codeThe data set used above is the experimental data set provided in pySpark’s own machine learning model. If there are requirements on data sets that can be obtained from here, I have built a resource warehouse of data sets that are often used in daily analysis and mining, the link is as follows:
Github.com/yishuihanha…
The screenshot of the Data Resource warehouse is shown below. The data sets used above are stored in the directory marked by the red box below.
The simple test output is as follows:
I am very glad to write something to share while reviewing my knowledge. If you think my content can or is enlightening and helpful to you, I also hope to get your encouragement and support, thank you!
Appreciate the author
Python Chinese community as a decentralized global technology community, to become the world’s 200000 Python tribe as the vision, the spirit of Chinese developers currently covered each big mainstream media and collaboration platform, and ali, tencent, baidu, Microsoft, amazon and open China, CSDN industry well-known companies and established wide-ranging connection of the technical community, Have come from more than 10 countries and regions tens of thousands of registered members, members from the ministry, tsinghua university, Peking University, Beijing university of posts and telecommunications, the People’s Bank of China, the Chinese Academy of Sciences, cicc, huawei, BAT, such as Google, Microsoft, government departments, scientific research institutions, financial institutions, and well-known companies at home and abroad, nearly 200000 developers to focus on the platform.
Click to become a Registered member of the Community.