Link to original article:tecdat.cn/?p=20781 

Original source:Tuoduan numbers according to the public account of the tribe

 

What is a neural network?

Artificial neural networks were originally developed by researchers who tried to mimic the neurophysiology of the human brain. By combining many simple computational elements (neurons or units) into highly interconnected systems, these researchers hope to produce complex phenomena such as intelligence. Neural network is a kind of flexible nonlinear regression discriminant model. By detecting the complex nonlinear relationships in the data, the neural network can help to make predictions about practical problems.

Neural networks are particularly useful for prediction problems with the following conditions:

  • There is no mathematical formula that relates input to output.
  • Predictive models are more important than explanatory models.
  • There’s a lot of training data.

Common applications of neural networks include credit risk assessment, marketing and sales forecasting.

NeuralNet is based on a multilayer perceptron (MLP) and has the following characteristics:

  • There’s any number of inputs
  • Use linear combination functions in the hidden layer and the output layer
  • Use the S-type activation function in the hidden layer
  • Has one or more hidden layers containing any number of units

 

Using neural network functions

The neuralNet trains the network with a minimal objective function.

When developing a neural network, you need to make many parameter choices: the number of inputs to use, the basic network architecture to use, the number of hidden layers to use, the number of units per hidden layer, the use of activation functions to use, and so on.

You probably don’t need any hidden layers at all. Linear models and generalized linear models can be used for a variety of applications. Moreover, even if the function to be learned is slightly nonlinear, if there is too little data or too much noise to accurately estimate the nonlinearity, it is possible to achieve better results using a simple linear model than using a complex nonlinear model. The easiest way to do this is to start with a network with no hidden units and then add one hidden unit at a time. The errors of each network are then estimated. When the error increases, stop adding hidden units.

With enough data, enough hidden units, and enough training time, an MLP with only one hidden layer can learn the accuracy of almost any function.

 

Generate independent SAS scoring code for the neural network model

 

After the neural network model has been trained and validated, it can be used to score new data. New data can be graded in a number of ways. One approach is to submit new Data, then run the model and use Data Mining to score the Data through SAS Enterprise Miner or SAS Visual Data Mining and Machine Learning to generate scoring output.

This example shows how to use neuralNet operations to generate independent SAS scoring code for an ANN model. SAS scoring code can be run in a SAS environment without the SAS Enterprise Miner license.

 

Create and train neural networks

AnnTrain will create and train an artificial neural network (ANN) for classification, regression functions.

This example uses the Iris data set to create a multilayer perceptron (MLP) neural network. The Iris data published by Fisher (1936) contained 150 observations. Sepal length, sepal width, petal length, and petal width were measured in millimeters from 50 specimens of three species each. The four measurement types become the input variables. The class name becomes the nominal target variable. The aim was to predict the species of iris by measuring the size of its petals and sepals.

You can load the dataset into the session by following the DATA steps.

 data mycas.iris;
    set sashelp.iris;
 run;
Copy the code

There are no missing values in Iris data. This is important because the annTrain operation removes observations that contain missing data from the model training. If the input data to be used for the neural network analysis contains observations with a large number of missing values, the missing values should be replaced or estimated before the model training is performed. Because the Iris data does not contain any missing values, this example does not perform variable substitutions.

This example uses annTrain to create and train a neural network. The neural network predicts the function of predicting the species of the iris based on the input of the length and width (in millimeters) of its sepals and petals.



    target="species"                                               
    inputs={"sepallength","sepalwidth","petallength","petalwidth"}  
    nominals={"species"}                                            
    hiddens={2}                                                     
    maxiter=1000                                                   
    seed=12345                                                 
    randDist="UNIFORM"                                            
    scaleInit=1                                                 
    combs={"LINEAR"}                                           
    targetAct="SOFTMAX"                                        
    errorFunc="ENTROPY"                                      
    std="MIDRANGE"                                             
    validTable=vldTable                                        
Copy the code
  1. usesampling.StratifiedoperationIrisPartition the input data by target variableSpecies.
  2. Add the partition indication column_Partind_To the output table. the_Partind_Columns contain integer values that map to the data partition.
  3. Create a sample partition with 30% of the table observationsSpecies. The remaining 70% of table observations form the second partition.
  4. The specified12345The random seed value to use for the sampling function.
  5. namedsampling_stratifiedOutput table created by operation (with new partition information column)iris_partitioned. If a table with that name exists in memory, the existing table will be newiris_partitionedTable contents overwrite.
  6. Specify all variables in the source table and transmit them to the sampled table.
  7. Create separate tables using the data from the newly added partition columns for neural network training and validation. Make training tabletrnTableFor all the observations in the tableiris_partitionedA subset of values, where the integer values of the columns_Partind_Is equal to 1.
  8. Create separate tables using data from newly added partition columns for neural network training and validation. Hypothesis verification tablevldTableIs all the observations in the tableiris_partitionedA subset of values, where the integer values of the columns_Partind_Equal to zero.
  9. annTrainThrough the use oftrnTableTable with target variables to create and train MLP neural networksSpecies.
  10. Specifies four input variables to be used as analysis variables for ANN analysis.
  11. Requires that the target variable beSpeciesAs a nominal variable for the analysis.
  12. The number of hidden neurons is specified for each hidden layer in the neural network feedforward model. For example,hiddens={2}Specify a hidden layer with two hidden neurons.
  13. Specifies the maximum number of iterations to perform while finding convergence of the objective function.
  14. Specifies the random seed used to perform sampling and partitioning tasks.
  15. The UNIFORM distribution is required to be used to randomly generate the initial neural network connection weights.
  16. Specifies the scaling factor for the join weights relative to the number of units in the preceding layer.scaleInitThe default value for the parameter is 1. Set the value of the parameterscaleInitA value of 2 increases the ratio of connection weights.
  17. Specifies the LINEAR combination function for the neurons in each hidden layer.
  18. Specify the activation function for the neuron in the output layer. By default, the SOFTMAX function is used for nominal variables.
  19. Specify an error function to train the network. ENTROPY is the default setting for the nominal target.
  20. Specifies the normalization to be used on interval variables. whenstdWhen the value of the argument is MIDRANGE, the variables are normalized to 0 and 1.
  21. Specifies the input table name to be used to validate the table. This can be done by usingoptmlOptParameter to stop the iterative process early.
  22. The specifiedNnet_train_modelAs an output table.
  23. Enable the neural algorithm solver optimization tool.
  24. Specify 250 maximum iterations for optimization, and 1E — 10 as the threshold stop value for the objective function.
  25. The LBFGS algorithm is enabled. LBFGS is an optimization algorithm in the family of quasi-Newtonian methods that approximates the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm by using limited computer memory.
  26. Use the frequency parameter to set the validation options. whenfrequencyParameter with a value of 1 is validated at each period. whenfrequencyIs 0, no validation is performed.

Output an overview of the display data.

Output: column information

Result from table.columnInfo

 

 

If you use the table.fetch command on the input table, you can see the sample data rows shown in output 2.

Output 2: Extracted rows

The result from table.fetch

 

If simple.freq uses the command on the input table, it can verify that there are 50 observations in each of the three categories, for a total of 150 observations in the input data table, as shown in output 3.

 

Output 3: Species frequency

 

Results from simple.freq

 

After Iris successfully completes the neuralnet.anntrain training process for the input table, the result will be a training data iteration history containing the objective function, loss, and validation error columns, as shown in output 4.

 

Output 4: Optimize iteration history

 

Results from neuronet.anntrain

 

Below the Iteration History table, you should see the Convergence state table. For successful neural network models, the “convergence state” should report “optimization has converged”, as shown in output 5.

 

Output 5: convergence

 

 

Successful model training involves output model summary results, as shown in output 6.

 

Output 6: model information

These results reaffirm key model-building factors, such as model type; Abstract of input, hidden and output nodes of the target variable neural network model; Weight and deviation parameters; Final target value; And the classification error of the score verification data set.

At the bottom of the table, you will see the final percentage of misclassification errors determined by the validation data. If you use this neural network model as a predictive function, and your data comes from the same data distribution as the Iris validation table, you can expect 93 — 94 percent of the species predictions to be correct.

 

A neural network model was used to score the input data

After the neural network model has been trained and validated, it can be used to score new data. The most common technique is to use a Data Mining environment to generate scoring output through SAS Enterprise Miner or SAS Visual Data Mining and Machine Learning to submit new Data and run models to score the new Data.

With the trained neural network, the neural network model and annScore operation can be used to score the new input data, as shown below:


    table=vldTable
    modelTable="train_model";
Copy the code
  1. Identify the training data sheet. The training data isiris_partitionedIn the partition indicator column (_partind_) is 0.
  2. Validating the data table. Validation data isiris_partitionedTable, in the partition indicator column (_partind_) is 1.
  3. Score the training data. Submit input data, which will be scored by a trained neural network model. Because the data to be graded in this code block is model training data, you should expect the scoring code to read all 105 observations and mispredict the target variable value with 0% misclassification. The model training data contains known target values, so the classification error of 0% should be expected when scoring the model training data.
  4. Score validation data. This operation submits input data, which is scored by a trained neural network model in a SAS data mining environment. Validation data contains known target values, but the training algorithm does not read the validation data. The algorithm predicts the target value for each observation in the validation data, and then compares the predicted value with the known value. The percentage of the classification error is calculated by subtracting from 1 the percentage of the classification that is correctly predicted. A lower percentage of classification error usually indicates better model performance.

The validation data contains 30% of the original input data observations and is layered by the target variable Species. The original data contained 50 observations for each category; The validation data (30%) contained 15 observations for each of the three species for a total of 45 observations. If 42 out of 45 observations in the validation data are correctly classified, the misclassification error of the model is 6.67%.

 

Most popular insight

1. R language uses neural network to improve nelson-Siegel model fitting yield curve analysis

2. R language can predict the fitting neural network and visualize the results

3. Python uses genetic algorithm – neural network – fuzzy logic control algorithm to analyze the lottery

Python for NLP: Multi-label text LSTM neural network classification using Keras

5. Using R language to realize the neural network to predict the stock example

6. Deep learning image classification of small data sets based on Keras in R language

7. The example of SEQ2SEQ model for NLP is realized by using Keras

8. Deep learning model optimization based on grid search algorithm in Python to analyze sugar

9. Matlab uses Bayesian optimization for deep learning