• By Han Xinzi @Showmeai
• Tutorial address: www.showmeai.tech/tutorials/4…
• This paper addresses: www.showmeai.tech/article-det…
• Statement: All rights reserved, please contact the platform and the author and indicate the source
• Check out ShowMeAI for more highlights

The introduction

This article wants to take everyone to complete the machine learning application process again, we will explain to the machine learning algorithm based on Python, used in the structured data and unstructured data (image), hope that through the article content to help you to review basic knowledge of machine learning, in the case study and application of machine learning the basic process to solve the problem.

The following two libraries will be used in this article to implement the machine learning algorithm:

• Scikit-learn: One of the most commonly used python machine learning algorithm tools.
• Keras: Convenient deep learning neural network building application tool library.

Library for these two tools usage, you can also pass ShowMeAI article AI quick modeling tools | Scikit – Learn to use guidelines and AI quick modeling tools | Keras guidelines to quickly query and use.

In this article, we will cover the following:

• Problem abstraction and understanding
• Data preparation and processing (preprocessing, feature extraction, feature engineering, etc.)
• Various machine learning algorithms
• Analysis and comparison of experimental results
• Model selection and tuning

The machine learning algorithms we will cover include KNN, naive Bayes, logistic regression, SVM, decision tree, random forest, perceptron, feedforward neural network and convolutional neural network.

1. Environment preparation

Work must first sharpen his doing, we install the necessary first Python library (of course, we also recommend using integrated environmental anaconda, specific installation and setup can reference ShowMeAI articles graphic Python | installation and environment Settings complete) :

• Numpy: Scientific computation for Python.
  • Focus on knowledge, please view the data scientific tools quick | Numpy guidelines or graphical data analysis: from entry to master Numpy explanation tutorial in the series.
• PIL: a simple image processing library.
• Scikit-learn: Contains multiple machine learning algorithms.
  • Key knowledge please check AI quick modeling tools | Scikit – Learn to use guidelines
• Kears and TensorFlow: for deep learning. This tutorial can use only the CPU version of TensorFlow.
  • Key knowledge please check AI quick modeling tools | TensorFlow modeling tools use guide, AI quick | Keras use guide
• OpenCV: This tutorial does not use OpenCV directly, but the imUtils library relies on it.
  • Key knowledge please check AI quick modeling tools | OpenCV use guide
• Imutils: Image processing/computer vision library.

PIP can be installed using the following command:$ pip install numpy
$ pip install pillow
$ pip install --upgrade scikit-learn
$ pip install tensorflow # or tensorflow-gpu
$ pip install keras
$ pip install opencv-contrib-python
$ pip install --upgrade imutils
Copy the code

2. The data set

Because this article introduces the different modeling of structured and unstructured data, we use two data sets here.

2.1 Iris (Iris) data set

The first dataset is the IRIS (Iris) dataset, which is an entry level dataset. The entire dataset is numerical data, a structured table of data, with each row representing a sample and each column representing a different attribute.

This data set mainly collects data of three different kinds of iris, which are as follows:

• Iris Setosa
• Iris Versicolor
• Iris Virginica

Corresponding to the last column in the diagram, Class label, and then there are four attributes, respectively:

• Sepal length: length of a Sepal
• Sepal width: Sepal width
• Petal length: the length of a Petal
• Petal width: Petal width

For this data set, our goal is to train a machine learning model to correctly classify each sample category based on the given four attributes, which is a typical classification task.

2.2 Image data set

The second dataset is an image dataset. It includes three scenes of Coast, Forest and Highway, with a total of 948 pictures. We need to build a model to complete the classification of categories. The specific number of pictures of each category is as follows:

• Coast: 360
• Forest: 328
• Highway: 260

3. Application steps of machine learning

We apply machine learning algorithms in different scenarios, and there are general steps. For example, the following is a typical machine learning application process:

Of course, not all of these steps are necessary, and we may tweak the details of some of them.

3.1 Problem abstraction and understanding

In response to our question, ask yourself:

• What kind of data set is it? Numerical, categorical or graphic?
• What is the ultimate goal of the model?
• How to define and measure “accuracy”?
• Based on our current machine learning knowledge, which algorithms work well for these kinds of problems?

The first question is relatively simple, and the last question, as you gain more experience with machine learning, can be answered more accurately and quickly.

3.2 Data Preparation and processing

Data preparation and processing, including data preprocessing and feature engineering. This typically involves loading data, checking data, exploratory data analysis (EDA), and data preprocessing to determine the required feature extraction or feature engineering.

Feature extraction is the process of quantifying data by applying some algorithm in some way. For example, for image data, we can calculate the distribution of pixel intensity in the image by calculating the histogram. In this way, we can get the features that describe the color of the image.

Feature engineering is the process of converting raw input data into a feature representation that better describes a potential problem. You can check out ShowMeAI’s machine learning feature system to see common approaches to feature engineering.

3.3 Multi-model application

The next step is to select a variety of candidate machine learning algorithms and apply them to the data set. The kit we installed contains many machine learning algorithms, such as the following model, which can be used for classification:

• Linear model (logistic regression, linear SVM)
• Nonlinear models (RBF, SVM, gradient descent classifier)
• Trees and integration-based models (decision trees, random forests)
• Neural network (multilayer perceptron, convolutional neural network)

For model selection, of course, a lot of decisions need to be made according to experimental results, but we also have some prior experience, such as:

• For dense multi-feature data sets, the random forest algorithm performs well.
• Logistic regression algorithm can deal with high dimensional sparse data well.
• For image data, CNNs works very well.

The sciKit-learn tool library is an example of how to choose a model.

4. Build machine learning process and experiment analysis

We built the following directory of code files, including four code files and a folder of 3scenes images (containing data sets of three scenes). There is no need to store iris data set separately, and it can be directly loaded by scikit-learn library.

Three scenes ├ ─ ─ │ ├ ─ ─ a squadron [360 entries] │ ├ ─ ─ forest [328 entries] │ └ ─ ─ highway 260 entries] [├ ─ ─ iris_classifier. Py ├ ─ ─ ├─ nn_ir.py ├─ basic_cnn.pyCopy the code

4.1 Structured data modeling

Firstly, irIS_classifier is implemented. Here, we directly use the machine learning algorithm of SKLearn to classify iris data sets.

Import the required libraries
from sklearn.neighbors import KNeighborsClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.neural_network import MLPClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from sklearn.datasets import load_iris
import argparse

# set parameters
ap = argparse.ArgumentParser()
ap.add_argument("-m"."--model".type=str, default="knn".help="type of python machine learning model to use")
args = vars(ap.parse_args())

# define a dictionary to save models and select which model to load based on the key
models = {
    "knn": KNeighborsClassifier(n_neighbors=1),
    "naive_bayes": GaussianNB(),
    "logit": LogisticRegression(solver="lbfgs", multi_class="auto"),
    "svm": SVC(kernel="rbf", gamma="auto"),
    "decision_tree": DecisionTreeClassifier(),
    "random_forest": RandomForestClassifier(n_estimators=100),
    "mlp": MLPClassifier()
}
Copy the code

Models include these algorithms from front to back: KNN, naive Bayes, logistic regression, SVM, decision tree, random forest, perceptron, etc.

We call the corresponding function in SkLearn to implement the corresponding algorithm. Here, we directly use a dictionary of models to store the initialization of different models, and then call the corresponding model according to the parameter –model. For example, python iris_classifier.py –model KNN calls the KNN algorithm model.

Next comes the data loading part:

print("Loading data...")
dataset = load_iris()
trainX, testX, trainY, testY = train_test_split(dataset.data, dataset.target, random_state=3, test_size=0.2)
Copy the code

Load_iris () in sklearn.datasets is called directly to load the data, and train_test_split is used to split the training set and dataset, where 80% of the data is the training set and 20% is the test set.

Finally, the training model and prediction:

# Training model
print("Apply the '{}' model to modeling...".format(args["model"]))
model = models[args["model"]]
model.fit(trainX, trainY)

# Predict and output a report of classification results
print("Evaluating model effectiveness...")
predictions = model.predict(testX)
print(classification_report(testY, predictions, target_names=dataset.target_names))
Copy the code

4.2 Image data modeling

A similar procedure is used to build the code image_classifier.py for the three scene image datasets:

# import tool library
from sklearn.preprocessing import LabelEncoder
from PIL import Image
from imutils import paths
import numpy as np
import os
Copy the code

LabelEncoder is used to encode the label from a string to an integer, and the rest of the items are used to deal with image correlation.

For image data, if the input model directly USES the original pixel information, most of the effect of machine learning algorithms is not ideal, so the method of feature extraction, mainly image mean and standard deviation of the color channel information statistics, a total of RGB3 channels, each channel to calculate the mean and standard deviation, and then together, get a six – dimensional characteristics, The function looks like this:

def extract_color_stats(image) :
    Divide the image into THREE RGB channels, then calculate the mean and standard deviation of each channel separately, and return: param image: :return: ""
    (R, G, B) = image.split()
    features = [np.mean(R), np.mean(G), np.mean(B), np.std(R), np.std(G), np.std(B)]

    return features
Copy the code

It will also define a Models dictionary, which will not be posted here, and the image load section will look like this:

Load data and extract features
print("Extract image features...")
imagePaths = paths.list_images(args['dataset'])
data = []
labels = []

Loop over all image data
for imagePath in imagePaths:
    Load the image, then calculate the image's color channel statistics
    image = Image.open(imagePath)
    features = extract_color_stats(image)
    data.append(features)

    # Save the tag information of the image
    label = imagePath.split(os.path.sep)[-2]
    labels.append(label)

# Encode the tag from a string to an integer type
le = LabelEncoder()
labels = le.fit_transform(labels)

Divide training set and test set, 80% of data as training set, the rest 20% as test set
trainX, testX, trainY, testY = train_test_split(data, labels, test_size=0.2)
Copy the code

The above code completes the path information of loading pictures, and then traverses in turn, reads pictures, extracts features, extracts label information, saves features and label information, and then codes labels, and then divides training set and test set.

This is followed by the same training model and prediction code as the previous classifier. The full version code is as follows:

# import tool library
from sklearn.neighbors import KNeighborsClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.neural_network import MLPClassifier
from sklearn.preprocessing import LabelEncoder
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from PIL import Image
from imutils import paths
import numpy as np
import argparse
import os

# Extract image features
def extract_color_stats(image) :
    Divide the image into THREE RGB channels, then calculate the mean and standard deviation of each channel separately, and return: param image: :return: ""
    (R, G, B) = image.split()
    features = [np.mean(R), np.mean(G), np.mean(B), np.std(R), np.std(G), np.std(B)]

    return features


# set parameters
ap = argparse.ArgumentParser()
ap.add_argument("-d"."--dataset".type=str, default="3scenes".help="path to directory containing the '3scenes' dataset")
ap.add_argument("-m"."--model".type=str, default="knn".help="type of python machine learning model to use")
args = vars(ap.parse_args())

# define a dictionary to save models and select which model to load based on the key
models = {
    "knn": KNeighborsClassifier(n_neighbors=1),
    "naive_bayes": GaussianNB(),
    "logit": LogisticRegression(solver="lbfgs", multi_class="auto"),
    "svm": SVC(kernel="rbf", gamma="auto"),
    "decision_tree": DecisionTreeClassifier(),
    "random_forest": RandomForestClassifier(n_estimators=100),
    "mlp": MLPClassifier()
}

Load data and extract features
print("Extract image features...")
imagePaths = paths.list_images(args['dataset'])
data = []
labels = []

Loop over all image data
for imagePath in imagePaths:
    Load the image, then calculate the image's color channel statistics
    image = Image.open(imagePath)
    features = extract_color_stats(image)
    data.append(features)

    # Save the tag information of the image
    label = imagePath.split(os.path.sep)[-2]
    labels.append(label)

# Encode the tag from a string to an integer type
le = LabelEncoder()
labels = le.fit_transform(labels)

Divide training set and test set, 80% of data as training set, the rest 20% as test set
trainX, testX, trainY, testY = train_test_split(data, labels, random_state=3, test_size=0.2)
# print('trainX numbers={}, testX numbers={}'.format(len(trainX), len(testX)))

# Training model
print("[Modeling using the '{}' model".format(args["model"]))
model = models[args["model"]]
model.fit(trainX, trainY)

# Predict and output classification results report
print("Model evaluation")
predictions = model.predict(testX)
print(classification_report(testY, predictions, target_names=le.classes_))
Copy the code

With these two pieces of code in place, we can run the code to compare the performance of different algorithms on the two data sets.

4.3 Modeling comparison of different models

(1) KNN

One of the simplest classification algorithms for K-nearest Neighbors classifier. The algorithm depends on the distance between feature vectors. Simply put, THE KNN algorithm classifies the unknown data points by having the most categories in the k closest samples. Detailed explain about the KNN can learn reading ShowMeAI graphic machine | KNN algorithm and its application.

Here, we first run image_classifier.py and call the default model KNN to see the experimental results of KNN on iris data set, as shown below:

$! Python iris_classifier.py --model KNN... Applying the 'KNN' model to modeling... Evaluating model effectiveness... Precision Recall F1-Score support Setosa 1.00 1.00 1.00 10 versicolor 0.90 0.90 0.90 10 virginica 0.90 0.90 0.90 10 Accuracy 0.93 30 Macro AVG 0.93 0.93 0.93 30 weighted AVG 0.93 0.93 0.93 30Copy the code

The accuracy rate, recall rate, F1 and the number of test sets of each category are given, namely, precision, recall, F1-score and support respectively. According to the last row and first column, it can be seen that KNN achieves 93% accuracy.

Then the experimental results on the three-scene image data set are as follows:

$! Python image_classifier.py --model KNN... Applying the 'KNN' model to modeling... Evaluating model effectiveness... Precision Recall F1-Score Support Coast 0.84 0.68 0.75 105 Forest 0.78 0.77 0.77 78 highway 0.56 0.78 0.65 54 Micro AVG 0.73 0.73 237 Macro AVG 0.72 0.74 0.72 237 weighted AVG 0.75 0.73 0.73 237Copy the code

Here KNN achieved 75% accuracy.

Ps: In fact, different times of running this algorithm will have different results. The main reason is that the code does not set the parameter random_state when dividing the training set and the test set. This will result in different images of the training set and the test set each time.

(2) Naive Bayes

Then the naive bayes algorithm, detailed explain about the naive bayes algorithm can learn reading ShowMeAI graphic machine |, a naive bayes algorithm.

Test the two datasets respectively, and the results are as follows:

$! Py --model naive_bayes... Modeling using the 'naive_Bayes' model... Evaluating model effectiveness... Precision Recall F1-Score Support Setosa 1.00 1.00 1.00 15 versicolor 1.00 0.92 0.96 12 virginica 0.92 1.00 0.96 11 Micro AVG 0.97 0.97 0.97 38 macro AVG 0.97 0.97 0.97 38 weighted AVG 0.98 0.97 0.97 38Copy the code

$! Py --model naive_bayes... Modeling using the 'naive_Bayes' model... Evaluating model effectiveness... Precision Recall F1-Score Support Coast 0.69 0.40 0.50 88 Forest 0.68 0.82 0.74 84 Highway 0.61 0.78 0.68 65 Micro AVG 0.65 0.65 0.65 237 Macro AVG 0.66 0.67 0.64 237 weighted AVG 0.66 0.65 0.64 237Copy the code

Similarly, naive Bayes has 98% accuracy on IRIS, but only 66% accuracy on image datasets.

So, can we show that the KNN algorithm is better than naive Bayes? Of course, it can’t. The above results only show that KNN algorithm is superior to naive Bayes algorithm in three-scene image data set. In fact, each algorithm has its own advantages and disadvantages and applicable scenarios, it can not be generalized to say that an algorithm is superior to another algorithm at any time, which needs to be analyzed on a case-by-case basis.

(3) Logistic regression

Followed by logistic regression algorithm, detailed explanation about logistic regression algorithm can learn reading ShowMeAI graphic machine |, a logistic regression algorithm.

Test the two datasets respectively, and the results are as follows:

$! Python iris_classifier.py -- Model logit... Apply the 'logit' model to modeling... Evaluating model effectiveness... Precision Recall F1-Score Support Setosa 1.00 1.00 1.00 15 versicolor 1.00 0.92 0.96 12 virginica 0.92 1.00 0.96 11 Micro AVG 0.97 0.97 0.97 38 macro AVG 0.97 0.97 0.97 38 weighted AVG 0.98 0.97 0.97 38Copy the code

$! Python image_classifier.py -- Model logit... Apply the 'logit' model to modeling... Evaluating model effectiveness... Precision Recall F1-Score Support Coast 0.67 0.67 0.67 92 Forest 0.79 0.82 0.80 82 Highway 0.61 0.57 0.59 63 Micro AVG 0.70 0.70 0.70 237 Macro AVG 0.69 0.69 0.69 237 weighted AVG 0.69 0.70 0.69 237Copy the code

Similarly, logistic regression was 98 percent accurate on IRIS, but only 69 percent accurate on image datasets

(4) Support vector machine SVM

Then the SVM algorithm, detailed explanation on the SVM algorithm can be reading ShowMeAI diagram, rounding machine learning | support vector machine (SVM) model.

Test the two datasets respectively, and the results are as follows:

$! Python iris_classifier.py --model SVM... Applying the 'SVM' model to modeling... Evaluating model effectiveness... Precision Recall F1-Score Support Setosa 1.00 1.00 1.00 15 versicolor 1.00 0.92 0.96 12 virginica 0.92 1.00 0.96 11 Micro AVG 0.97 0.97 0.97 38 macro AVG 0.97 0.97 0.97 38 weighted AVG 0.98 0.97 0.97 38Copy the code

$! Python image_classifier.py --model SVM... Applying the 'SVM' model to modeling... Evaluating model effectiveness... Precision Recall F1-Score Support Coast 0.84 0.76 0.80 92 Forest 0.86 0.93 0.89 84 Highway 0.78 0.80 0.79 61 Micro AVG 0.83 0.83 0.83 237 Macro AVG 0.83 0.83 0.83 237Copy the code

Similarly, SVM has 98% accuracy on IRIS, but only 83% accuracy on image data sets.

(5) the decision tree

Then the decision tree algorithm, detailed explanation about the decision tree algorithm can learn reading ShowMeAI graphic machine |, a decision tree model.

Test the two datasets respectively, and the results are as follows:

$! Py -- Model decision_tree... Modeling using the 'decision_tree' model... Evaluating model effectiveness... Precision Recall F1-Score support Setosa 1.00 1.00 1.00 15 versicolor 0.92 0.92 0.92 12 virginica 0.91 0.91 0.91 11 Micro AVG 0.95 0.95 0.95 38 macro AVG 0.94 0.94 0.94 38 weighted AVG 0.95 0.95 0.95 38Copy the code

$! Python image_classifier.py -- Model decision_tree Modeling using the 'decision_tree' model... Evaluating model effectiveness... Precision Recall F1-Score Support Coast 0.71 0.74 0.72 85 Forest 0.76 0.80 0.78 83 Highway 0.77 0.68 0.72 69 Micro AVG 0.74 0.74 237 Macro AVG 0.75 0.74 0.74 237 weighted AVG 0.74 0.74 0.74 237Copy the code

Similarly, decision trees were 98 percent accurate on IRIS, but only 74 percent accurate on image datasets.

(6) Random forest

Followed by random forest algorithm, detailed explain about the random forest algorithm can learn reading ShowMeAI graphic machine | random forest, a classification model.

Test the two datasets respectively, and the results are as follows:

$! Python iris_classifier.py --model random_forest Apply the 'random_forest' model to modeling... Evaluating model effectiveness... Precision Recall F1-Score Support Setosa 1.00 1.00 1.00 15 versicolor 1.00 0.83 0.91 12 virginica 0.85 1.00 0.92 11 Micro AVG 0.95 0.95 0.95 38 macro AVG 0.95 0.94 0.94 38 weighted AVG 0.96 0.95 0.95 38Copy the code

$! Python image_classifier.py --model random_forest... Apply the 'random_forest' model to modeling... Evaluating model effectiveness... Precision Recall F1-Score Support Coast 0.80 0.83 0.81 84 Forest 0.92 0.84 0.88 90 Highway 0.77 0.81 0.79 63 Micro AVG 0.83 0.83 237 Macro AVG 0.83 0.83 0.83 237 weighted AVG 0.84 0.83 0.83 237Copy the code

Similarly, random forests were 96 percent accurate at IRIS, but only 84 percent accurate at image datasets.

Note that generally, if the decision tree algorithm is good, the random forest algorithm should also achieve good or even better results, because the random forest is actually a combination of multiple decision trees for classification and prediction through integrated learning method.

(7) Multilayer perceptron

Finally, the multi-layer perceptron algorithm tests the two data sets respectively, and the results are as follows:

$! Python iris_classifier.py --model... Applying the 'MLP' model to modeling... Evaluating model effectiveness... Precision Recall F1-Score Support Setosa 1.00 1.00 1.00 15 versicolor 1.00 0.92 0.96 12 virginica 0.92 1.00 0.96 11 Micro AVG 0.97 0.97 0.97 38 macro AVG 0.97 0.97 0.97 38 weighted AVG 0.98 0.97 0.97 38Copy the code

$! Python image_classifier.py --model MLP... Applying the 'MLP' model to modeling... Evaluating model effectiveness... Precision Recall F1-Score Support Coast 0.72 0.91 0.80 86 Forest 0.92 0.89 0.90 79 Highway 0.79 0.58 0.67 72 Micro AVG 0.80 0.80 0.80 237 Macro AVG 0.81 0.79 0.79 237 weighted AVG 0.81 0.80 0.80 237Copy the code

Similarly, multilayer perceptrons were 98 percent accurate on IRIS, but only 81 percent accurate on image datasets.

(8) Neural network

Finally, the deep learning algorithm is implemented, namely nn_iris.py and basic_cnn.py.

The first is the implementation of nn_iris.py, and again the first is the import library and data handling:

# import tool library
from keras.models import Sequential
from keras.layers.core import Dense
from keras.optimizers import SGD
from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from sklearn.datasets import load_iris

Load Iris data set, then divide training set and test set, 80% data as training set, the rest 20% as test set
print("Loading data...")
dataset = load_iris()
(trainX, testX, trainY, testY) = train_test_split(dataset.data,
                                                  dataset.target, test_size=0.2)

# Encode the tag with a unique heat vector
lb = LabelBinarizer()
trainY = lb.fit_transform(trainY)
testY = lb.transform(testY)
Copy the code

We use Keras to realize the neural network, and then we need to one-hot encode the label, that is, the single-heat vector encoding.

Then it is to build the structure of the network model and the training and prediction code:

Define the network model using Keras
model = Sequential()
model.add(Dense(3, input_shape=(4,), activation="sigmoid"))
model.add(Dense(3, activation="sigmoid"))
model.add(Dense(3, activation="softmax"))

# Use gradient descent training model
print('Training network... ')
opt = SGD(lr=0.1, momentum=0.9, decay=0.1 / 250)
model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=["accuracy"])
H = model.fit(trainX, trainY, validation_data=(testX, testY), epochs=250, batch_size=16)

# prediction
print('Evaluate model effectiveness')
predictions = model.predict(testX, batch_size=16)
print(classification_report(testY.argmax(axis=1), predictions.argmax(axis=1), target_names=dataset.target_names))
Copy the code

The above code builds a neural network with 3 fully connected layers, the first two layers using Sigmoid activation function, and then the last layer is the output layer, so softmax is used to turn the output into probability value. The optimization algorithm selected random gradient descent SGD, the loss function is categorical_crossentropy, the number of iterations is 250, each batch of data batch_size is 16.

The full version code is as follows:

Load the library
from keras.models import Sequential
from keras.layers.core import Dense
from keras.optimizers import SGD
from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from sklearn.datasets import load_iris

Load Iris data set, then divide training set and test set, 80% data as training set, the rest 20% as test set
print("Loading data...")
dataset = load_iris()
(trainX, testX, trainY, testY) = train_test_split(dataset.data,
                                                  dataset.target, test_size=0.2)

# Encode the tag with a unique heat vector
lb = LabelBinarizer()
trainY = lb.fit_transform(trainY)
testY = lb.transform(testY)

Define the network model using Keras
model = Sequential()
model.add(Dense(3, input_shape=(4,), activation="sigmoid"))
model.add(Dense(3, activation="sigmoid"))
model.add(Dense(3, activation="softmax"))

# Use gradient descent training model
print('Training network... ')
opt = SGD(lr=0.1, momentum=0.9, decay=0.1 / 250)
model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=["accuracy"])
H = model.fit(trainX, trainY, validation_data=(testX, testY), epochs=250, batch_size=16)

# prediction
print('Evaluating model effectiveness... ')
predictions = model.predict(testX, batch_size=16)
print(classification_report(testY.argmax(axis=1), predictions.argmax(axis=1), target_names=dataset.target_names))
Copy the code

Run the python nn_iris.py command. The output is as follows:

$ python nn_iris.py Using TensorFlow backend. Loading data... Training network... Train on 112 samples, validate on 38 samples Epoch 1/250 2022-02-08 10:28:19.104933: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not Compiled to use: AVX2 AVX512F FMA 112/112 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 2 0 s ms/step - loss: 1.1454 acc: 0.3214 - val_loss: 1.1867 - val_acc: 0.2368 Epoch 2/250 112/112 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 0 s 48 us/step - loss: 1.0828 - ACC: 0.3929 - val_loss: 1.2132 - val_ACC: 0.5000 Epoch 3/250 112/112 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 0 s 47 us/step - loss: 1.0491 acc: 0.5268 - val_loss: 1.0593-val_ACC: 0.4737... Epoch 248/250 112/112 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] 0 s 46 us/step - loss: 0.1319 acc: 0.9554 - val_loss: 0.0407 - val_acc: 1.0000 Epoch 249/250 112/112 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] 0 s 46 us/step - loss: 0.1024 acc: 0.9643-VAL_loss: 0.1595 - val_ACC: 0.8947 Epoch 250/250 112/112 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 0 s 47 us/step - loss: 0.0795 acc: 0.9821 - val_loss: 0.0335-VAL_ACC: 1.0000 Evaluation model effect... Precision Recall F1-Score Support Setosa 1.00 1.00 1.00 9 versicolor 1.00 1.00 1.00 10 virginica 1.00 1.00 1.00 19 AVg / Total 1.00 1.00 1.00 38Copy the code

This is 100 percent accuracy.

(9) CNN

Finally, we’ll apply the convolutional neural network, and we’ll implement the basic_cnn.py code.

Again, import the necessary library functions first:

# import tool library
from keras.models import Sequential
from keras.layers.convolutional import Conv2D
from keras.layers.convolutional import MaxPooling2D
from keras.layers.core import Activation
from keras.layers.core import Flatten
from keras.layers.core import Dense
from keras.optimizers import Adam
from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from PIL import Image
from imutils import paths
import numpy as np
import argparse
import os

# configure parameters
ap = argparse.ArgumentParser()
ap.add_argument("-d"."--dataset".type=str, default="3scenes".help="path to directory containing the '3scenes' dataset")
args = vars(ap.parse_args())

Copy the code

Keras is also imported to establish CNN network model. Besides, PIL and IMutils are also imported because image data is processed.

Then it is to load the data and divide the training set and test set. For the loaded data, the original image pixel data is directly used here, and the image data only needs to be adjusted in a uniform size, which is uniformly adjusted to 32×32 and normalized to the range of [0,1].

Load data and extract features
print("Extract image features...")
imagePaths = paths.list_images(args['dataset'])
data = []
labels = []

Loop over all image data
for imagePath in imagePaths:
    Load the image, then resize it to 32×32 and normalize it to [0,1]
    image = Image.open(imagePath)
    image = np.array(image.resize((32.32))) / 255.0
    data.append(image)

    # Save the tag information of the image
    label = imagePath.split(os.path.sep)[-2]
    labels.append(label)

# Encode the tag from a string to an integer
lb = LabelBinarizer()
labels = lb.fit_transform(labels)

# Divide training set and test set
(trainX, testX, trainY, testY) = train_test_split(np.array(data), np.array(labels), test_size=0.25)
Copy the code

Then, a four-layer CNN network structure was defined, including three convolutional layers and the last output layer. Adam was used instead of SGD for optimization algorithm. The code looks like this:

Define the structure of CNN network model
model = Sequential()
model.add(Conv2D(8, (3.3), padding="same", input_shape=(32.32.3)))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2.2), strides=(2.2)))
model.add(Conv2D(16, (3.3), padding="same"))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2.2), strides=(2.2)))
model.add(Conv2D(32, (3.3), padding="same"))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2.2), strides=(2.2)))
model.add(Flatten())
model.add(Dense(3))
model.add(Activation("softmax"))

# Training model
print("Training network...")
opt = Adam(lr=1e-3, decay=1e-3 / 50)
model.compile(loss="categorical_crossentropy", optimizer=opt, metrics=["accuracy"])
H = model.fit(trainX, trainY, validation_data=(testX, testY),
              epochs=50, batch_size=32)

# prediction
print("Evaluating model effectiveness...")
predictions = model.predict(testX, batch_size=32)
print(classification_report(testY.argmax(axis=1),
                            predictions.argmax(axis=1), target_names=lb.classes_))
Copy the code

Run the python basic_cnn.py command. The following output is displayed:

$ python basic_cnn.py Using TensorFlow backend. Loading image data... Training network... Train on 711 samples, Validate on 237 samples of Epoch 1/50 711/711 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] 0 s 629 us/step - loss: 1.0647 acc: 0.4726 - val_loss: 0.9920 - val_acc: 0.5359 Epoch 2/50 711/711 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] 0 s 313 us/step - loss: 0.9200-ACC: 0.6188 - val_loss: 0.7778 - val_ACC: 0.6624 Epoch 3/50 711/711 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] 0 s 308 us/step - loss: 0.6775 acc: 0.7229 - val_loss: 0.5310-VAL_ACC: 0.7553... Epoch 48/50 711/711 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] 0 s 307 us/step - loss: 0.0627 acc: 0.9887 - val_loss: 0.2426 - val_acc: 0.9283 Epoch 49/50 711/711 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] 0 s 310 us/step - loss: 0.0608 acc: 0.9873 - VAL_loss: 0.2236-val_ACC: 0.9325 Epoch 50/50 711/711 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] 0 s 307 us/step - loss: 0.0587 acc: 0.9887 - val_loss: 0.2525-VAL_ACC: 0.9114 Evaluation model effect... Precision Recall F1-Score Support Coast 0.85 0.96 0.90 85 Forest 0.99 0.94 0.97 88 Highway 0.91 0.80 0.85 64 AVG/total 0.92 0.91 0.91 237Copy the code

The accuracy of CNN is 92%, which is superior to the results of several previous machine learning algorithms.

5. Summary

In this simple machine learning tutorial article, we call the existing library to apply the corresponding machine learning algorithm, and solve two simple scenarios. Through this simple introductory tutorial, I hope you understand:

(1) No algorithm is perfect and can completely apply to all scenarios. Even the current popular deep learning method has its limitations, so it should be analyzed on a case-by-case basis!

(2) Classic 5-step machine learning operation process:

• Problem abstraction and understanding
• Data preparation and processing (preprocessing, feature extraction, feature engineering, etc.)
• Various machine learning algorithms
• Analysis and comparison of experimental results
• Model selection and tuning

The resources

• AI quick modeling tools | Scikit – Learn to use guidelines
• AI quick modeling tools | Keras use guide
• Diagram of machine learning algorithm | from entry to master series

ShowMeAIRecommended series of tutorials

• Illustrated Python programming: From beginner to Master series of tutorials
• Illustrated Data Analysis: From beginner to master series of tutorials
• The mathematical Basics of AI: From beginner to Master series of tutorials
• Illustrated Big Data Technology: From beginner to master
• Illustrated Machine learning algorithms: Beginner to Master series of tutorials
• Machine learning: Teach you how to play machine learning series

Related articles recommended

• Application practice of Python machine learning algorithm
• SKLearn introduction and simple application cases
• SKLearn most complete application guide
• XGBoost modeling applications in detail
• LightGBM modeling applications in detail
• Python Machine Learning Integrated Project – E-commerce sales estimates
• Python Machine Learning Integrated Project — E-commerce Sales Estimation
• Machine learning feature engineering most complete interpretation
• Application of Featuretools
• AutoML Automatic machine learning modeling