Beginner’s machine learning bootcamp!

The original link: www.pyimagesearch.com/2019/01/14/…

By Adrian Rosebrock

This is a hands-on introduction to implementing machine learning algorithms in Python and running models on numeric and image data sets. By the time you’re done with this tutorial, you should be ready to start your machine learning journey!

This tutorial uses the following two libraries to implement machine learning algorithms:

• scikit-learn
• Keras

In addition, you will learn:

• Evaluate your problems
• Data preparation (raw data, feature extraction, feature engineering, etc.)
• Examine various machine learning algorithms
• Check experimental results
• Learn more about the best performing algorithms

The machine learning algorithms used in this paper include:

1. KNN
2. Naive Bayes
3. Logistic regression
4. SVM
5. The decision tree
6. Random forests
7. perceptron
8. Multilayer forward network
9. CNNs

Install the necessary Python machine learning libraries

Before starting this tutorial, make sure you have the following Python libraries installed:

• Numpy: For numerical processing in Python
• PIL: a simple image processing library
• scikit-learn: contains a variety of machine learning algorithms (note that the version 0.2+ is required, so the installation command below should be added--upgrade )
• Kears and TensorFlow: for deep learning. This tutorial can only use the CPU version of TensorFlow
• OpenCV: This tutorial does not use OpenCV, but the Imutils library relies on it;
• Imutils: The author’s image processing/computer vision library

The installation commands are as follows. Virtual environments (for example, anaconda is used to create a new environment) are recommended:

$ pip install numpy
$ pip install pillow
$ pip install --upgrade scikit-learn
$ pip install tensorflow # or tensorflow-gpu
$ pip install keras
$ pip install opencv-contrib-python
$ pip install --upgrade imutils
Copy the code

The data set

This tutorial uses two data sets to help you better understand the performance of each machine learning algorithm.

The first data set is the Iris data set. This data set is the equivalent of typing “Hello, World!” when you first learn a programming language.

This data set is a numeric data, as shown in the figure below, which is actually a table data, with each row representing a sample and each column representing a different attribute. This data set mainly collects the data of three different irises, which are:

• Iris Setosa
• Iris Versicolor
• Iris Virginica

The last column in the diagram is Class label, and then there are four attributes, which are:

• Sepal length– The length of the Sepal
• Sepal width– The width of the Sepal
• Petal Length — The length of a Petal
• Petal Width — Petal width

This data set is probably one of the simplest machine learning data sets, and is usually used to teach programmers and engineers the basics of machine learning and pattern recognition.

For this data set, our goal is to train a machine learning model to correctly classify the categories of each sample based on the four attributes given.

It should be noted that one of the categories and the other two categories are linearly separable, but the two categories are not linearly separable, so we need to adopt a nonlinear model to classify them. Of course, in real life, machine learning algorithms using nonlinear models are very common.

The second dataset is a three-scene image dataset. This is to help beginners learn how to process image data and which algorithm performs best on both sets of data.

The following is a partial sample of images from this three-scene dataset, which includes three scenes of forest, highway and coastline. In total, there are 948 images. The specific number of images for each category is as follows:

• Coast: 360
• Forest: 328
• Highway: 260

Modeling the Shape of the Scene This three-scene dataset is sampled from an eight-scene dataset by Oliva and Torralba in a 2001 paper Modeling the Shape of the scene: a holistic representation of the spatial envelope

Steps to machine learning in Python

Whenever implementing a machine learning algorithm, it is recommended to start with the following process:

• Evaluate your problems
• Data preparation (raw data, feature extraction, feature engineering, etc.)
• Examine various machine learning algorithms
• Check experimental results
• Learn more about the best performing algorithms

This process will improve and optimize as you gain experience with machine learning, but for starters, this is the process I recommend when getting started.

So, get started! The first step is to evaluate our questions and ask ourselves:

• What type of data set is it? Numerical, categorical or graphic?
• What is the ultimate goal of the model?
• How to define and measure “accuracy”?
• What algorithms, based on your current machine learning knowledge, work well for this kind of problem?

This last question is important because the more you use Python for machine learning, the more experience you will gain. From previous experience, you probably know that there is one algorithm that works just fine.

So, the next step is to prepare the data, which is data preprocessing and feature engineering.

Generally, this step involves loading the data from the hard drive, examining the data, and then deciding whether to do feature extraction or feature engineering.

Feature extraction is the process of applying some algorithm to quantify the data in some way. For example, for image data, we can use the method of calculating histogram to calculate the distribution of pixel intensity in the image. In this way, we can get the characteristics that describe the image color.

Feature engineering is the process of transforming raw input data into a feature representation that better describes the underlying problem. Of course, feature engineering is a more advanced technology, and it is recommended to use this method to process data after some experience with machine learning.

The third step is to examine the various machine learning algorithms, that is, to implement a series of machine learning algorithms and apply them to the data set.

Here are several different types of machine learning algorithms you should include in your toolbox:

1. Linear models (e.g., logistic regression, linear SVM)
2. Nonlinear models (such as RBF SVM, gradient descent classifier)
3. Trees and integration-based models (such as decision trees and random forests)
4. Neural networks (e.g., multilayer perceptrons, convolutional neural networks)

A robust (stable) set of machine learning models should be selected to evaluate the problem, since our goal is to determine which algorithms perform well and which poorly for the current problem.

After deciding which models to use, the next step is to train the models and test them on the data set, observing the performance results of each model on the data set.

After many experiments, you may just have a “sixth sense” of which algorithm works best on which data set.

For example, you’ll find:

• For data sets with many features, random forest algorithm works very well;
• The logistic regression algorithm can handle the sparse data of high dimension well.
• For image data, CNNs works very well.

The above experience, of course, requires you to do more hands-on, more practical understanding of different machine learning algorithms!

Let’s do it!

The next step is to start typing code to implement the machine learning algorithm and test it on both data sets. The code file directory for this tutorial is as follows, consisting of four code files and a folder called 3Scenes, which is the three-scene dataset, and the Iris dataset is loaded directly from the Scikit-Learn library.

├── 3scenes │ ├── coast [360-entries] │ ├── Forest [360-entries] │ ├── Highway [360-entries] ├── Classify_ilis.py ├── Classify_images.py ├── nn_iris.py ├─ basic_nn.pyCopy the code

Code and data set files can be in the public account background, that is, the public account session interface reply “PY_ML” to obtain!

The first is the implementation of Classify_iris. py, which uses machine learning algorithms to classify Iris data sets.

First import the required libraries:

from sklearn.neighbors import KNeighborsClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.neural_network import MLPClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from sklearn.datasets import load_iris
import argparse

# Set parameters
ap = argparse.ArgumentParser()
ap.add_argument("-m"."--model", type=str, default="knn", help="type of python machine learning model to use")
args = vars(ap.parse_args())

# Define a dictionary for storing models. Select which models to load based on key
models = {
    "knn": KNeighborsClassifier(n_neighbors=1),
    "naive_bayes": GaussianNB(),
    "logit": LogisticRegression(solver="lbfgs", multi_class="auto"),
    "svm": SVC(kernel="rbf", gamma="auto"),
    "decision_tree": DecisionTreeClassifier(),
    "random_forest": RandomForestClassifier(n_estimators=100),
    "mlp": MLPClassifier()
}
Copy the code

You can see in the SkLearn library that the code for several machine learning algorithms we will implement is integrated, including:

• KNN
• Naive Bayes
• Logistic regression
• SVM
• The decision tree
• Random forests
• perceptron

For example, if you want to create a neighborsclassifier for sklearn and neighbors, run the command KNeighborsClassifier() in sklearn. If you want to create a neighborsclassifier for neighbors, run the command n_neighbors. The number of nearest neighbors.

We use a models dictionary to store the initialization of different models, and then call the corresponding model based on the parameter –model. For example, run python classify_irs.py –model KNN to call the KNN algorithm model.

Next comes the loading data section:

print("[INFO] loading data...")
dataset = load_iris()
(trainX, testX, trainY, testY) = train_test_split(dataset.data,
	dataset.target, random_state=3, test_size=0.25)
Copy the code

Load_iris () in sklear.datasets is called directly to load the data, and then train_test_split is used to divide the training set and the dataset. Here 75% data is used as the training set and 25% as the test set.

The last part is the training model and prediction:

# Training model
print("[INFO] using '{}' model".format(args["model"]))
model = models[args["model"]]
model.fit(trainX, trainY)

Predict and output a classification result report
print("[INFO] evaluating")
predictions = model.predict(testX)
print(classification_report(testY, predictions, target_names=dataset.target_names))
Copy the code

The full code code is as follows:

from sklearn.neighbors import KNeighborsClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.neural_network import MLPClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from sklearn.datasets import load_iris
import argparse

# Set parameters
ap = argparse.ArgumentParser()
ap.add_argument("-m"."--model".type=str, default="knn".help="type of python machine learning model to use")
args = vars(ap.parse_args())

# Define a dictionary for storing models. Select which models to load based on key
models = {
    "knn": KNeighborsClassifier(n_neighbors=1),
    "naive_bayes": GaussianNB(),
    "logit": LogisticRegression(solver="lbfgs", multi_class="auto"),
    "svm": SVC(kernel="rbf", gamma="auto"),
    "decision_tree": DecisionTreeClassifier(),
    "random_forest": RandomForestClassifier(n_estimators=100),
    "mlp": MLPClassifier()
}

# Load the Iris data set and divide the training and test sets. 75% of the data is used as the training set and the remaining 25% is used as the test set
print("[INFO] loading data...")
dataset = load_iris()
(trainX, testX, trainY, testY) = train_test_split(dataset. Data, dataset. Target, random_state=3, test_size=0.25)# Training model
print("[INFO] using '{}' model".format(args["model"]))
model = models[args["model"]]
model.fit(trainX, trainY)

Predict and output a classification result report
print("[INFO] evaluating")
predictions = model.predict(testX)
print(classification_report(testY, predictions, target_names=dataset.target_names))
Copy the code

Next, the classification prediction code classiFY_images.py of the three-scene image data set is used, which is actually quite similar to the code of ClassiFY_iris. py. The first part is imported into the library, and the following lines of code are added:

from sklearn.preprocessing import LabelEncoder
from PIL import Image
from imutils import paths
import numpy as np
import os
Copy the code

LabelEncoder is used to encode the label from string to integer, and the rest of the items deal with image correlation.

For image data, if the input model directly USES the original pixel information, most of the effect of machine learning algorithms is not ideal, so the method of feature extraction, mainly image mean and standard deviation of the color channel information statistics, a total of RGB three channels, each channel to calculate mean and standard deviation, then unifies in together, A six-dimensional feature is obtained, and the function looks like this:

def extract_color_stats(image):
    Param image: :return: ""
    (R, G, B) = image.split()
    features = [np.mean(R), np.mean(G), np.mean(B), np.std(R), np.std(G), np.std(B)]

    return features
Copy the code

It will also define a models dictionary. The code is the same, so I won’t post it here, and then the code in the image loading section looks like this:

Load data and extract features
print("[INFO] extracting image features...")
imagePaths = paths.list_images(args['dataset'])
data = []
labels = []

Loop through all image data
for imagePath in imagePaths:
    # Load the image, then calculate the color channel statistics of the image
    image = Image.open(imagePath)
    features = extract_color_stats(image)
    data.append(features)

    # Save the tag information of the picture
    label = imagePath.split(os.path.sep)[2 -]
    labels.append(label)

# Encodes the tag from a string to an integer
le = LabelEncoder()
labels = le.fit_transform(labels)

# Divide the training set and test set, 75% of the data is used as the training set and the remaining 25% is used as the test set
(trainX, testX, trainY, testY) = train_test_split(data, labels, test_size=0.25)
Copy the code

The above code loads the path information of the image from the hard disk, then iterates, reads the image, extracts the features, extracts the tag information, saves the features and tags, encodes the tags, and then divides the training set and test set.

Then the same training model and prediction code, again unchanged, will not be listed here.

The full version is as follows:

from sklearn.neighbors import KNeighborsClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.neural_network import MLPClassifier
from sklearn.preprocessing import LabelEncoder
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from PIL import Image
from imutils import paths
import numpy as np
import argparse
import os


def extract_color_stats(image):
    ' 'Param image: :return: 'param image: :return:'' '
    (R, G, B) = image.split()
    features = [np.mean(R), np.mean(G), np.mean(B), np.std(R), np.std(G), np.std(B)]

    return features


# Set parameters
ap = argparse.ArgumentParser()
ap.add_argument("-d"."--dataset".type=str, default="3scenes".help="path to directory containing the '3scenes' dataset")
ap.add_argument("-m"."--model".type=str, default="knn".help="type of python machine learning model to use")
args = vars(ap.parse_args())

# Define a dictionary for storing models. Select which models to load based on key
models = {
    "knn": KNeighborsClassifier(n_neighbors=1),
    "naive_bayes": GaussianNB(),
    "logit": LogisticRegression(solver="lbfgs", multi_class="auto"),
    "svm": SVC(kernel="rbf", gamma="auto"),
    "decision_tree": DecisionTreeClassifier(),
    "random_forest": RandomForestClassifier(n_estimators=100),
    "mlp": MLPClassifier()
}
Load data and extract features
print("[INFO] extracting image features...")
imagePaths = paths.list_images(args['dataset'])
data = []
labels = []

Loop through all image data
for imagePath in imagePaths:
    # Load the image, then calculate the color channel statistics of the image
    image = Image.open(imagePath)
    features = extract_color_stats(image)
    data.append(features)

    # Save the tag information of the picture
    label = imagePath.split(os.path.sep)[-2]
    labels.append(label)

# Encodes the tag from a string to an integer
le = LabelEncoder()
labels = le.fit_transform(labels)

# Divide the training set and test set, 75% of the data is used as the training set and the remaining 25% is used as the test set
(trainX, testX, trainY, testY) = train_test_split(data, labels, random_state=3, test_size=0.25)
# print('trainX numbers={}, testX numbers={}'.format(len(trainX), len(testX)))

# Training model
print("[INFO] using '{}' model".format(args["model"]))
model = models[args["model"]]
model.fit(trainX, trainY)

Predict and output classification results report
print("[INFO] evaluating...")
predictions = model.predict(testX)
print(classification_report(testY, predictions, target_names=le.classes_))
Copy the code

With these two pieces of code completed, we can run the code and compare the performance of the different algorithms on the two data sets.

Due to the space, I will omit the introduction of each algorithm in the original text. For more details, see my previous introduction of machine learning algorithms:

• Comparison of common Machine Learning Algorithms (Part 1)
• Summary and Comparison of common Machine Learning Algorithms
• Comparison of common machine Learning algorithms

KNN

Here we run classify_IRS and call the default model KNN to see the experimental results of KNN on the Iris data set, as shown below:

The accuracy rate, recall rate, F1 and the number of test sets for each category are given, namely, corresponding to precision, recall, F1-score and support, respectively. According to the last row and the first column, it can be seen that KNN achieved 95% accuracy.

The experimental results on the three-scene image data set are followed:

Here KNN achieves 72% accuracy.

(ps: In fact, running the algorithm, different times have different results, given the original author is 75%, the main reason is because when divided into training set and test set, random_state code is not set parameters, resulting in each run of training set and testing set of images is different, so the results will be different!

Naive Bayes

Then the naive Bayesian algorithm is used to test the two data sets respectively. The results are as follows:

Similarly, naive Bayes is 98% accurate on Iris, but only 63% accurate on image data sets.

So, can we say that KNN is better than naive Bayes?

Of course, no. The above results can only show that KNN algorithm is better than naive Bayesian algorithm in the three-scene image data set.

In fact, each algorithm has its own advantages and disadvantages and applicable scenarios. It can not be said that one algorithm is better than another algorithm at any time, which needs to be analyzed on a case-by-case basis.

Logistic regression

The logistic regression algorithm is followed to test the two data sets respectively, and the results are as follows:

Similarly, logistic regression was 98% accurate on Iris, but only 77% accurate on the image data set (compared to 69% accuracy for the original author).

Support vector machine SVM

Then the SVM algorithm is used to test the two data sets, and the results are as follows:

Similarly, SVM has 98% accuracy in Iris, but only 76% accuracy in image data set (compared with 83% accuracy of the author of the original text, mainly because there is a big difference in the category coast).

The decision tree

Next, the decision tree algorithm is used to test the two data sets. The results are as follows:

Similarly, the decision tree has 98% accuracy on Iris, but only 71% accuracy on the image data set (compared to 74% accuracy of the original author’s decision tree).

Random forests

Then the random forest algorithm was used to test the two data sets, and the results were as follows:

Similarly, random forest has 96% accuracy on Iris, but only 77% accuracy on image data set (compared with 84% accuracy for the original author’s decision tree).

Note that in general, if the decision tree algorithm does well, the random forest algorithm should do well or even better, because the random forest is actually multiple decision trees grouped together to make classification predictions using an ensemble learning method.

Multilayer perceptron

Finally, the multi-layer perceptron algorithm is used to test two data sets, and the results are as follows:

Similarly, the multilayer perceptron has 98% accuracy on Iris, but only 79% accuracy on the image data set (compared to 81% accuracy for the original author’s decision tree).

Deep learning and deep neural networks

The neural network

Finally, the algorithm to achieve deep learning, namely nn_iris.py and basic_cnn.py, two pieces of code.

(Note the version problem of TensorFlow and Keras, I used TF=1.2 and Keras=2.1.5)

The first is the implementation of nn_iris.py, and the first is the import of the library and the processing of the data:

from keras.models import Sequential
from keras.layers.core import Dense
from keras.optimizers import SGD
from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from sklearn.datasets import load_iris

# Load the Iris data set and divide the training and test sets. 75% of the data is used as the training set and the remaining 25% is used as the test set
print("[INFO] loading data...")
dataset = load_iris()
(trainX, testX, trainY, testY) = train_test_split(dataset.data,
                                                  dataset.target, test_size=0.25)

# Single-hot code the tag
lb = LabelBinarizer()
trainY = lb.fit_transform(trainY)
testY = lb.transform(testY)
Copy the code

Here, we will use Keras to realize the neural network, and then we need to carry out one-hot coding for the label, that is, single-hot coding.

Then there is the structure of the network model and the training and prediction code:

# Define the network model using Keras
model = Sequential()
model.add(Dense(3, input_shape=(4,), activation="sigmoid"))
model.add(Dense(3, activation="sigmoid"))
model.add(Dense(3, activation="softmax"))

Use gradient descent training model
print('[INFO] training network... ')
opt = SGD(lr=0.1, momentum=0.9, decay=0.1 / 250)
model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=["accuracy"])
H = model.fit(trainX, trainY, validation_data=(testX, testY), epochs=250, batch_size=16)

# prediction
print('[INFO] evaluating network... ')
predictions = model.predict(testX, batch_size=16)
print(classification_report(testY.argmax(axis=1), predictions.argmax(axis=1), target_names=dataset.target_names))
Copy the code

Here is the neural network with 3 layers of full connection layer defined, the first two layers adopt Sigmoid activation function, and the last layer is the output layer, so softMax is used to change the output into probability value. Then, an optimization algorithm using SGD is defined. The loss function is categorical_crossentropy, the number of iterations is 250, and the batch_size is 16.

The full version is as follows:

from keras.models import Sequential
from keras.layers.core import Dense
from keras.optimizers import SGD
from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from sklearn.datasets import load_iris

# Load the Iris data set and divide the training and test sets. 75% of the data is used as the training set and the remaining 25% is used as the test set
print("[INFO] loading data...")
dataset = load_iris()
(trainX, testX, trainY, testY) = train_test_split(dataset. Data, dataset. Target, test_size=0.25)# Single-hot code the tag
lb = LabelBinarizer()
trainY = lb.fit_transform(trainY)
testY = lb.transform(testY)

# Define the network model using Keras
model = Sequential()
model.add(Dense(3, input_shape=(4,), activation="sigmoid"))
model.add(Dense(3, activation="sigmoid"))
model.add(Dense(3, activation="softmax"))

Use gradient descent training model
print('[INFO] training network... ') opt = SGD(lr=0.1, momentum=0.9, decay= 0.1/250) model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=["accuracy"])
H = model.fit(trainX, trainY, validation_data=(testX, testY), epochs=250, batch_size=16)

# prediction
print('[INFO] evaluating network... ')
predictions = model.predict(testX, batch_size=16)
print(classification_report(testY.argmax(axis=1),
                            predictions.argmax(axis=1), target_names=dataset.target_names))
Copy the code

Run the python nn_iris.py command, and the output is as follows:

This is 100% accurate, same as the original. Of course, in fact, the results given in the original text are shown in the figure below, and we can see that the number of categories is different.

CNN

The last is to implement basic_cnn.py this code.

Again, import the required library functions first:

from keras.models import Sequential
from keras.layers.convolutional import Conv2D
from keras.layers.convolutional import MaxPooling2D
from keras.layers.core import Activation
from keras.layers.core import Flatten
from keras.layers.core import Dense
from keras.optimizers import Adam
from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from PIL import Image
from imutils import paths
import numpy as np
import argparse
import os

# Set parameters
ap = argparse.ArgumentParser()
ap.add_argument("-d"."--dataset", type=str, default="3scenes",
                help="path to directory containing the '3scenes' dataset")
args = vars(ap.parse_args())
Copy the code

Keras should also be imported to build CNN’s network model. In addition, since it is to process image data, PIL and imutils should also be imported.

Then load the data and divide the training set and test set. For the loading data, the original image pixel data is directly used here, and the image data only needs to be uniformly adjusted to 32×32 and normalized to the range of [0,1].

Load data and extract features
print("[INFO] extracting image features...")
imagePaths = paths.list_images(args['dataset'])
data = []
labels = []

Loop through all image data
for imagePath in imagePaths:
    # load the image, then resize it to 32×32 and normalize it to [0,1]
    image = Image.open(imagePath)
    image = np.array(image.resize((32.32))) / 255.0
    data.append(image)

    # Save the tag information of the picture
    label = imagePath.split(os.path.sep)[2 -]
    labels.append(label)

# Code the tag from a string to an integer
lb = LabelBinarizer()
labels = lb.fit_transform(labels)

Divide the training set and test set
(trainX, testX, trainY, testY) = train_test_split(np.array(data), np.array(labels), test_size=0.25)
Copy the code

Then, a 4-layer CNN network structure is defined, including 3 layers of convolution layer and the last output layer. The optimization algorithm adopts Adam instead of SGD. The code looks like this:

# Define CNN network model structure
model = Sequential()
model.add(Conv2D(8, (3.3), padding="same", input_shape=(32.32.3)))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2.2), strides=(2.2)))
model.add(Conv2D(16, (3.3), padding="same"))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2.2), strides=(2.2)))
model.add(Conv2D(32, (3.3), padding="same"))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2.2), strides=(2.2)))
model.add(Flatten())
model.add(Dense(3))
model.add(Activation("softmax"))

# Training model
print("[INFO] training network...")
opt = Adam(lr=1e-3, decay=1e-3 / 50)
model.compile(loss="categorical_crossentropy", optimizer=opt, metrics=["accuracy"])
H = model.fit(trainX, trainY, validation_data=(testX, testY),
              epochs=50, batch_size=32)

# prediction
print("[INFO] evaluating network...")
predictions = model.predict(testX, batch_size=32)
print(classification_report(testY.argmax(axis=1),
                            predictions.argmax(axis=1), target_names=lb.classes_))
Copy the code

The full version is as follows:

from keras.models import Sequential from keras.layers.convolutional import Conv2D from keras.layers.convolutional import  MaxPooling2D from keras.layers.core import Activation from keras.layers.core import Flatten from keras.layers.core import Dense from keras.optimizers import Adam from sklearn.preprocessing import LabelBinarizer from sklearn.model_selection import train_test_split from sklearn.metrics import classification_report from PIL import Image from imutils import paths import numpy as np import argparse import os# Set parameters
ap = argparse.ArgumentParser()
ap.add_argument("-d"."--dataset".type=str, default="3scenes".help="path to directory containing the '3scenes' dataset")
args = vars(ap.parse_args())

Load data and extract features
print("[INFO] extracting image features...")
imagePaths = paths.list_images(args['dataset'])
data = []
labels = []

Loop through all image data
for imagePath in imagePaths:
    # load the image, then resize it to 32×32 and normalize it to [0,1]Image = image.open (imagePath) image = np.array(image.resize((32, 32))) / 255.0 data.append(image)# Save the tag information of the picture
    label = imagePath.split(os.path.sep)[-2]
    labels.append(label)

# Code the tag from a string to an integer
lb = LabelBinarizer()
labels = lb.fit_transform(labels)

Divide the training set and test set
(trainX, testX, trainY, testY) = train_test_split(np.array(data), np.array(array), test_size=0.25)# Define CNN network model structure
model = Sequential()
model.add(Conv2D(8, (3, 3), padding="same", input_shape=(32, 32, 3)))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))
model.add(Conv2D(16, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))
model.add(Conv2D(32, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))
model.add(Flatten())
model.add(Dense(3))
model.add(Activation("softmax"))

# Training model
print("[INFO] training network...")
opt = Adam(lr=1e-3, decay=1e-3 / 50)
model.compile(loss="categorical_crossentropy", optimizer=opt, metrics=["accuracy"])
H = model.fit(trainX, trainY, validation_data=(testX, testY),
              epochs=50, batch_size=32)

# prediction
print("[INFO] evaluating network...")
predictions = model.predict(testX, batch_size=32)
print(classification_report(testY.argmax(axis=1),
                            predictions.argmax(axis=1), target_names=lb.classes_))
Copy the code

Run the python basic_cnn.py command. The following output is displayed:

CNN’s accuracy rate is 90%, which is better than the results of several previous machine learning algorithms.

---

summary

In the end, it’s just a right machine learning was a complete beginners tutorial, is actually a simple call existing library to implement the corresponding machine learning algorithms, simple feelings of how to use the machine learning algorithm for beginners, as in learning a programming language, to the code examples to knock up books, and then run the code, see oneself write the running result of the program.

Here’s what you should know from this simple introduction:

1. No algorithm is perfect and can be completely applicable to all scenarios. Even the current popular deep learning method has its limitations, so it should be analyzed on a case-by-case basis!
2. Remember the recommended 5-step machine learning process from the beginning, and review it again:
   • Evaluate your problems
   • Data preparation (raw data, feature extraction, feature engineering, etc.)
   • Examine various machine learning algorithms
   • Check experimental results
   • Learn more about the best performing algorithms
3. The last point is that the results of my algorithm are different from the results of the original author. This is actually the reason why each time the data is sampled, the partition of the training set and the test set is different! This also shows that data is very important, for machine learning, good data is important!

Then, according to the tutorial, you can continue to learn more about each type of machine learning algorithms, understand the basic principles of each algorithm and the implementation, try to achieve manually, rather than simply calls the existing libraries, so more can deepen impression, here recommend the machine learning field, classic machine learning algorithms are introduced, and will take you step by step implementation algorithm!

Finally, I highly recommend you to read the blog of the original author, who is also a great god, and his blog address is as follows:

www.pyimagesearch.com/

His blog includes tutorials and articles on Opencv, Python, machine learning, and deep learning, and he likes to learn in the real world, so many of the articles are about learning a particular knowledge point or algorithm by practicing in the real world. Just as this article introduces the beginner to machine learning by implementing these common machine learning algorithms on two different types of data sets.

---

The code and data sets for this tutorial are obtained as follows:

1. Follow the public account “Machine Learning and Computer Vision”
2. Leave a message “PY_ML” in the background of wechat official account

Welcome to follow my wechat official account – Machine Learning and computer vision, or scan the qr code below, we communicate, learn and progress!

Past excellent recommendation

Machine Learning Series

• Model evaluation, overfitting, underfitting, and hyperparameter tuning methods
• Comparison of common machine Learning algorithms
• Comparison of common Machine Learning Algorithms (Part 1)
• Introduction to Machine Learning (2)– How to Build a Complete Machine Learning Project
• Data Preprocessing in Feature Engineering (Part 1)

Github Project & Resource Tutorial recommended

• [Github project recommended] a better site to read and find papers
• TensorFlow (TensorFlow
• Must read AI and deep learning blog
• [Tutorial] A simple and easy to understand TensorFlow tutorial
• [Resources] Recommend some Python books and tutorials, both beginners and advanced!
• [Github project recommended] Machine learning knowledge table
• Here are three tools to help you make the most of Github