Article/Ali Tao department
F(x) Team
– 
Su Chuan


Some students may be confused when learning machine learning, do not know how to start, read a lot of classical books introduced a variety of algorithms, but still do not know how to use it to solve problems, even if you know, and find the need to prepare the environment, training and deployment of the machine, ah, good trouble.


Today, I’m going to show you an easy way to get started. I’m going to give you samples and code that you can follow to experience machine learning on your Own Mac


In the Demo below, the end result is that given an image, you can predict the category of the image. For example, if we train a model with a cat or a dog, then the model will learn about cats and dogs, and if we train a model with a button or a search box, then the model will learn about this button or a search box.


If you want to see how machine learning can be used to solve a real problem, you can see how deep Learning can be used to identify UI Components. From problem definition, algorithm selection, sample preparation, model training, model evaluation, model service deployment, to model application are introduced.


Environment to prepare

Installation Anaconda

Download address: www.anaconda.com/products/in…


After the installation is successful, run the following command on the CLI to make the environment variables take effect immediately:

$ source ~/.bashrcCopy the code


You can run the following command to view environment variables


$ cat ~/.bashrcCopy the code

You can see that anaconda’s environment variable has been automatically added to the.bashrc file


Run the following command:

$ conda listCopy the code

As you can see, there are many packages already installed in Anaconda. If you use these packages, you don’t need to install them. The Python environment is already installed.


Note: If the installation fails, reinstall, when prompted where to install, select “change installation location”, the installation location is not selected by default, where to install your own choice, can be placed under “Applications”.



Installation-related Dependencies

Keras, TensorFlow, and Opencv-Python are not available in Anaconda and need to be installed separately.

$ pip install keras
$ pip install tensorflow
$ pip install opencv-pythonCopy the code


Sample preparation

There are only four categories: Button, Keyboard, Searchbar and Switch, and each category has about 200 samples.




Model training

Development training logic

Create a new project, train-project, with the following file structure:


. ├ ─ ─ CNN_net. Py ├ ─ ─ the dataset ├ ─ ─ nn_train. Py └ ─ ─ utils_paths. PyCopy the code


The entry file code is as follows. The logic here is to input the prepared sample into the image classification algorithm SimpleVGGNet and set some training parameters, such as learning rate, Epoch, Batch Size, and then execute this training logic to get a model file.

# nn_train.py
from CNN_net import SimpleVGGNet
from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

from keras.optimizers import SGD
from keras.preprocessing.image import ImageDataGenerator
import utils_paths
import matplotlib.pyplot as plt
from cv2 import cv2
import numpy as np
import argparse
import random
import pickle

import os

# Read data and tags
print("------ start reading data ------")
data = []
labels = []

# Get the image data path for subsequent reading
imagePaths = sorted(list(utils_paths.list_images('./dataset')))
random.seed(42)
random.shuffle(imagePaths)

image_size = 256
# Iterates to read data
for imagePath in imagePaths:
    # Read image data
    image = cv2.imread(imagePath)
    image = cv2.resize(image, (image_size, image_size))
    data.append(image)
    # Read tags
    label = imagePath.split(os.path.sep)[-2]
    labels.append(label)

data = np.array(data, dtype="float"255.0 labels = Np. array(labels)# Data set segmentation
(trainX, testX, trainY, testY) = train_test_split(data,labels, test_size=0.25, random_state=42)

Convert the tag to one-hot encoding format
lb = LabelBinarizer()
trainY = lb.fit_transform(trainY)
testY = lb.transform(testY)

# Data enhancement processingAug = ImageDataGenerator(rotation_range=30, width_shift_range=0.1, height_shift_range=0.1, shear_range=0.2, Zoom_range = 0.2, horizontal_flip = True, fill_mode ="nearest")

# Build convolutional neural networks
model = SimpleVGGNet.build(width=256, height=256, depth=3,classes=len(lb.classes_))

# Set initialization hyperparameters

Vector #
INIT_LR = 0.01
# Epoch  
# Set 5 here in order to complete the training as soon as possible, you can set it higher, such as 30
EPOCHS = 5   
# Batch Size
BS = 32

# Loss function, compile model
print("------ Start training the network ------")
opt = SGD(lr=INIT_LR, decay=INIT_LR / EPOCHS)
model.compile(loss="categorical_crossentropy", optimizer=opt,metrics=["accuracy"])

# Training network model
H = model.fit_generator(
    aug.flow(trainX, trainY, batch_size=BS),
    validation_data=(testX, testY), 
    steps_per_epoch=len(trainX) // BS,
    epochs=EPOCHS
)


# test
print("------ Test network ------")
predictions = model.predict(testX, batch_size=32)
print(classification_report(testY.argmax(axis=1), predictions.argmax(axis=1), target_names=lb.classes_))

# Draw the resulting curve
N = np.arange(0, EPOCHS)
plt.style.use("ggplot")
plt.figure()
plt.plot(N, H.history["loss"], label="train_loss")
plt.plot(N, H.history["val_loss"], label="val_loss")
plt.plot(N, H.history["accuracy"], label="train_acc")
plt.plot(N, H.history["val_accuracy"], label="val_acc")
plt.title("Training Loss and Accuracy")
plt.xlabel("Epoch #")
plt.ylabel("Loss/Accuracy")
plt.legend()
plt.savefig('./output/cnn_plot.png')

# Save model
print("------ Save model ------")
model.save('./cnn.model.h5')
f = open('./cnn_lb.pickle'."wb")
f.write(pickle.dumps(lb))
f.close()Copy the code


In actual application scenarios, if the data set is large, the epoch is also large, and the training is performed on high-performance machines. Now to complete the training task on the native Mac, we only give a few samples to train the model, and the epoch is also very small (5), of course, the recognition accuracy of the model will be very poor, but the purpose of this article is to complete a machine learning task on the native Mac.


Start training

Run the following command to start training:

$ python nn_train.pyCopy the code


The training logs are as follows:

After the training, two files are generated in the current directory: the model file cnn.model.h5 and the loss function curve output/cnn_plot. PNG



Model to evaluate

Now that we have the model file CNn.model.h5, we can write a prediction script and execute the script locally to predict the classification of an image.

$ python predict.pyCopy the code

# predict.py
import allspark
import io
import numpy as np
import json
from PIL import Image
import requests
import threading
import cv2
import os
import tensorflow as tf
from tensorflow.keras.models import load_model
import time

model = load_model('./train/cnn.model.h5')
The input for # pred should be an array of images that have been converted to a Numpy array
# pred = model.predict(['./validation/button/button-demoplus-20200216-16615.png'])

The model output is an array, and the maximum index is the predicted value
Label = [
    "button"."keyboard"."searchbar"."switch"
    ]
testPath = "./test/button.png"

images = []
image = cv2.imread(testPath)
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

image = cv2.resize(image,(256,256))
images.append(image)
images = np.asarray(images)

pred = model.predict(images)

print(pred)

max_ = np.argmax(pred)
print('The forecast result is:',Label[max_])
Copy the code

If you want to know the accuracy of this model, you can also input a batch of data with known classification into the model, and then compare the classification predicted by the model with the real classification to calculate the accuracy and recall rate.



Model service deployment

Developing model services

But in practice, we predict the category of an image by requesting an API for a given image. We need to write a model service, deploy it to the remote end, and get a deployed model service API.


Now we can write a model service and deploy it locally.


# Model service app.py
import allspark
import io
import numpy as np
import json
from PIL import Image
import requests
import threading
import cv2
import tensorflow as tf
from tensorflow.keras.models import load_model


with open('label.json') as f:
    mp = json.load(f)
labels = {value:key for key,value in mp.items()}

def create_opencv_image_from_stringio(img_stream, cv2_img_flag=-1):
  img_stream.seek(0)
  img_array = np.asarray(bytearray(img_stream.read()), dtype=np.uint8)
  image_temp = cv2.imdecode(img_array, cv2_img_flag)
  if image_temp.shape[2] == 4:
    image_channel3 = cv2.cvtColor(image_temp, cv2.COLOR_BGRA2BGR)
    image_mask = image_temp[:,:,3] #.reshape(image_temp.shape[0],image_temp.shape[1], 1)
    image_mask = np.stack((image_mask, image_mask, image_mask), axis = 2)
    index_mask = np.where(image_mask == 0)
    image_channel3[index_mask[0], index_mask[1], index_mask[2]] = 255
    return image_channel3
  else:
    return image_temp

def get_string_io(origin_path):
  r = requests.get(origin_path, timeout=2)
  stringIo_content = io.BytesIO(r.content)
  return stringIo_content

def handleReturn(pred, percent, msg_length):
  result = {
    "content":[]
  }
  argm = np.argsort(-pred, axis = 1)
  for i in range(msg_length):
      label = labels[argm[i, 0]]
      index = argm[i, 0]
      if(pred[i, index] > percent):
        confident = True
      else:
        confident = False
      result['content'].append({'isConfident': confident, 'label': label})
  return result


def process(msg, model):
  msg_dict = json.loads(msg)
  percent = msg_dict['threshold']
  msg_dict = msg_dict['images']
  msg_length = len(msg_dict)
  desire_size = 256
  
  images = []
  for i in range(msg_length):
    image_temp = create_opencv_image_from_stringio(get_string_io(msg_dict[i]))
    image_temp = cv2.cvtColor(image_temp, cv2.COLOR_BGR2RGB)
    image = cv2.resize(image_temp, (256, 256))  
    images.append(image)
  images = np.asarray(images)
  pred = model.predict(images)
  return bytes(json.dumps(handleReturn(pred, percent, msg_length)) ,'utf-8')  


def worker(srv, thread_id, model):
  while True:
    msg = srv.read()
    try:
      rsp = process(msg, model)
      srv.write(rsp)
    except Exception as e:
      srv.error(500,bytes('invalid data format'.'utf-8'))

if __name__ == '__main__':
    desire_size = 256
    model = load_model('./cnn.model.h5')
    
    context = allspark.Context(4)
    queued = context.queued_service()

    workers = []
    for i in range(10):
        t = threading.Thread(target=worker, args=(queued, i, model))
        t.setDaemon(True)
        t.start()
        workers.append(t)
    for t in workers:
        t.join()


Copy the code


Deployment model service

When the model service is written and deployed locally, you need to install the environment. First create a model service project: deploy-project, copy cnn.model.h5 into this project, and install the environment under this project.

.├ ── app.py ├── nn.model.h5 ├─ labelCopy the code


Installation environment

3. Python language -3.2 Building a Development environment -3.2.3 Using pre-built Development Images (recommended)


Install the Docker

You can view the Mac Docker installation documentation directly

Installation with Homebrew requires current Homebrew: https://brew.sh
$ brew cask install dockerCopy the code

After the installation, the Docker icon will appear on the desktop.


Create a virtual environment for Anaconda


To create a Python environment using conda, specify a directory with a fixed name: ENV$conda create -p ENV python=3.7Install the EAS Python SDK$ENV/bin/PIP install http://eas-data.oss-cn-shanghai.aliyuncs.com/sdk/allspark-0.9-py2.py3-none-any.whlInstall other dependencies
$ ENV/bin/pip install tensorflow keras opencv-python

Activate the virtual environment
$ conda activate ./ENV

Exit the virtual environment (when not in use)
$ conda deactivateCopy the code


Run the Docker environment

/Users/chang/Desktop/ml-test/deploy-project

sudo docker run -ti -v  /Users/chang/Desktop/ml-test/deploy-project:/home -p 8080:8080  
registry.cn-shanghai.aliyuncs.com/eas/eas-python-base-image:py3.6-allspark-0.8Copy the code


Local deployment

Now you are ready to deploy locally, execute the following command:

cd /home
./ENV/bin/python app.pyCopy the code

The following log shows that the deployment was successful.

Once deployed, the model service can be accessed using localhost:8080/predict.


We use the curl command to send a post request predicting the image category:


curl -X POST 'localhost:8080/predict' \
-H 'Content-Type: application/json' \
-d '{" images ": /" https://img.alicdn.com/tfs/TB1W8K2MeH2gK0jSZJnXXaT1FXa-638-430.png ", "threshold" : 0.5}'Copy the code


The predicted results are as follows:

{"content": [{"isConfident": true."label": "keyboard"}}]Copy the code


The complete code

Clone code repository directly: github.com/imgcook/ml-…


After installing the environment, run it directly with the following commands.

# 1. Training model
$ cd train-project
$ python nn_train.py

# Generate model file: cnn.model.h5

# 2. Copy the model file to deploy-Project and deploy the model service
Install the model service runtime first
$ conda activate ./ENV
$ sudo docker run -ti -v  /Users/chang/Desktop/ml-test/deploy-project:/home -p 8080:8080  registry.cn-shanghai.aliyuncs.com/eas/eas-python-base-image:py3.6-allspark-0.8
$ cd /home
$ ./ENV/bin/python app.py

Localhost :8080/predict

# 3. Access the model service
curl -X POST 'localhost:8080/predict' \
-H 'Content-Type: application/json' \
-d '{" images ": /" https://img.alicdn.com/tfs/TB1W8K2MeH2gK0jSZJnXXaT1FXa-638-430.png ", "threshold" : 0.5}'Copy the code


The last

So, to summarize the process of using deep learning here. We chose SimpleVGGNet as the image classification algorithm (equivalent to a function), passed the prepared data to the function, ran the function (learning the characteristics and labels of the data set) and got an output, the model file Model.h5.


The model file can take an image as input, predict what that image is, and output the prediction. But if we want the model to run online, we need to write a model service (API) and deploy it online to get an HTTP API that we can call directly in production.