The author | PRUDHVI VARMA compile | source of vitamin k | Analytics Indiamag
In today’s world, AI is already used by most business operations and is very easy to deploy thanks to advanced deep learning frameworks. These deep learning frameworks provide high-level programming interfaces to help us design deep learning models. Using a deep learning framework, it reduces developer work by providing built-in library functions that allow us to build models faster and easier.
In this paper, we will build the same deep learning framework, namely convolutional neural network image classification for the same data set in Keras, PyTorch, and Caffe, and compare the implementations of all these approaches. Finally, we’ll see how the CNN model built by PyTorch outperforms its peers with built-in Keras and Caffe.
Topics covered in this article
-
How to choose a deep learning framework.
-
Advantages and disadvantages of Keras
-
Pros and cons of PyTorch
-
Caffe’s strengths and weaknesses
-
Implement the CNN model in Keras, PyTorch, and Caffe.
Choose a deep learning framework
When choosing deep learning framework, there are some indicators to find the best framework, it should provide the interface of the parallel computing, good operating model, and a large number of built-in package, it should optimize performance, but also want to consider our business problems and flexibility, are these before we choose deep learning framework to consider basic questions. Let’s compare the three most commonly used deep learning frameworks Keras, Pytorch, and Caffe.
Keras
Keras, an open source framework developed by Google engineer Francois Chollet, is a deep learning framework that makes it easy to use and evaluate our models with just a few lines of code.
If you’re not familiar with deep learning, Keras is the best framework for beginners to get started. Keras is very beginner friendly and easy to work with Python, and it has many pre-training models (VGG, Inception, etc.). Not only is it easy to learn, but it supports Tensorflow as a back end.
Limitations of using Keras
-
Keras needs to improve some features
-
We need to sacrifice speed for user-friendliness
-
Sometimes even using a GPU takes a long time.
A practical implementation using the Keras framework
In the following code snippet, we will import the required libraries.
import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras import backend as K
Copy the codeSuper parameters:
batch_size = 128
num_classes = 10
epochs = 12
img_rows, img_cols = 28.28
(x_train, y_train), (x_test, y_test) = mnist.load_data()
Copy the codeIn the code snippet below, we will build a deep learning model with several layers and assign optimizers, activation functions, and loss functions.
model = Sequential()
model.add(Conv2D(32, kernel_size=(3.3),
activation='relu',
input_shape=input_shape))
model.add(Conv2D(64, (3.3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2.2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss=keras.losses.categorical_crossentropy,
optimizer=keras.optimizers.Adam(),
metrics=['accuracy'])
Copy the codeIn the following code snippet, we will train and evaluate the model.
model.fit(x_train, y_train,
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_data=(x_test, y_test))
score = model.evaluate(x_test, y_test, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])
Copy the code
PyTorch
PyTorch is an open source framework developed by the Facebook research team. It is an implementation of the deep learning model. It provides all the services and features provided by the Python environment, it allows automatic differentiation, and helps speed up the process of back propagation. Torchaudio, Torchtext, can work flexibly in NLP, computer vision. PyTorch is more flexible for researchers than developers.
Limitations of PyTorch
-
PyTorch is more popular among researchers than developers.
-
It lacks productivity.
Implemented using the PyTorch framework
Install the required libraries
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.utils.data.dataloader as dataloader
import torch.optim as optim
from torch.utils.data import TensorDataset
from torchvision import transforms
from torchvision.datasets import MNIST
Copy the codeIn the code snippet below, we load the data set and split it into a training set and a test set.
train = MNIST('./data', train=True, download=True, transform=transforms.Compose([
transforms.ToTensor(),
]), )
test = MNIST('./data', train=False, download=True, transform=transforms.Compose([
transforms.ToTensor(),
]), )
dataloader_args = dict(shuffle=True, batch_size=64,num_workers=1, pin_memory=True)
train_loader = dataloader.DataLoader(train, **dataloader_args)
test_loader = dataloader.DataLoader(test, **dataloader_args)
train_data = train.train_data
train_data = train.transform(train_data.numpy())
Copy the codeIn the code snippet that follows, we will build our model and set up the activation function and optimizer.
class Model(nn.Module) :
def __init__(self) :
super(Model, self).__init__()
self.fc1 = nn.Linear(784.548)
self.bc1 = nn.BatchNorm1d(548)
self.fc2 = nn.Linear(548.252)
self.bc2 = nn.BatchNorm1d(252)
self.fc3 = nn.Linear(252.10)
def forward(self, x) :
a = x.view((-1.784))
b = self.fc1(a)
b = self.bc1(b)
b = F.relu(b)
b = F.dropout(b, p=0.5)
b = self.fc2(b)
b = self.bc2(b)
b = F.relu(b)
b = F.dropout(b, p=0.2)
b = self.fc3(b)
out = F.log_softmax(b)
return out
model = Model()
model.cuda()
optimizer = optim.SGD(model.parameters(), lr=0.001)
Copy the codeIn the code snippet below, we will train our model, and while training, we will specify the loss function, known as cross entropy.
model.train()
losses = []
for epoch in range(12) :for batch_idx, (data,data_1) in enumerate(train_loader):
data,data_1 = Variable(data.cuda()), Variable(target.cuda())
optimizer.zero_grad()
y_pred = model(data)
loss = F.cross_entropy(y_pred, target)
losses.append(loss.data[0])
loss.backward()
optimizer.step()
if batch_idx % 100= =1:
print('\r Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
epoch,
batch_idx * len(data),
len(train_loader.dataset), 100. * batch_idx / len(train_loader),
loss.data[0]),
end=' ')
print(a)Copy the code
# Evaluation model
evaluate=Variable(test_loader.dataset.test_data.type_as(torch.FloatTensor())).cuda()
output = model(evaluate)
predict = output.data.max(1) [1]
pred = pred.eq(evaluate.data)
accuracy = pred.sum()/pred.size()[0]
print('Accuracy:', accuracy)
Copy the code
Caffe
Convolutional Architecture for Fast Feature Embedding (Caffe) is an open source deep learning framework developed by Yangqing Jia. The framework supports researchers and industrial applications in the field of artificial intelligence.
Most developers use Caffe for its speed, which can process 60 million images a day using an NVIDIA K40 GPU. Caffe has many contributors to update and maintain the framework, and Caffe works well with computer vision models compared to other areas of deep learning.
Caffe’s limitations
Caffe doesn’t have a higher-level API, so it’s hard to experiment.
In Caffe, to deploy our model, we need to compile the source code.
Install the Caffe
! apt install -y caffe-tools-cpuCopy the codeImport the required libraries
import os
import numpy as np
import math
import caffe
import lmdb
Copy the codeIn the following code snippet, we specify the hardware environment.
os.environ["GLOG_minloglevel"] = '2'
CAFFE_ROOT="/caffe"
os.chdir(CAFFE_ROOT)
USE_GPU = True
if USE_GPU:
caffe.set_device(0)
caffe.set_mode_gpu()
else:
caffe.set_mode_cpu()
caffe.set_random_seed(1)
np.random.seed(24)
Copy the codeIn the code snippet that follows, we define image_generator and BATch_generator for data transformation.
def image_generator(db_path) :
db_handle = lmdb.open(db_path, readonly=True)
with db_handle.begin() as db:
cur = db.cursor()
for _, value in cur:
datum = caffe.proto.caffe_pb2.Datum()
datum.ParseFromString(value)
int_x = caffe.io.datum_to_array(datum)
x = np.asfarray(int_x, dtype=np.float32) t
yield x - 128
def batch_generator(shape, db_path) :
gen = image_generator(db_path)
res = np.zeros(shape)
while True:
for i in range(shape[0]):
res[i] = next(gen)
yield res
Copy the codeIn the code snippet below, we will give the path to the MNIST dataset.
num_epochs = 0
iter_num = 0
db_path = "content/mnist/mnist_train_lmdb"
db_path_test = "content/mnist/mnist_test_lmdb"
base_lr = 0.01
gamma = 1e-4
power = 0.75
for epoch in range(num_epochs):
print("Starting epoch {}".format(epoch))
input_shape = net.blobs["data"].data.shape
for batch in batch_generator(input_shape, db_path):
iter_num += 1
net.blobs["data"].data[...] = batch net.forward()for name, l in zip(net._layer_names, net.layers):
for b inl.blobs: b.diff[...] = net.blob_loss_weights[name] net.backward() learning_rate = base_lr * math.pow(1 + gamma * iter_num, - power)
for l in net.layers:
for b inl.blobs: b.data[...] -= learning_rate * b.diffif iter_num % 50= =0:
print("Iter {}: loss={}".format(iter_num, net.blobs["loss"].data))
if iter_num % 200= =0:
print("Testing network: accuracy={}, loss={}".format(*test_network(test_net, db_path_test)))
Copy the code
Using the following code snippet, we will get the final accuracy.
print("Training finished after {} iterations".format(iter_num))
print("Final performance: accuracy={}, loss={}".format(*test_network(test_net, db_path_test)))
Copy the code
conclusion
In this paper, we demonstrate the implementation of CNN image classification model using three well-known frameworks: Keras, PyTorch and Caffe. We can see that the CNN model developed by PyTorch is superior to the CNN model developed by Keras and Caffe in terms of accuracy and speed.
As a beginner, I started with Keras, which is a very simple framework for beginners, but its applications are limited. But PyTorch and Caffe are very powerful frameworks for speed, optimization, and parallel computing.
The original link: analyticsindiamag.com/keras-vs-py…
Welcome to panchuangai blog: panchuang.net/
Sklearn123.com/
Welcome to docs.panchuang.net/