MNIST classification (CPU version +GPU version)

I. Introduction to the data set

Before starting MNIST, I believe that we are familiar with MNIST data sets. Here I will not repeat the above, but just list the key parameters: categories: numbers 0~9 total 10 categories; Quantity: a total of 70,000 grayscale images, including 60,000 training images, 10,000 testing images, and each image has a label, number 0 corresponds to label 0, number 1 corresponds to label 1, and so on…… Pixel size: 28 x 28 Number of channels: single channel

Two. Network introduction

Here is a relatively simple network, most of its introduction can be seen in various forums and papers and journals, here is also not too much about the definition of network class:

import torch.nn as nn class Net(nn.Module): def __init__(self, in_c=784, out_c=10): Super (Net, self).__init__() # define full connection layer self.fc1 = nn.Linear(in_c, Self.act1 = nn.ReLU(inplace=True) self.fc2 = nn.Linear(512, 256) self.act2 = nn.ReLU(inplace=True) self.fc3 = nn.Linear(256, 128) self.act3 = nn.ReLU(inplace=True) self.fc4 = nn.Linear(128, out_c) def forward(self, x): x = self.act1(self.fc1(x)) x = self.act2(self.fc2(x)) x = self.act3(self.fc3(x)) x = self.fc4(x) return xCopy the code

Three. Manual CPU version

The network has, This starts the MNIST classification process of the dynamic compression CPU version 3.1 Get the class (network) 3.2 Get the training set and test set 3.3 Load the training set and test set (DataLoader) 3.4 Define the loss function — cross entropy function 3.5 Define the optimizer — random gradient descent 3.6 Create for loop A and create an empty array to record the training loss and accuracy, respectively: Create an empty array distribution to record test losses and accuracies (note that this should be separated from training) 3.8 Set training times 3.9 Build for loop B and set training accuracies and losses in for Loop 3.10 In for loop B: Network starts training loading 3.11 In for loop B: 3.12 In for loop B: Image and tag Variable processing 3.13 in for loop B: Network forward propagation out = net(IMG) Loss = criterion(out, label) 3.14 In for cycle B: record error 3.15 In for cycle B: Calculate classification accuracy 3.16 End B cycle: At this time, it is still in the A loop, append loss and append precision 3.17. New loop in the A loop at this time is the test set (test set is not trained), other unchanged

CPU version demo:

import time import torch.nn as nn from torch import optim from torch.autograd import Variable from torch.utils.data import DataLoader from torchvision.datasets import mnist from torchvision import transforms class Net(nn.Module): def __init__(self, in_c=784, out_c=10): Super (Net, self).__init__() # define full connection layer self.fc1 = nn.Linear(in_c, Self.act1 = nn.ReLU(inplace=True) self.fc2 = nn.Linear(512, 256) self.act2 = nn.ReLU(inplace=True) self.fc3 = nn.Linear(256, 128) self.act3 = nn.ReLU(inplace=True) self.fc4 = nn.Linear(128, out_c) def forward(self, x): x = self.act1(self.fc1(x)) x = self.act2(self.fc2(x)) x = self.act3(self.fc3(x)) x = self.fc4(x) return x t1 = Train =True, transform=transforms.ToTensor(), Mnist ('./data', train=False, transform=transforms.ToTensor(), Train_data = DataLoader(train_set, batch_size=64, Shuffle =True) # test_data = DataLoader(test_set, batch_size=64, Shuffle =True) # Define loss function -- cross entropy criterion = nn.CrossEntropyLoss() # define optimizer -- Random gradient descent optimizer = Optim. SGD (net. The parameters (), lr = 0.01, Weight_decay =0.00005) the boxing = [] # loss of training acces = [] # eval_acces = [] # Nums_epoch = 20 for epoch in range(nums_epoch): Train_loss = 0 # Set the initial value of training loss train_ACC = 0 # Set the initial value of training accuracy net.train() for batch, (img, label) in enumerate(train_data): img = img.reshape(img.size(0), Out = net(img) Loss = criterion(out, Error train_loss +=loss.item() # calculate the classification accuracy _, pred = out.max(1) num_correct = (pred == label).sum().item() acc = num_correct / img.shape[0] if (batch +1) % 200 == 0: print('[INFO] Epoch-{}-Batch-{}: Train: Loss-{:.4f},Accuracy-{:.4f}'.format(epoch+1, batch+1, loss.item(),acc)) train_acc += acc losses.append(train_acc / len(train_data)) acces.append(train_acc / len(train_data)) # eval_loss = 0 eval_acc = 0 # eval_loss = 0 eval_acc = 0 img = img.reshape(img.size(0),-1) img = Variable(img) label = Variable(label) out = net(img) loss = criterion(out, label) eval_loss += loss.item() _, pred = out.max(1) num_correct = (pred == label).sum().item() acc = num_correct / img.shape[0] eval_acc += acc Eval_losses. Append (eval_loss/len(test_data)) eval_acces.append(eval_acc/len(test_data)) # Print the parameter set_epoch = epoch+1 set_lossTrain = train_loss / len(train_data) set_AccTrain = train_acc / len(train_data) set_lossEval = eval_loss / len(test_data) set_AccEval = eval_acc / len(test_data) print('[INFO] Epoch-{}: Train: Loss-{:.4f},Accuracy-{:.4f} |Test:Loss-{:.4f}, Accuracy-{:.4f}'.format(set_epoch, set_lossTrain, set_AccTrain, set_lossEval, set_AccEval)) t2 = time.time() t = t2 - t1 print(t)Copy the code

CPU version demo(output) :

[INFO] Epoch-1-Batch-200: Train: Loss-2.2869,Accuracy-0.2500
[INFO] Epoch-1-Batch-400: Train: Loss-2.2628,Accuracy-0.4062
[INFO] Epoch-1-Batch-600: Train: Loss-2.2056,Accuracy-0.5156
[INFO] Epoch-1-Batch-800: Train: Loss-1.9502,Accuracy-0.6875
[INFO] Epoch-1: Train: Loss-2.1569,Accuracy-0.0020 |Test:Loss-1.5634, Accuracy-0.6395
[INFO] Epoch-2-Batch-200: Train: Loss-0.9539,Accuracy-0.8125
[INFO] Epoch-2-Batch-400: Train: Loss-0.8835,Accuracy-0.7500
[INFO] Epoch-2-Batch-600: Train: Loss-0.5718,Accuracy-0.8281
[INFO] Epoch-2-Batch-800: Train: Loss-0.5851,Accuracy-0.8125
[INFO] Epoch-2: Train: Loss-0.8250,Accuracy-0.0034 |Test:Loss-0.5095, Accuracy-0.8504
[INFO] Epoch-3-Batch-200: Train: Loss-0.4938,Accuracy-0.7500
[INFO] Epoch-3-Batch-400: Train: Loss-0.5644,Accuracy-0.8438
[INFO] Epoch-3-Batch-600: Train: Loss-0.4656,Accuracy-0.8750
[INFO] Epoch-3-Batch-800: Train: Loss-0.4800,Accuracy-0.8438
[INFO] Epoch-3: Train: Loss-0.4432,Accuracy-0.0035 |Test:Loss-0.3720, Accuracy-0.8914
[INFO] Epoch-4-Batch-200: Train: Loss-0.3568,Accuracy-0.8750
[INFO] Epoch-4-Batch-400: Train: Loss-0.3659,Accuracy-0.8594
[INFO] Epoch-4-Batch-600: Train: Loss-0.3843,Accuracy-0.8281
[INFO] Epoch-4-Batch-800: Train: Loss-0.3291,Accuracy-0.8906
[INFO] Epoch-4: Train: Loss-0.3600,Accuracy-0.0037 |Test:Loss-0.3328, Accuracy-0.9015
[INFO] Epoch-5-Batch-200: Train: Loss-0.2843,Accuracy-0.8906
[INFO] Epoch-5-Batch-400: Train: Loss-0.2729,Accuracy-0.9375
[INFO] Epoch-5-Batch-600: Train: Loss-0.2628,Accuracy-0.9219
[INFO] Epoch-5-Batch-800: Train: Loss-0.1479,Accuracy-0.9531
[INFO] Epoch-5: Train: Loss-0.3174,Accuracy-0.0039 |Test:Loss-0.2917, Accuracy-0.9161
[INFO] Epoch-6-Batch-200: Train: Loss-0.3273,Accuracy-0.9062
[INFO] Epoch-6-Batch-400: Train: Loss-0.2906,Accuracy-0.9375
[INFO] Epoch-6-Batch-600: Train: Loss-0.2957,Accuracy-0.9062
[INFO] Epoch-6-Batch-800: Train: Loss-0.2804,Accuracy-0.9375
[INFO] Epoch-6: Train: Loss-0.2839,Accuracy-0.0039 |Test:Loss-0.2652, Accuracy-0.9247
[INFO] Epoch-7-Batch-200: Train: Loss-0.3675,Accuracy-0.8906
[INFO] Epoch-7-Batch-400: Train: Loss-0.3041,Accuracy-0.8906
[INFO] Epoch-7-Batch-600: Train: Loss-0.2421,Accuracy-0.9375
[INFO] Epoch-7-Batch-800: Train: Loss-0.1761,Accuracy-0.9219
[INFO] Epoch-7: Train: Loss-0.2561,Accuracy-0.0039 |Test:Loss-0.2401, Accuracy-0.9319
[INFO] Epoch-8-Batch-200: Train: Loss-0.1390,Accuracy-0.9531
[INFO] Epoch-8-Batch-400: Train: Loss-0.1204,Accuracy-0.9688
[INFO] Epoch-8-Batch-600: Train: Loss-0.1118,Accuracy-0.9844
[INFO] Epoch-8-Batch-800: Train: Loss-0.1276,Accuracy-0.9844
[INFO] Epoch-8: Train: Loss-0.2306,Accuracy-0.0041 |Test:Loss-0.2178, Accuracy-0.9365
[INFO] Epoch-9-Batch-200: Train: Loss-0.4543,Accuracy-0.9062
[INFO] Epoch-9-Batch-400: Train: Loss-0.3267,Accuracy-0.9219
[INFO] Epoch-9-Batch-600: Train: Loss-0.1870,Accuracy-0.9531
[INFO] Epoch-9-Batch-800: Train: Loss-0.3354,Accuracy-0.9062
[INFO] Epoch-9: Train: Loss-0.2094,Accuracy-0.0039 |Test:Loss-0.2016, Accuracy-0.9412
[INFO] Epoch-10-Batch-200: Train: Loss-0.1400,Accuracy-0.9219
[INFO] Epoch-10-Batch-400: Train: Loss-0.2871,Accuracy-0.9219
[INFO] Epoch-10-Batch-600: Train: Loss-0.1343,Accuracy-0.9531
[INFO] Epoch-10-Batch-800: Train: Loss-0.2881,Accuracy-0.8906
[INFO] Epoch-10: Train: Loss-0.1906,Accuracy-0.0039 |Test:Loss-0.1805, Accuracy-0.9460
[INFO] Epoch-11-Batch-200: Train: Loss-0.2244,Accuracy-0.9688
[INFO] Epoch-11-Batch-400: Train: Loss-0.1173,Accuracy-0.9688
[INFO] Epoch-11-Batch-600: Train: Loss-0.1551,Accuracy-0.9531
[INFO] Epoch-11-Batch-800: Train: Loss-0.1560,Accuracy-0.9531
[INFO] Epoch-11: Train: Loss-0.1748,Accuracy-0.0041 |Test:Loss-0.1693, Accuracy-0.9504
[INFO] Epoch-12-Batch-200: Train: Loss-0.2438,Accuracy-0.9688
[INFO] Epoch-12-Batch-400: Train: Loss-0.0888,Accuracy-0.9688
[INFO] Epoch-12-Batch-600: Train: Loss-0.0938,Accuracy-0.9688
[INFO] Epoch-12-Batch-800: Train: Loss-0.1019,Accuracy-0.9688
[INFO] Epoch-12: Train: Loss-0.1611,Accuracy-0.0041 |Test:Loss-0.1562, Accuracy-0.9515
[INFO] Epoch-13-Batch-200: Train: Loss-0.2955,Accuracy-0.9219
[INFO] Epoch-13-Batch-400: Train: Loss-0.3402,Accuracy-0.9062
[INFO] Epoch-13-Batch-600: Train: Loss-0.1040,Accuracy-0.9688
[INFO] Epoch-13-Batch-800: Train: Loss-0.1147,Accuracy-0.9844
[INFO] Epoch-13: Train: Loss-0.1491,Accuracy-0.0040 |Test:Loss-0.1475, Accuracy-0.9562
[INFO] Epoch-14-Batch-200: Train: Loss-0.0578,Accuracy-1.0000
[INFO] Epoch-14-Batch-400: Train: Loss-0.0836,Accuracy-0.9688
[INFO] Epoch-14-Batch-600: Train: Loss-0.1362,Accuracy-0.9688
[INFO] Epoch-14-Batch-800: Train: Loss-0.0897,Accuracy-0.9531
[INFO] Epoch-14: Train: Loss-0.1387,Accuracy-0.0041 |Test:Loss-0.1441, Accuracy-0.9561
[INFO] Epoch-15-Batch-200: Train: Loss-0.1424,Accuracy-0.9844
[INFO] Epoch-15-Batch-400: Train: Loss-0.0657,Accuracy-0.9844
[INFO] Epoch-15-Batch-600: Train: Loss-0.0836,Accuracy-0.9688
[INFO] Epoch-15-Batch-800: Train: Loss-0.1404,Accuracy-0.9688
[INFO] Epoch-15: Train: Loss-0.1289,Accuracy-0.0042 |Test:Loss-0.1301, Accuracy-0.9608
[INFO] Epoch-16-Batch-200: Train: Loss-0.1637,Accuracy-0.9219
[INFO] Epoch-16-Batch-400: Train: Loss-0.0509,Accuracy-1.0000
[INFO] Epoch-16-Batch-600: Train: Loss-0.2507,Accuracy-0.9375
[INFO] Epoch-16-Batch-800: Train: Loss-0.0801,Accuracy-0.9688
[INFO] Epoch-16: Train: Loss-0.1205,Accuracy-0.0041 |Test:Loss-0.1252, Accuracy-0.9610
[INFO] Epoch-17-Batch-200: Train: Loss-0.0761,Accuracy-0.9688
[INFO] Epoch-17-Batch-400: Train: Loss-0.0439,Accuracy-1.0000
[INFO] Epoch-17-Batch-600: Train: Loss-0.2204,Accuracy-0.9062
[INFO] Epoch-17-Batch-800: Train: Loss-0.0640,Accuracy-0.9844
[INFO] Epoch-17: Train: Loss-0.1128,Accuracy-0.0041 |Test:Loss-0.1211, Accuracy-0.9617
[INFO] Epoch-18-Batch-200: Train: Loss-0.0907,Accuracy-0.9844
[INFO] Epoch-18-Batch-400: Train: Loss-0.0587,Accuracy-0.9844
[INFO] Epoch-18-Batch-600: Train: Loss-0.0478,Accuracy-1.0000
[INFO] Epoch-18-Batch-800: Train: Loss-0.0532,Accuracy-0.9844
[INFO] Epoch-18: Train: Loss-0.1057,Accuracy-0.0042 |Test:Loss-0.1113, Accuracy-0.9654
[INFO] Epoch-19-Batch-200: Train: Loss-0.1051,Accuracy-0.9531
[INFO] Epoch-19-Batch-400: Train: Loss-0.1953,Accuracy-0.9219
[INFO] Epoch-19-Batch-600: Train: Loss-0.1334,Accuracy-0.9531
[INFO] Epoch-19-Batch-800: Train: Loss-0.1170,Accuracy-0.9531
[INFO] Epoch-19: Train: Loss-0.0991,Accuracy-0.0040 |Test:Loss-0.1087, Accuracy-0.9662
[INFO] Epoch-20-Batch-200: Train: Loss-0.0581,Accuracy-1.0000
[INFO] Epoch-20-Batch-400: Train: Loss-0.0779,Accuracy-0.9531
[INFO] Epoch-20-Batch-600: Train: Loss-0.1448,Accuracy-0.9375
[INFO] Epoch-20-Batch-800: Train: Loss-0.0859,Accuracy-0.9688
[INFO] Epoch-20: Train: Loss-0.0934,Accuracy-0.0041 |Test:Loss-0.1075, Accuracy-0.9661
143.9970304965973
Copy the code

4.CPU2GPU improvement measures

In the process of CPU2GPU, the basic flow remains the same. Here, help is needed: Device = torch. Device (‘cuda’ if torch. Cuda.is_available () else ‘CPU ‘) 4.1 Precautions

Flexible use: To (device) can make the network run more smoothly, and the batch can be larger and resources can be allocated more rationally. Cudnn is needed here, so import torch. Backends.cudnn as cudnn 4.1.3. Carry out to(device) 4.1.4 for network, data, tag and loss function respectively. Allocate more batches rationally

4.2 Details are as follows:

2Perform on the networkTo (Device) and using CUDNN 4.2.2Allocate morebatch holdingsTo usetorch.nn 4.2.4Respectively for training and testingimg labelforto(device)

GPU version demo

import time import torch import torch.nn as nn import torch.backends.cudnn as cudnn from torch import optim from torch.autograd import Variable from torch.utils.data import DataLoader from torchvision.datasets import mnist from torchvision import transforms class Net(nn.Module): def __init__(self, in_c=784, out_c=10): Super (Net, self).__init__() # define full connection layer self.fc1 = nn.Linear(in_c, Self.act1 = nn.ReLU(inplace=True) self.fc2 = nn.Linear(512, 256) self.act2 = nn.ReLU(inplace=True) self.fc3 = nn.Linear(256, 128) self.act3 = nn.ReLU(inplace=True) self.fc4 = nn.Linear(128, out_c) def forward(self, x): x = self.act1(self.fc1(x)) x = self.act2(self.fc2(x)) x = self.act3(self.fc3(x)) x = self.fc4(x) return x t1 = Device ('cuda' if cudd.cuda.is_available () else 'CPU ') NET = net () cudnn.benchmark = Train =True, transform=transforms.ToTensor(), train=True, transform=transforms.ToTensor(), Mnist ('./data', train=False, transform=transforms.ToTensor(), Train_data = DataLoader(train_set, batch_size=640, Shuffle =True) # test_dataloader (test_set, batch_size=320, Shuffle = True) # define the loss function -- cross entropy criterion = torch. Nn. CrossEntropyLoss () to (device) # define the optimizer, stochastic gradient descent optimizer = Optim. SGD (net. The parameters (), lr = 0.01, Weight_decay =0.00005) the boxing = [] # loss of training acces = [] # eval_acces = [] # Nums_epoch = 20 for epoch in range(nums_epoch): Train_loss = 0 # Set the initial value of training loss train_ACC = 0 # Set the initial value of training accuracy net.train() for batch, (img, label) in enumerate(train_data): img = img.reshape(img.size(0), -1) img = Variable(img) img = img.to(device) label = Variable(label) label = label.to(device) # Forward spread out = net(img) loss  = criterion(out, Error train_loss += loss.item() # calculate the classification accuracy _, pred = out.max(1) num_correct = (pred == label).sum().item() acc = num_correct / img.shape[0] if (batch + 1) % 200 == 0:  print( '[INFO] Epoch-{}-Batch-{}: Train: Loss-{:.4f},Accuracy-{:.4f}'.format(epoch + 1, batch + 1, loss.item(), acc)) train_acc += acc losses.append(train_acc / len(train_data)) acces.append(train_acc / len(train_data)) eval_loss = 0 eval_acc = 0 # eval_acc = 0 img = img.reshape(img.size(0), -1) img = Variable(img) img = img.to(device) label = Variable(label) label = label.to(device) out = net(img) loss = criterion(out, label.to(device)) eval_loss += loss.item() _, pred = out.max(1) num_correct = (pred == label).sum().item() acc = num_correct / img.shape[0] eval_acc += acc Eval_losses. Append (eval_loss/len(test_data)) eval_acces.append(eval_acc/len(test_data)) # Print the parameter set_epoch = epoch + 1  set_lossTrain = train_loss / len(train_data) set_AccTrain = train_acc / len(train_data) set_lossEval = eval_loss / len(test_data) set_AccEval = eval_acc / len(test_data) print('[INFO] Epoch-{}: Train: Loss-{:.4f},Accuracy-{:.4f} |Test:Loss-{:.4f}, Accuracy-{:.4f}'.format(set_epoch, set_lossTrain, set_AccTrain, set_lossEval, set_AccEval)) t2 = time.time() t = t2 - t1 print(t)Copy the code

GPU Version Output

[INFO] Epoch - 1: "Train" : Loss - 2.3025, Accuracy 0.0000 | Test: Loss - 2.2982, Accuracy 0.1018 [INFO] Epoch - 2: "Train" : Loss - 2.2950, Accuracy 0.0000 | Test: Loss - 2.2902, Accuracy 0.1034 [INFO] Epoch - 3: "Train" : Loss - 2.2867, Accuracy 0.0000 | Test: Loss - 2.2812, Accuracy 0.1491 [INFO] Epoch - 4: "Train" : Loss - 2.2762, Accuracy 0.0000 | Test: Loss - 2.2685, Accuracy 0.3172 [INFO] Epoch - 5: "Train" : Loss - 2.2611, Accuracy 0.0000 | Test: Loss - 2.2498, Accuracy 0.4545 [INFO] Epoch - 6: "Train" : Loss - 2.2383, Accuracy 0.0000 | Test: Loss - 2.2205, Accuracy 0.5208 [INFO] Epoch - 7: "Train" : Loss - 2.2020, Accuracy 0.0000 | Test: Loss - 2.1736, Accuracy 0.5357 [INFO] Epoch - 8: "Train" : Loss - 2.1407, Accuracy 0.0000 | Test: Loss - 2.0909, Accuracy 0.5215 [INFO] Epoch - 9: "Train" : Loss - 2.0353, Accuracy 0.0000 | Test: Loss - 1.9529, Accuracy 0.5164 [INFO] Epoch - 10: "Train" : Loss - 1.8638, Accuracy 0.0000 | Test: Loss - 1.7380, Accuracy - 0.5524 [INFO] Epoch - 11: "Train" : Loss - 1.6231, Accuracy 0.0000 | Test: Loss - 1.4687, Accuracy 0.6281 [INFO] Epoch - 12: "Train" : Loss - 1.3598, Accuracy 0.0000 | Test: Loss - 1.2147, Accuracy 0.7030 [INFO] Epoch - 13: "Train" : Loss - 1.1373, Accuracy 0.0000 | Test: Loss - 1.0222, Accuracy - 0.7411 [INFO] Epoch - 14: "Train" : Loss - 0.9642, Accuracy 0.0000 | Test: Loss - 0.8700, Accuracy 0.7768 [INFO] Epoch - 15: "Train" : Loss - 0.8361, Accuracy 0.0000 | Test: Loss - 0.7663, Accuracy 0.7890 [INFO] Epoch - 16: "Train" : Loss - 0.7451, Accuracy 0.0000 | Test: Loss - 0.6880, Accuracy - 0.8087 [INFO] Epoch - 17: "Train" : Loss - 0.6791, Accuracy 0.0000 | Test: Loss - 0.6312, Accuracy 0.8223 [INFO] Epoch - 18: "Train" : Loss - 0.6299, Accuracy 0.0000 | Test: Loss - 0.5862, Accuracy - 0.8302 [INFO] Epoch - 19: "Train" : Loss - 0.5907, Accuracy 0.0000 | Test: Loss - 0.5545, Accuracy 0.8388 [INFO] Epoch - 20: "Train" : Loss - 0.5588, Accuracy 0.0000 | Test: Loss - 0.5313, Accuracy to 0.8452 84.11509966850281Copy the code

Self tapping should still not quite right, still hope correct | since ali YunTianChi instead

MNIST classification (CPU version +GPU version)

I. Introduction to the data set

Two. Network introduction

Three. Manual CPU version

4.CPU2GPU improvement measures

Related Posts

For patch generation automatic inference code conversion

Decision tree model

Python convert position coordinates to each other (Baidu coordinates, WGS_84 coordinates, Mars coordinates)