You may be familiar with the seemingly endless machine learning that has emerged recently, but do you know how to train machine learning models? Typically, a given machine learning model is trained on specific data for a specific task. This training process is very resource and time consuming, and because the generated model is task-specific, it does not reach its full potential.
Very good performance neural networks are often the result of many fine-tuning by researchers or practitioners. Can these trained models be used for broader categorization of tasks? Transfer learning involves applying existing machine learning models to untrained scenarios.
Just as humans do not discard what they have previously learned and start over each time they start a new task, transfer learning allows a machine learning model to apply the acquired knowledge to the training process for a new task, thus expanding the scope for combining computing and expertise and using it as fuel for the original model. In short, transfer learning can save training time and extend the availability of existing machine learning models. Transfer learning is a valuable technique for models that require large amounts of data to be trained from scratch (and that data is temporarily unavailable).
Being familiar with complex concepts and applying these concepts in practice are two very different things. In recent years, transfer learning has shown promising prospects in many fields and is a very active field of contemporary machine learning research. If you’re looking for a complete guide to deep and transferred learning (learning from scratch), then this book can be your first stop.
Python Transfer learning
Advanced deep learning and neural network models are implemented using TensorFlow and Keras
By Dipanjan Sarkar. Translated by Zhang Haoran
The editors recommend
1. Learn quickly
This book will help you to clarify the key basic concepts of machine learning and deep learning, and describe important deep learning architectures, including convolutional neural networks, deep neural networks, recursive neural networks, short and long memory neural networks, and capsule networks.
2. Consolidate transfer learning theory with practical operation
By reading this book, the reader will have a deep understanding of the concepts of transfer learning, and grasp model freezing, model tuning, and pre-training models (including VGG, Inception, and ResNet models) with relevant programming cases.
3. Master practical skills
The book focuses on a number of real-world cases and problems in different fields, such as computer vision, audio analysis, and natural language processing.
As a technical person (such as a programmer), it is important to keep your knowledge up to date and learn how to use relevant tools and techniques. This book is designed to help Python practitioners use the techniques in their respective fields. The structure of the book is roughly divided into the following three parts:
- Foundation of deep learning;
- Transfer learning essentials;
- Transfer learning case studies.
Transfer Learning is a Machine Learning (ML) technology, which refers to the acquisition of knowledge from the training of a series of Machine Learning problems and the use of this knowledge to train other similar types of problems.
The book is divided into three parts: The first part is the foundation of deep learning, which introduces the basic knowledge of machine learning, the basic knowledge of deep learning and the architecture of deep learning; The second part is the essence of transfer learning, which introduces the basic knowledge and power of transfer learning. The third part is a case study of transfer learning, which introduces image recognition and classification, text document classification, audio event recognition and classification, DeepDream algorithm, style transfer, automatic image scan generator, image coloring and so on. This book is for data scientists, machine learning engineers, and data analysts, as well as readers interested in machine learning and transfer learning. Before reading this book, readers are expected to have a basic understanding of machine learning and Python programming.
directory
Part 1 fundamentals of deep learning
Chapter 1 Fundamentals of Machine learning 2
1.1 What is Machine learning 3
1.1.1 Formal Definition of Machine learning 4
1.1.2 Shallow learning and Deep learning 4
1.2 Machine learning Algorithm 5
1.2.1 Supervised Learning 6
1.2.2 Unsupervised Learning 9
1.3 CRISP – DM 11
1.3.1 Service Understanding 12
1.3.2 Data Understanding 12
1.3.3 Data Preparation 13
1.3.4 Modeling 13
1.3.5 Assessment 13
1.3.6 Deployment 14
1.4 Standard Machine learning Workflows 14
1.4.1 Data Retrieval 15
1.4.2 Data Preparation 15
1.4.3 Modeling 16
1.4.4 Model evaluation and tuning 17
1.4.5 Deployment and Monitoring 21
1.5 Exploratory data analysis 22
1.6 Feature extraction and feature engineering 29
1.7 Feature selection 33
1.8 Conclusion 34
Chapter 2 Essentials of Deep Learning 35
2.1 What is Deep learning 35
2.2 Deep learning Framework 37
2.3 Creating a GPU-supported cloud deep learning environment 40
2.3.1 Choosing a cloud provider 41
2.3.2 Configuring a Virtual Server 41
2.3.3 Configuring a Virtual Server 45
2.3.4 Installing and Upgrading Deep learning Dependencies 47
2.3.5 Accessing the Deep Learning Cloud 52
2.3.6 Verifying GPU 53 in a Deep Learning Environment
2.4 Creating a robust gPU-enabled internal deep learning environment 55
2.5 Fundamentals of neural networks 55
2.5.1 A simple linear neuron 55
2.5.2 Optimization problem based on gradient 56
2.5.3 Jacobian and Hessian matrices 60
2.5.4 Chain rule of Derivatives
2.5.5 Stochastic gradient Descent 61
2.5.6 Nonlinear neural unit 64
2.5.7 Learn a simple nonlinear unit — Logic unit 66
2.5.8 Loss function 67
2.5.9 Data Representation 68
2.5.10 Multilayer neural network 71
2.5.11 Back propagation — Training deep neural network 73
2.5.12 Challenges in neural network learning 75
2.5.13 Initializing Model Parameters 78
2.5.14 Upgrading SGD 80
2.5.15 Over-fitting and under-fitting of neural network 81
2.5.16 Hyperparameters of neural network 84
2.6 Conclusion 85
Chapter 3 Understanding deep learning architecture 86
3.1 Neural network Architecture 86
3.2 Various neural network architectures 88
3.2.1 Multilayer perceptron and deep neural network 88
3.2.2 Self-coding neural network 88
3.2.3 Variational autoencoder 89
3.2.4 Generative adversarial network 91
3.2.5 Convolutional neural network 93
3.2.6 Capsule network 103
3.2.7 Recursive neural network 107
3.2.8 Memory neural network 114
3.2.9 Neural Turing machine 117
3.2.10 Attention-based neural network Model 121
3.3 Conclusion 122
Part 2: Transfer learning essentials
Chapter 4 Foundations of Transfer learning 124
4.1 Introduction to Transfer learning 124
4.2 Transfer learning Strategy 127
4.3 Transfer learning and Deep Learning
4.3.1 Transfer learning Methodology 130
4.3.2 Pre-training model 131
4.3.3 Application 131
4.4 Deep Transfer Learning Type 132
4.4.1 Domain adaptation 132
4.4.2 Domain confusion 132
4.4.3 Multitasking 133
4.4.4 One-time learning 133
4.4.5 Zero sample learning 134
4.5 Challenges of transfer learning 134
4.5.1 Negative Migration 134
4.5.2 Migration boundary 135
4.6 Summary 135
Chapter 5 Unleashing the Power of Transfer Learning 136
5.1 The necessity of transfer learning 137
5.1.1 Elaboration of Real World problems 137
5.1.2 Construction of Dataset 138
5.1.3 Description Method 140
5.2 Build CNN model 140 from scratch
5.2.1 Basic CNN model 143
5.2.2 Regularized CNN model 146
5.2.3 Image enhancement CNN model 148
5.3 Utilizing transfer learning using pre-trained CNN models 153
5.3.1 Understanding the VGG-16 Model 154
5.3.2 Pre-trained CNN model 156 as feature extractor
5.3.3 Pre-trained CNN model 162 using image enhancement as feature extractor
5.3.4 Pre-trained CNN model 164 using fine-tuning and image enhancement
5.4 Evaluating our deep learning model 168
5.4.1 Prediction of the model on a sample test image
5.4.2 Visualize the perception of CNN model 170
5.4.3 Evaluating model performance on test data 174
5.5 Summary 177
Part 3 case study of transfer learning
Chapter 6 Image Recognition and Classification 180
6.1 Image classification based on deep learning 180
6.2 Baseline dataset 181
6.3 State-of-the-art depth image classification model 182
6.4 Image classification and transfer learning 183
6.4.1 CIFAR-10 Data set 183
6.4.2 Breed identification data set 189
6.5 Conclusion 199
Chapter 7 Classification of Text Documents 200
7.1 Text Classification 201
7.1.1 Traditional Text Classification 201
7.1.2 Disadvantages of BoW model 202
7.1.3 Baseline dataset 203
7.2 Word Forms 204
7.2.1 Word2vec model 204
7.2.2 Word2vec model using Gensim framework 205
7.2.3 GloVe Model 208
7.3 CNN Document Model 210
7.3.1 Building a Comment Sentiment classifier 216
7.3.2 Which word embedding changes the most 220
7.3.3 Application of transfer learning in IMDB Dataset 220
7.3.4 Embedding Wordvec on a complete IMDB data set for training 223
7.3.5 Creating document Summaries using CNN Model 224
7.3.6 Multi-category Classification using CNN Model 228
7.3.7 Document Embedding Visualization 231
7.4 Conclusion 233
Chapter 8 Audio Event Recognition and Classification 234
8.1 Understanding audio Event classification 235
8.2 Exploratory analysis of audio events 236
8.3 Feature engineering and representation methods of audio events 245
8.4 Audio Event classification using transfer Learning 249
8.4.1 Construct dataset 250 based on basic features
8.4.2 Feature Extraction using transfer learning 251
8.4.3 Building a classification model 253
8.4.4 Evaluating the performance of the classifier 257
8.5 Building a deep learning audio event recognizer 260
8.6 Conclusion 263
Chapter 9 DeepDream 264
9.1 Introduction 264
9.1.1 Algorithms in computer vision Pareidolia 265
9.1.2 Visual Feature Diagram 267
9.2 DeepDream algorithm 273
9.3 Summary 277
Chapter 10 Style Transfer 278
10.1 Understanding Neural style Transfer 279
10.2 Image preprocessing method 280
10.3 Construction of loss function 282
10.3.1 Content loss 282
10.3.2 Loss of style 283
10.3.3 Total variation loss 283
10.3.4 Total loss function 284
10.4 Creating a Custom Optimizer 285
10.5 Style transfer combat 286
10.6 Conclusion 290
Chapter 11 Automatic Image scan generator 291
11.1 Understanding Image Description 292
11.2 Clear goals 293
11.3 Understanding Data 294
11.4 Methods of constructing automatic image description system 295
11.4.1 Conceptual methods 295
11.4.2 Hands-on Methods 299
11.5 Image feature extraction using Transfer Learning 301
11.6 Build a vocabulary for descriptions 306
11.7 Building an image description dataset generator 308
11.8 Construction of image language encoder-decoder deep learning model 314
11.9 Training image description deep learning model 316
11.10 Evaluation of image description deep learning model 320
11.10.1 Loading Data and Model 320
11.10.2 Understanding greedy search and cluster search 321
Implement a description generator based on cluster search 321
11.10.4 Understand and implement BLEU score 323
11.10.5 Evaluating model performance on test data 324
11.11 Automatic Image Description Combat 327
11.11.1 Image Description of Sample Outdoor Scene 329
11.11.2 Image description of popular motion Samples 332
11.11.3 Room for future improvement 334
11.12 Conclusion 334
Chapter 12 image Coloring 335
12.1 Problem Statement 336
12.2 Color Images 336
12.2.1 Color theory 337
12.2.2 Color models and colors
Room 337
12.2.3 Reexamining statement of Question 339
12.3 Construction of a deep coloring neural network
12.3.1 Pretreatment 341
12.3.2 Loss function 342
12.3.3 Coder 343
12.3.4 Transfer learning — Feature extraction 344
12.3.5 Fusion Layer 344
12.3.6 Decoder 345
12.3.7 Post-processing 347
12.3.8 Training and Results
12.4 Challenge 350
12.5 Further improvement 350
12.6 Summary 350