Content introduction

From metric learning to deep metric learning, this article introduces a package in PyTorch that greatly simplifies using deep metric learning.

This article is originally posted on wechat public account “PyTorch Developer Community”.

Metric Learning is a method frequently used in machine Learning. It can construct corresponding Metric functions based on a series of observations to learn the distance or difference between data and effectively describe the similarity between samples.

CUB200 sample data set, often used as a benchmark for metric learning

This measure function returns a small distance value for highly similar observations; For observations that differ greatly, a large distance value is returned.

When the sample size is small, metric learning shows significant advantages in the accuracy and efficiency of classification tasks.

DML: Multi-category, small sample classification

However, Deep Metric Learning (DML), which combines Deep Learning and Metric Learning, is the real king when the classification task to be dealt with is very complex and has the characteristics of multiple categories and small samples.

Depth Metric Learning is also called Distance Metric Learning. Compared with metric learning, deep metric learning can do nonlinear mapping of input features.

By training a CNN-based nonlinear feature extraction module or coder, depth metric learning can embed extracted image features into neighboring locations, and distinguish different image features with the help of Euclidean distance and Cosine isometric measures.

Then, depth metric learning combined with k-nearest neighbor, support vector machine and other classification algorithms can use the extracted image features to complete the target recognition task without considering the number of categories.

Import numpy as np # Dist = Np.square (NP.sum (a-b)**2) # Dist = Np. sum(NP. abs(A-B)) # Chebyshev distance (NP. abs(A-B)) # Cosine distance similarity = (np.sum(A * B))/(nP.linalg. norm(A)) /(nP.linalg. norm(A))Copy the code

Common distance functions in depth metric learning

Deep metric learning performs well in some extreme classification tasks (with numerous categories and insufficient sample size) in the CV field, and is applied to face recognition, pedestrian re-recognition, image retrieval, target tracking, feature matching and other scenarios.

In the past, deep metric learning was used in programs, mainly relying on engineers to write code from zero to one, which was time-consuming and prone to bugs. ** can now rely on an open source library that encapsulates several commonly used modules, and ** can be called directly, saving time and effort.

PML: Make deep metric learning a breeze

Pytorch-metric-learning (PML) is an open source library that makes complex deep metric learning algorithms easier and more user-friendly.

Pytorch -metric- Learning has two main features

1. Easy to use

You can use metric learning in your program by adding just two lines of code; You can mine pairs and triplets by calling a single function.

2. Highly flexible

The fusion of loss, Miner, trainer and other modules, can achieve a variety of algorithm combinations in the existing code.

The PML consists of nine modules, each of which can be used individually or combined into a complete training/test workflow

Nine modules in PyTorch – Metric – Learning

**1, Loss: ** can be applied to a variety of Loss functions

from pytorch_metric_learning.distances import CosineSimilarity
from pytorch_metric_learning.reducers import ThresholdReducer
from pytorch_metric_learning.regularizers import LpRegularizer
from pytorch_metric_learning import losses
loss_func = losses.TripletMarginLoss(distance = CosineSimilarity(), 
             reducer = ThresholdReducer(high=0.3), 
              embedding_regularizer = LpRegularizer())
Copy the code

Sample code for customizing TripletMarginLoss function

Distance: includes various categories of computing pairwise Distance or similarity between input embedding

Reducer: ** changes from several loss values to a single loss value

** Regularizer: ** Regularizes weights and embedding vectors

Miner: **PML provides two types of mining functions: subset batch Miner and tuple Miner

from pytorch_metric_learning import miners, losses
miner = miners.MultiSimilarityMiner()
loss_func = losses.TripletMarginLoss()


# your training loop
for i, (data, labels) in enumerate(dataloader):
  optimizer.zero_grad()
  embeddings = model(data)
  hard_pairs = miner(embeddings, labels)
  loss = loss_func(embeddings, labels, hard_pairs)
  loss.backward()
  optimizer.step()
Copy the code

Add mining functionality with the Tripletmurginloss function

6, Sampler: Torch.utils.data. The extension of the Sampler class determines the composition of the batch of samples

**7. Trainer: ** Provides access to metric learning algorithms, such as data enhancement, additional networks, and so on

**8, Tester: ** Enter the model and data set to find accuracy metrics based on nearest neighbor (faISS installation package is required to use this module)

9, Util:

  • AccuracyCalculator: Given a Query and reference embedding vector, several accuracy indexes are calculated
  • Inference model:utils.inferenceContains classes for finding matching pairs in Batch or a set of pairs
  • Logging Preset: Provides log data hooks, early stop logs during model training, validation, and storage.

The loss function can be customized using Distance, Reducer and Regularizer modules

PML hands-on practice

PyTorch version requirements

Pytorch – Metric – Learning V0.9.90 and above: Torch ≥ 1.6

Pytorch – Metric – Learning V0.9.90 below: No version requirement, but test version Torch ≥ 1.2

Pip

pip install pytorch-metric-learning
Copy the code

Get the latest version

pip install pytorch-metric-learning --pre
Copy the code

Install on Windows

PIP install torch = = = 1.6.0 torchvision = = = 0.7.0 -f https://download.pytorch.org/whl/torch_stable.html PIP install pytorch-metric-learningCopy the code

To add evaluation and logging capabilities, install the unofficial PYPI version of the FAisS-GPU

pip install pytorch-metric-learning[with-hooks]
Copy the code

Or faiss – CPU

pip install pytorch-metric-learning[with-hooks-cpu]
Copy the code

Conda

conda install pytorch-metric-learning -c metric-learning -c pytorch
Copy the code

Making address:

Github.com/KevinMusgra…

Google Colab:

Github.com/KevinMusgra…

Related papers:

Arxiv.org/pdf/2008.09…

Reference:

Html.rhhz.net/tis/html/20…

Analyticsindiamag.com/guide-to-py…