When you hear “search by map”, do you think of baidu, Google and other search engines search by map function? In fact, you can build a picture search system of their own: their own photo library; Select a picture to search in the library, and get a number of similar pictures.
Milvus, as a similarity search engine for massive feature vectors, aims to help analyze the increasingly large amount of unstructured data and explore the huge value behind it. In order to enable Milvus to be applied to similar image retrieval scenarios, we designed an image search system based on Milvus and image feature extraction model VGG.
1. Data preparation
Note: You can also use other image data to load. The supported image formats are. JPG and. PNG.
2. System Overview
3. VGG model
VGGNet, developed by researchers from the Visual Geometry Group at the University of Oxford and Google DeepMind, is the first location task and second classification task in ILSVRC-2014. Its outstanding contribution is to prove that the effect of the model can be effectively improved by increasing the network depth with small convolution (3*3), and VGGNet has good generalization ability for other data sets. VGG model performs better than GoogleNet in multiple transfer learning tasks, and VGG model is the preferred algorithm for extracting CNN features from images. Therefore, VGG is selected as the deep learning model in this scheme.
VGGNet explored the relationship between the depth of CNN and its performance. By repeatedly stacking 3*3 small convolutional kernels and 2*2 maximum pooling layer, VGGNet successfully constructed CNN with 16-19 layers of depth. In this scheme, the VGG16 model provided by Keras application module (Keras.Applications) is used.
(1) VGG16 structure
VGG16 contains 13 Convolutional layers, 3 Fully connected layers, and 5 Pool layers. Among them, the convolutional layer and the fully connected layer have weight coefficients, so they are also called weight layers. The total number is 13+3=16, which is the source of 16 in VGG16. (The pooling layer does not involve weights and therefore does not belong to the weight layer and is not counted).
(2) VGG16 characteristics
-
The convolution layer adopts the same convolution kernel parameters
-
The same pooling kernel parameters are used for all pooling layers
-
The model is composed of several convolutional layers and pooling layers stacked (stack), which is easy to form a deep network structure
(3) VGG16 block structure
(4) Weight parameters
Convolution kernel weightand
Full connection layer weight. Therefore, it has high fitting ability.
4. The API is introduced
(1) train
methods | name | type |
---|---|---|
POST | File | string |
from preprocessor.vggnet import VGGNetnorm_feat = model.vgg_extract_feat(img_path)Copy the code
from indexer.index import milvus_client, insert_vectorsstatus, ids = insert_vectors(index_client, table_name, vectors)Copy the code
from diskcache import Cachefor i in range(len(names)): cache[ids[i]] = names[i]Copy the code
(2) process
(3) count
(4) search
methods | Num | file |
---|---|---|
POST | topk (int) | image file |
from preprocessor.vggnet import VGGNetnorm_feat = model.vgg_extract_feat(img_path)Copy the code
from milvus import Milvus, IndexType, MetricType, Statusstatus, results = client.search_vectors(table_name=table_name, query_records=vectors, top_k=top_k, nprobe=16)Copy the code
from diskcache import Cachedef query_name_from_ids(vids): res = [] cache = Cache(default_cache_dir) for i in vids: if i in cache: res.append(cache[i]) return resCopy the code
(5) delete
5. Build an image
(1) Construct pic-Search-WebServer image
$ git clone https://github.com/milvus-io/bootcamp.git$ cd bootcamp/solutions/pic_search/webserver# build mirror $docker build -t PIC - search - webserver. # | view image generated $docker images - search - webserver grep PICCopy the code
$docker pull milvusbootcamp/PIC - search - webserver: 0.1.0 fromCopy the code
(2) Build pic-Search-WebClient image
$ git clone https://github.com/milvus-io/bootcamp.git$ cd bootcamp/solutions/pic_search/webclient# build mirror $docker build -t PIC - search - webclient. # | view image generated $docker images - search - webclient grep PICCopy the code
$docker pull milvusbootcamp/PIC - search - webclient: 0.1.0 fromCopy the code
6. Deploy the system
Step 1 Start Milvus Docker
Step 2 Start pic-search-WebServer Docker
$ docker run -d --name zilliz_search_images_demo \-v IMAGE_PATH1:/tmp/pic1 \-v IMAGE_PATH2:/tmp/pic2 \-p 35000:5000 \-e "DATA_PATH=/tmp/images-data" \-e "MILVUS_HOST = 192.168.1.123"\ milvusbootcamp/PIC - search - webserver: 0.1.0 fromCopy the code
Step 3 Start pic-search-WebClient Docker
$ docker run --name zilliz_search_images_demo_web \-d --rm -p 8001:80 \-eAPI_URL = http://192.168.1.123:35000 \ milvusbootcamp/PIC - search - webclient: 0.1.0 fromCopy the code
7. Interface display
VGG official website:http://www.robots.ox.ac.uk/~vgg/research/very_deep/VGG lot:https://github.com/machrisaa/tensorflow-vgg
github.com/milvus-io/milvus
milvus.io
milvusio.slack.com
= >Click on the menu
“Online communication” =>Add ZILLIZ to the group