Inspection listdata/train.txtImage and save the result inresult.json
./darknet detector test cfg/coco.data cfg/yolov4.cfg yolov4.weights -ext_output -dont_show -out result.json < data/train.txt
Copy the code
Inspection listdata/train.txtImage and save the result inresult.txt
./darknet detector test cfg/coco.data cfg/yolov4.cfg yolov4.weights -dont_show -ext_output < data/train.txt > result.txt
Copy the code
How to train
1. How to construct your own training data
Download data set annotation tool, download address: (pan.baidu.com/s/1EE52cDSt…). (Password: 4b2q) or Spire Web.
Github address: github.com/jario-jin/s…
1.1 Open the annotation software spireImageTools_x.x.x.xe
First go to Tools->Setting… , fill in a Save Path (all annotation files will be stored in this folder)
If the image is captured, this step is tuned
Then, go to Tools->Video to Image
1.2 Open the image to be marked
Input->Image Dir, find the folder of the Image to be marked, Ctrl+A, select all, open
Go to Tools->Annotate Image->Instance Label to Annotate the Image
Fill in the name of the target to be labeled in label, then drag the dialog box to one side to start labeling in the main window, zoom in and out the image with the mouse wheel, press and hold the left button to move the visible image area, and continuously click the left button to surround the target box. When using Yolo training, click 2 points
1.3 Output the annotations in Yolo format for training
After the annotation is complete, press Ctrl+O
Then pull out the following 4 files for Yolo training
2. Start YOLO training
Using YOLOv4 and YOLOv3:
For the selected model, download the pre-training weights:
CFG /yolov4-custom. CFG copy it and rename it to yolov4-obj.cfg (obj can be a custom name)
Change batch to batch=64
Modify subDivisions to subDivisions =16
Modify MAX_Batches to (Number of categories *2000But not less than4000), such as training 3 categoriesmax_batches=6000
Modify Steps to 0.8 and 0.9 of MAX_batches, e.g. steps= 48005400
Modify theclasses=80For the number of categories of the custom dataset, there are 3 main changes (3[yolo]Layer) :
Github.com/AlexeyAB/da…
Github.com/AlexeyAB/da…
Github.com/AlexeyAB/da…
Modify thefilters=255forfilters=(classes+5)x3In the three[yolo]The one before the layer[convolutional]Layer, respectively:
Github.com/AlexeyAB/da…
Github.com/AlexeyAB/da…
Github.com/AlexeyAB/da…
If you are using[Gaussian_yolo]Layer, modifyfilters=57forfilters=(classes+9)x3In the three[Gaussian_yolo]The one before the layer[convolutional]Layer, respectively:
Github.com/AlexeyAB/da…
Github.com/AlexeyAB/da…
Github.com/AlexeyAB/da…
For example, if classes=1, filters=18; If classes=2, filters=21. Filters =(classes+5)x3)
Create obj.names in the Darknet /data path, where each line is a target category name
Annotated data sets resulting in a fileYolo_categories.namesrenameobj.namesAnd on thedarknet/dataUnder the
Create obj.data under darknet/data:
The tutorial Darknet path is/home/user/darknet, this document uses this parameter as an example. Modify the parameters based on your own path. Create a new file called obj.data in /home/user/darknet/cfg/ and write:
Note: classes is the number of classes. For single-class detection problems, write 1
Place the image file (.jpg) and annotation file in the following path darknet\data\coco\
willscaled_imagesTo copy the images from the/home/user/darknet/data/coco/images/trainUnder the
willYolo_labelsCopy the annotation file from the/home/user/darknet/data/coco/images/trainUnder the
willYolo_20180908_234114.txtCopy to/home/user/darknet/data/cocoUnder the
Start training
Training instructions:./darknet detector train data/obj.data cfg/yolo-obj.cfg yolov4.conv.137
(The latest weight for the last 100 iterationsyolo-obj_last.weightsAre saved indarknet\backup\)
(For the weight per 1000 iterationsyolo-obj_xxxx.weightsAre saved indarknet\backup\)
(Close the Display window of Loss./darknet detector train data/obj.data cfg/yolo-obj.cfg yolov4.conv.137 -dont_show)
(View the training process in your browser./darknet detector train data/obj.data yolo-obj.cfg yolov4.conv.137 -dont_show -mjpeg_port 8090 -map“, then open Chrome and typehttp://ip-address:8090)
(If the mAP needs to be calculated during training, it needs to be calculated every 4 sessionsobj.dataFile Settingsvalid=valid.txt, run:./darknet detector train data/obj.data yolo-obj.cfg yolov4.conv.137 -map)
At the end of the training, the results are saved in darknet\backup\yolo- obj_finals.weights
If training is interrupted, you can select a saved weight to continue training, using./darknet detector train data/obj.data yolo-obj.cfg backup\yolo-obj_2000.weights
Note: During training, if nan occurs in AVG (Loss), then the training has gone wrong, which is normal if nan occurs in other fields. Note: If you need to change width= or height= in the CFG file, the new number needs to be divisible by 32. / Darknet Detector test data/obj.data yolo-obj.cfg yOLO -obj_8000.weights Note: After training, the inspection command is./ Darknet Detector test data/obj.data yolo-obj.cfg yOLO -obj_8000.weights. The subdivisions=16 in the CFG file need to be changed to 32 or 64 if there is Out of memory.
3. Training YOLOv3 – Tiny
Training YOLOV3-TINY is basically the same as selecting YOLOv4 and YOLOv3, with the following minor differences:
Download yolov3-tiny pre-training weights and run the command./darknet partial cfg/yolov3-tiny.cfg yolov3-tiny.weights yolov3-tiny.conv.15 15
Creating a Customcfgfileyolov3-tiny-obj.cfg(Can be copiedcfg/yolov3-tiny.cfgforyolov3-tiny-obj.cfg)
Run training commands:./darknet detector train data/obj.data yolov3-tiny-obj.cfg yolov3-tiny.conv.15
4. Multi-gpu training
First train 1000 times on 1 GPU./darknet detector train cfg/coco.data cfg/yolov4.cfg yolov4.conv.137
Stop training and use heavy weightsdarknet/backup/yolov4_1000.weightsAnd train and run on multiple Gpus/darknet detector train CFG /coco. Data CFG /yolov4.cfg /backup/yolov4_1000. Weights -gpus 0,1,2,3
Note: If nan occurs, you should lower the learning rate, such as 4 blocks of GPUlearning_rate=0.00065 (Learning_rate =0.00261/GPUs), and add burn_in= in CFG file to the original 4x, such as burn_in=4000
5. Training common program problems
Note: If the following error occurs
Need to modify the source code/home/user/darknet/SRC/data. C to the following code
list *get_paths(char *filename)
{
char *path;
FILE *file = fopen(filename, "r");
if(! file) file_error(filename); list *lines = make_list();while((path=fgetl(file))) {
list_insert(lines, path);
}
fclose(file);
return lines;
}
Copy the code
Save, make -j8 recompile below for normal training screen
6. When to stop training
In general, 2000 iterations for each category are sufficient, and the total number of iterations should not be less than 4000. However, if you want a more precise stop time, you can refer to the following instructions:
During training, you will see a series of training errors, and when the 0.XXXXXXX AVG parameter no longer decreases, it is time to stop training
0.60730 AVG – Average loss (error), the lower the better
If you find that the 0.xxxxXXX AVG no longer decreases after many iterations, it’s time to stop training. The final average loss ranged from 0.05 for small models and simple training data to 3.0 for large models and complex training data.
When training stops, you can startdarknet\backupTo retrieve the latest saved training weights.weightsAnd select the one that works best among them
For example, when training is stopped 9000 times, the model that works best may be one of the previous save weights (7000,8000,9000) because of the phenomenon of Overfiting. The performance of overfitting can be interpreted as that the detection effect is good on the training image, but the effect is not good on other images, so the training should be stopped as soon as possible (early stop point).
2.1 First, you need to specify the validating dataset valid=valid. TXT in obj.data. If you do not have the validating dataset, you can simply copy data\train. TXT to data\valid. TXT.
2.2 If you stop training after 9000 iterations, verify previous model weights using the following command:
Then compare the last line of output of each weight (7000,8000,9000), select mAP (Mean Average Precision) with the highest weight, or compare IoU (Intersect over Union) for selection.
For example, if the yolo-obj_8000.weights mAP is the highest, use this weight. Or add the -map parameter to the training:
./darknet detector train data/obj.data cfg/yolo-obj.cfg yolov4.conv.137 -map
Copy the code
The results are shown in the following figure. MAP is calculated every 4 phases (Epoch) on the validation set VALID =valid. TXT set in obj.data (1 phase = number of images in train_TXT/batch iteration).
Run the trained model, conduct target detection, and execute:
./darknet detector test data/obj.data cfg/yolo-obj.cfg yolo-obj_8000.weights
Copy the code
7. How to improve the detection effect
7.1 Skills of improving detection effect before training
CFG file random=1, can use multi-resolution input to increase detection effect: link
Increasing the network input resolution in the.cfg file (set any number divisible by 32, for example, height=608, width=608) increases the accuracy
Check whether every object in the image is marked and all objects in the image must be marked correctly. It is recommended to use the data management tool spire-image-Manager to check
The Loss is very large and the mAP is very low. Is the training wrong? Using the -show_imgs parameter in training allows you to visualize the target box truth value and check if there is a problem with the data set.
For every object you want to detect, there should be at least one similar instance in the training data set, including: shape, object side, relative size, rotation Angle, tilt azimuth, lighting, etc. Therefore, your training data set needs to contain images with different object attributes: scale, rotation, lighting, different sides, different backgrounds, etc. It is recommended to collect 2000 different images for each class of objects and iteratively train 2000* class number of times.
It is recommended to include images with unlabeled objects that you do not want to detect in the training dataset. Negative sample images do not require box marks (empty.txt files), the more the better.
The best way to mark a target is to mark only the visible part of the object, or mark the visible and overlapping parts of the object, or mark the part that is slightly more than the whole object (there is a gap), and mark the part that you want the detector to detect.
If there are many objects in a single image, you need to modify the parameter Max =200 or higher in the [yolo] layer or [region] layer (the global maximum number of objects detected is 0,0615234375*(width*height)).
If you want to detect small objects (the image is scaled toAfter less thanThe goal of)
Changes in the https://github.com/AlexeyAB/darknet/blob/6f718c257815a984253346bba8fb7aa756c55090/cfg/yolov4.cfg#L895layers = 23
Changes in the https://github.com/AlexeyAB/darknet/blob/6f718c257815a984253346bba8fb7aa756c55090/cfg/yolov4.cfg#L892stride=4
Changes in the https://github.com/AlexeyAB/darknet/blob/6f718c257815a984253346bba8fb7aa756c55090/cfg/yolov4.cfg#L989stride=4
If you want to detect both large and small objects, you can use the modified model:
All model – five yolo layer: raw.githubusercontent.com/AlexeyAB/da…
The small model – three yolo layer: raw.githubusercontent.com/AlexeyAB/da…
YOLOv4 – three yolo layer: raw.githubusercontent.com/AlexeyAB/da…
If the data class you are training needs to distinguish between left and right objects (such as detecting left and right, left and right directions in traffic signals), you cannot use left and right flip image enhancement. Set flip=0: github.com/AlexeyAB/da in the CFG file.
General rule – Your training data set should contain a set of relative sizes of targets to be detected:
That is, for every object in the test dataset, there must be at least one object in the training dataset with the same class and approximately the same relative size. If only objects occupying 80-90% of the image area are included in the training data, the trained network cannot detect objects occupying 1-10% of the image area.
If you want to speed up training (loss detection accuracy), you can set the parameter StopBACKWARD =1 in CFG file layer-136
Note the attributes of the object such as model, profile, illumination, scale, azimuth, etc. These are different objects from the internal perspective of the neural network. Therefore, the more objects you want to detect, the more complex the network model should be used.
If you want to outsource rectangular boxes more accurately, you can add three parameters to the [YOLO] layer: ignore_THresh =.9 iou_Normalizer =0.5 iou_Loss =giou, which increases [email protected] and decreases [email protected].
If you are familiar with detection networks, you can recalculate the Anchor of your custom dataset:./darknet detector calc_anchors data/obj.data -num_of_clusters 9 -width 416 -height 416And then set three in the CFG file[yolo]Layer 9 anchor frames. You need to change every one of them[yolo]The anchor frame index in the layermask=The first layer is greater thanThe anchor frame, the second layer has greater thanThe anchor frame of the third layer is the same. Also need to change each[yolo]Layer beforefilters=(classes + 5)*<number of mask>. If many of the calculated anchor frames do not fit under the appropriate layer – then try using the default anchor frame.
7.2 Skills to improve detection effect after training
Increase the resolution of network input in CFG file, for example,height=608.width=608Or,height=832.width=832, so that smaller targets can be detected.
How do you deploy a trained model on a DRONE
1. Preparation work on TX2
Ubuntu 18.04 is recommended (JetPack can be used)
CMake > = 3.8: cmake.org/download/
CUDA > = 10.0: developer.nvidia.com/cuda-toolki…
CuDNN >= 7.0 for CUDA >= 10.0 Developer.nvidia.com/rdp/cudnn-a…
OpenCV > = 2.4: opencv.org/releases.ht…
GCC
ROS Melodic: wiki.ros.org/melodic/Ins…
1.1 Install CUDA and cuDNN for TX2 using JetPack
Download the JetPack, address: developer.nvidia.com/embedded/je…
Go to sdkManager -[version].[build].deb, where version and build represent their respective numbers, and install the Debian package:
sudo apt install ./sdkmanager-[version].[build].deb
Copy the code
Once installed, type in Terminal
sdkmanager
Copy the code
Log in using your NVIDIA account
Selecting a development environment
Select Jetson in Product Category.
In the Hardware Configuration, select Target Hardware (Jetson TX2) and tick off host Machine
Select the version of JetPack in Target Operating System.
Click CONTINUE to go to the next step
Check the download components (if CUDA and cuDNN are installed only, check only the options in the red circle), select the storage path, and accept terms
Ensure that the Host computer and TX2 in the same LAN, input TX2 IP address can be installed
2. The deployment of Darknet – ROS
Download the darknet_ros source code
cd ~
cd catkin_ws/src
git clone --recursive https://github.com/leggedrobotics/darknet_ros.git
cd ../
Copy the code
compile
catkin_make -DCMAKE_BUILD_TYPE=Release
Copy the code
Load the trained CFG and weights into Darknet_ros
The/home/user/darknet/CFG/yolov3 – tiny. CFG and/home/user/darknet/backup just trained parameters Copy the files to CFG and weights in /home/user/catkin_ws/ SRC /darknet_ros/darknet_ros/yolo_network_config respectively Create yolov3-tiny-obj.yaml in /home/user/catkin_ws/ SRC /darknet_ros/darknet_ros/config
After less than
The goal of)
The anchor frame, the second layer has greater than
The anchor frame of the third layer is the same. Also need to change each
After less than
The goal of)
The anchor frame, the second layer has greater than
The anchor frame of the third layer is the same. Also need to change each
