darknet-训练自己的yolov3模型

川长思鸟来 2022-02-19 07:59 514阅读 0赞

目录

  • Yolo v3的使用方法

    • 安装darknet

    • 训练Pascal VOC格式的数据

      • 修改cfg文件中的voc.data
      • 修改VOC.names
      • 下载预训练卷积层权重
      • 修改cfg/yolov3-voc.cfg
    • 训练自己的模型

    • 测试Yolo模型

      • 测试单张图片：
    • 生成预测结果
    • 采用第三方compute_mAP
    • Reference

Yolo v3的使用方法

参考自@zhaonan

安装darknet

  • 下载库文件

    git clone https://github.com/pjreddie/darknet
    cd darknet

  • 修改Makefile

    GPU=1 #0或1
    CUDNN=1 #0或1
    OPENCV=0 #0或1
    OPENMP=0
    DEBUG=0

  • 编译

    make

  • 下载预训练模型

    wget https://pjreddie.com/media/files/yolov3.weights

  • 用预训练模型进行简单的测试

    ./darknet detect cfg/yolov3.cfg yolov3.weights data/dog.jpg

训练Pascal VOC格式的数据

  • 生成Labels，因为darknet不需要xml文件，需要.txt文件（格式： ）

用voc_label.py（位于./scripts）cat voc_label.py 共修改四处

  1. import xml.etree.ElementTree as ET
  2. import pickle
  3. import os
  4. from os import listdir, getcwd
  5. from os.path import join
  6. sets=[('2007', 'train'), ('2007', 'val'), ('2007', 'test')]  #替换为自己的数据集
  7. classes = ["head", "eye", "nose"]     #修改为自己的类别
  9. def convert(size, box):
  10.     dw = 1./(size[0])
  11.     dh = 1./(size[1])
  12.     x = (box[0] + box[1])/2.0 - 1
  13.     y = (box[2] + box[3])/2.0 - 1
  14.     w = box[1] - box[0]
  15.     h = box[3] - box[2]
  16.     x = x*dw
  17.     w = w*dw
  18.     y = y*dh
  19.     h = h*dh
  20.     return (x,y,w,h)
  21. def convert_annotation(year, image_id):
  22.     in_file = open('VOCdevkit/VOC%s/Annotations/%s.xml'%(year, image_id))  #将数据集放于当前目录下
  23.     out_file = open('VOCdevkit/VOC%s/labels/%s.txt'%(year, image_id), 'w')
  24.     tree=ET.parse(in_file)
  25.     root = tree.getroot()
  26.     size = root.find('size')
  27.     w = int(size.find('width').text)
  28.     h = int(size.find('height').text)
  29.     for obj in root.iter('object'):
  30.         difficult = obj.find('difficult').text
  31.         cls = obj.find('name').text
  32.         if cls not in classes or int(difficult)==1:
  33.             continue
  34.         cls_id = classes.index(cls)
  35.         xmlbox = obj.find('bndbox')
  36.         b = (float(xmlbox.find('xmin').text), float(xmlbox.find('xmax').text), float(xmlbox.find('ymin').text), float(xmlbox.find('ymax').text))
  37.         bb = convert((w,h), b)
  38.         out_file.write(str(cls_id) + " " + " ".join([str(a) for a in bb]) + '\n')
  39. wd = getcwd()
  40. for year, image_set in sets:
  41.     if not os.path.exists('VOCdevkit/VOC%s/labels/'%(year)):
  42.         os.makedirs('VOCdevkit/VOC%s/labels/'%(year))
  43.     image_ids = open('VOCdevkit/VOC%s/ImageSets/Main/%s.txt'%(year, image_set)).read().strip().split()
  44.     list_file = open('%s_%s.txt'%(year, image_set), 'w')
  45.     for image_id in image_ids:
  46.         list_file.write('%s/VOCdevkit/VOC%s/JPEGImages/%s.jpg\n'%(wd, year, image_id))
  47.         convert_annotation(year, image_id)
  48.     list_file.close()   
  49. os.system("cat 2007_train.txt 2007_val.txt > train.txt")     #修改为自己的数据集用作训练
  51. wget https://pjreddie.com/media/files/voc_label.py
  52. python voc_label.py

在VOCdevkit/VOC2007/labels/中：

  1. learner@learner-pc:~/darknet/scripts$ ls
  2. 2007_test.txt #0   dice_label.sh        imagenet_label.sh  VOCdevkit_original
  3. 2007_train.txt #1  gen_tactic.sh        train.txt #3        voc_label.py
  4. 2007_val.txt #2 get_coco_dataset.sh  VOCdevkit

这时darknet需要一个txt文件，其中包含了所有的图片

  1. cat 2007_train.txt 2007_val.txt 2012_*.txt > train.txt

修改cfg文件中的voc.data

  1. classes= 3    #修改为自己的类别数
  2. train  = /home/learner/darknet/data/voc/train.txt   #修改为自己的路径 or /home/learner/darknet/scripts/2007_test.txt
  3. valid  = /home/learner/darknet/data/voc/2007_test.txt   #修改为自己的路径 or /home/learner/darknet/scripts/2007_test.txt
  4. names = /home/learner/darknet/data/voc.names  #修改见voc.names
  5. backup = /home/learner/darknet/backup   #修改为自己的路径，输出的权重信息将存储其内

修改VOC.names

  1. head  #自己需要探测的类别，一行一个
  2. eye
  3. nose

下载预训练卷积层权重

  1. wget https://pjreddie.com/media/files/darknet53.conv.74

修改cfg/yolov3-voc.cfg

  1. [net]
  2. # Testing
  3.  batch=64
  4.  subdivisions=32   #每批训练的个数=batch/subvisions，根据自己GPU显存进行修改，显存不够改大一些
  5. # Training
  6. # batch=64
  7. # subdivisions=16
  8. width=416
  9. height=416
  10. channels=3
  11. momentum=0.9
  12. decay=0.0005
  13. angle=0
  14. saturation = 1.5
  15. exposure = 1.5
  16. hue=.1
  18. learning_rate=0.001
  19. burn_in=1000
  20. max_batches = 50200  #训练步数
  21. policy=steps
  22. steps=40000,45000  #开始衰减的步数
  23. scales=.1,.1
  27. [convolutional]
  28. batch_normalize=1
  29. filters=32
  30. size=3
  31. stride=1
  32. pad=1
  33. activation=leaky
  35. .....
  37. [convolutional]
  38. size=1
  39. stride=1
  40. pad=1
  41. filters=24   #filters = 3 * ( classes + 5 )   here,filters=3*(3+5)
  42. activation=linear
  44. [yolo]
  45. mask = 6,7,8
  46. anchors = 10,13,  16,30,  33,23,  30,61,  62,45,  59,119,  116,90,  156,198,  373,326
  47. classes=3    #修改为自己的类别数
  48. num=9
  49. jitter=.3
  50. ignore_thresh = .5
  51. truth_thresh = 1
  52. random=1
  54. [route]
  55. layers = -4
  57. [convolutional]
  58. batch_normalize=1
  59. filters=256
  60. size=1
  61. stride=1
  62. pad=1
  63. activation=leaky
  65. [upsample]
  66. stride=2
  68. [route]
  69. layers = -1, 61
  73. [convolutional]
  74. batch_normalize=1
  75. filters=256
  76. size=1
  77. stride=1
  78. pad=1
  79. activation=leaky
  81. [convolutional]
  82. batch_normalize=1
  83. size=3
  84. stride=1
  85. pad=1
  86. filters=512
  87. activation=leaky
  89. [convolutional]
  90. batch_normalize=1
  91. filters=256
  92. size=1
  93. stride=1
  94. pad=1
  95. activation=leaky
  97. [convolutional]
  98. batch_normalize=1
  99. size=3
  100. stride=1
  101. pad=1
  102. filters=512
  103. activation=leaky
  105. [convolutional]
  106. batch_normalize=1
  107. filters=256
  108. size=1
  109. stride=1
  110. pad=1
  111. activation=leaky
  113. [convolutional]
  114. batch_normalize=1
  115. size=3
  116. stride=1
  117. pad=1
  118. filters=512
  119. activation=leaky
  121. [convolutional]
  122. size=1
  123. stride=1
  124. pad=1
  125. filters=24    #filters = 3 * ( classes + 5 )   here,filters=3*(3+5)
  126. activation=linear
  128. [yolo]
  129. mask = 3,4,5
  130. anchors = 10,13,  16,30,  33,23,  30,61,  62,45,  59,119,  116,90,  156,198,  373,326
  131. classes=3  #修改为自己的类别数
  132. num=9
  133. jitter=.3
  134. ignore_thresh = .5
  135. truth_thresh = 1
  136. random=1
  138. [route]
  139. layers = -4
  141. [convolutional]
  142. batch_normalize=1
  143. filters=128
  144. size=1
  145. stride=1
  146. pad=1
  147. activation=leaky
  149. [upsample]
  150. stride=2
  152. [route]
  153. layers = -1, 36
  157. [convolutional]
  158. batch_normalize=1
  159. filters=128
  160. size=1
  161. stride=1
  162. pad=1
  163. activation=leaky
  165. [convolutional]
  166. batch_normalize=1
  167. size=3
  168. stride=1
  169. pad=1
  170. filters=256
  171. activation=leaky
  173. [convolutional]
  174. batch_normalize=1
  175. filters=128
  176. size=1
  177. stride=1
  178. pad=1
  179. activation=leaky
  181. [convolutional]
  182. batch_normalize=1
  183. size=3
  184. stride=1
  185. pad=1
  186. filters=256
  187. activation=leaky
  189. [convolutional]
  190. batch_normalize=1
  191. filters=128
  192. size=1
  193. stride=1
  194. pad=1
  195. activation=leaky
  197. [convolutional]
  198. batch_normalize=1
  199. size=3
  200. stride=1
  201. pad=1
  202. filters=256
  203. activation=leaky
  205. [convolutional]
  206. size=1
  207. stride=1
  208. pad=1
  209. filters=24    #filters = 3 * ( classes + 5 )   here,filters=3*(3+5)
  210. activation=linear
  212. [yolo]
  213. mask = 0,1,2
  214. anchors = 10,13,  16,30,  33,23,  30,61,  62,45,  59,119,  116,90,  156,198,  373,326
  215. classes=3   #修改为自己的类别数
  216. num=9
  217. jitter=.3
  218. ignore_thresh = .5
  219. truth_thresh = 1
  220. random=1

训练自己的模型

1 单GPU训练：./darknet -i <gpu_id> detector train <data_cfg> <train_cfg> <weights>

  1. ./darknet detector train cfg/voc.data cfg/yolov3-voc.cfg darknet53.conv.74

2 多GPU训练，格式为0,1,2,3：./darknet detector train <data_cfg> <model_cfg> <weights> -gpus <gpu_list>

  1. ./darknet detector train cfg/voc.data cfg/yolov3-voc.cfg darknet53.conv.74 -gpus 0,1,2,3

测试Yolo模型

测试单张图片：

  • 测试单张图片，需要编译时有OpenCV支持：./darknet detector test <data_cfg> <test_cfg> <weights> <image_file> #本次测试无opencv支持
  • <test_cfg>文件中batch和subdivisions两项必须为1。
  • 测试时还可以用-thresh和-hier选项指定对应参数。

  • ./darknet detector test cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc_20000.weights Eminem.jpg

    批量测试图片

    • yolov3-voc.cfg(cfg文件夹下)文件中batch和subdivisions两项必须为1。

    • 在detector.c中增加头文件：

      1. #include <unistd.h>  /* Many POSIX functions (but not all, by a large margin) */
      2. #include <fcntl.h>   /* open(), creat() - and fcntl() */

  • 在前面添加GetFilename(char p)函数

    1. #include "darknet.h"
    2. #include <sys/stat.h>  //需增加的头文件
    3. #include<stdio.h>
    4. #include<time.h>
    5. #include<sys/types.h>  //需增加的头文件
    6. static int coco_ids[] = {1,2,3,4,5,6,7,8,9,10,11,13,14,15,16,17,18,19,20,21,22,23,24,25,27,28,31,32,33,34,35,36,37,38,39,40,41,42,43,44,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,67,70,72,73,74,75,76,77,78,79,80,81,82,84,85,86,87,88,89,90};
    8. char *GetFilename(char *p)
    9. { 
    10.     static char name[30]={""};
    11.     char *q = strrchr(p,'/') + 1;
    12.     strncpy(name,q,20);
    13.     return name;
    14. }

  • 用下面代码替换detector.c文件（example文件夹下）的void test_detector函数（注意有3处要改成自己的路径）

    void test_detector(char datacfg, char cfgfile, char weightfile, char filename, float thresh, float hier_thresh, char *outfile, int fullscreen)
    {

    1. list *options = read_data_cfg(datacfg);
    2. char *name_list = option_find_str(options, "names", "data/names.list");
    3. char **names = get_labels(name_list);
    5. image **alphabet = load_alphabet();
    6. network *net = load_network(cfgfile, weightfile, 0);
    7. set_batch_network(net, 1);
    8. srand(2222222);
    9. double time;
    10. char buff[256];
    11. char *input = buff;
    12. float nms=.45;
    13. int i=0;
    14. while(1){
    15.     if(filename){
    16.         strncpy(input, filename, 256);
    17.         image im = load_image_color(input,0,0);
    18.         image sized = letterbox_image(im, net->w, net->h);
    19.     //image sized = resize_image(im, net->w, net->h);
    20.     //image sized2 = resize_max(im, net->w);
    21.     //image sized = crop_image(sized2, -((net->w - sized2.w)/2), -((net->h - sized2.h)/2), net->w, net->h);
    22.     //resize_network(net, sized.w, sized.h);
    23.         layer l = net->layers[net->n-1];
  1.             float *X = sized.data;
  2.             time=what_time_is_it_now();
  3.             network_predict(net, X);
  4.             printf("%s: Predicted in %f seconds.\n", input, what_time_is_it_now()-time);
  5.             int nboxes = 0;
  6.             detection *dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, 0, 1, &nboxes);
  7.             //printf("%d\n", nboxes);
  8.             //if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms);
  9.             if (nms) do_nms_sort(dets, nboxes, l.classes, nms);
  10.                 draw_detections(im, dets, nboxes, thresh, names, alphabet, l.classes);
  11.                 free_detections(dets, nboxes);
  12.             if(outfile)
  13.              {
  14.                 save_image(im, outfile);
  15.              }
  16.             else{
  17.                 save_image(im, "predictions");
  18. #ifdef OPENCV
  19.                 cvNamedWindow("predictions", CV_WINDOW_NORMAL); 
  20.                 if(fullscreen){
  21.                 cvSetWindowProperty("predictions", CV_WND_PROP_FULLSCREEN, CV_WINDOW_FULLSCREEN);
  22.                 }
  23.                 show_image(im, "predictions",0);
  24.                 cvWaitKey(0);
  25.                 cvDestroyAllWindows();
  26. #endif
  27.             }
  28.             free_image(im);
  29.             free_image(sized);
  30.             if (filename) break;
  31.          } 
  32.         else {
  33.             printf("Enter Image Path: ");
  34.             fflush(stdout);
  35.             input = fgets(input, 256, stdin);
  36.             if(!input) return;
  37.             strtok(input, "\n");
  39.             list *plist = get_paths(input);
  40.             char **paths = (char **)list_to_array(plist);
  41.              printf("Start Testing!\n");
  42.             int m = plist->size;
  43.             if(access("/home/learner/darknet/data/outv3tiny_dpj",0)==-1)//"/home/learner/darknet/data"修改成自己的路径
  44.             {
  45.               if (mkdir("/home/learner/darknet/data/outv3tiny_dpj",0777))//"/home/learner/darknet/data"修改成自己的路径
  46.                {
  47.                  printf("creat file bag failed!!!");
  48.                }
  49.             }
  50.             for(i = 0; i < m; ++i){
  51.              char *path = paths[i];
  52.              image im = load_image_color(path,0,0);
  53.              image sized = letterbox_image(im, net->w, net->h);
  54.         //image sized = resize_image(im, net->w, net->h);
  55.         //image sized2 = resize_max(im, net->w);
  56.         //image sized = crop_image(sized2, -((net->w - sized2.w)/2), -((net->h - sized2.h)/2), net->w, net->h);
  57.         //resize_network(net, sized.w, sized.h);
  58.         layer l = net->layers[net->n-1];
  61.         float *X = sized.data;
  62.         time=what_time_is_it_now();
  63.         network_predict(net, X);
  64.         printf("Try Very Hard:");
  65.         printf("%s: Predicted in %f seconds.\n", path, what_time_is_it_now()-time);
  66.         int nboxes = 0;
  67.         detection *dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, 0, 1, &nboxes);
  68.         //printf("%d\n", nboxes);
  69.         //if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms);
  70.         if (nms) do_nms_sort(dets, nboxes, l.classes, nms);
  71.         draw_detections(im, dets, nboxes, thresh, names, alphabet, l.classes);
  72.         free_detections(dets, nboxes);
  73.         if(outfile){
  74.             save_image(im, outfile);
  75.         }
  76.         else{
  78.              char b[2048];
  79.             sprintf(b,"/home/learner/darknet/data/outv3tiny_dpj/%s",GetFilename(path));//"/home/leaner/darknet/data"修改成自己的路径
  81.             save_image(im, b);
  82.             printf("save %s successfully!\n",GetFilename(path));
  83. /*
  84. #ifdef OPENCV
  85.             //cvNamedWindow("predictions", CV_WINDOW_NORMAL); 
  86.             if(fullscreen){
  87.                 cvSetWindowProperty("predictions", CV_WND_PROP_FULLSCREEN, CV_WINDOW_FULLSCREEN);
  88.             }
  89.             //show_image(im, "predictions");
  90.             //cvWaitKey(0);
  91.             //cvDestroyAllWindows();
  92. #endif*/
  93.         }
  95.         free_image(im);
  96.         free_image(sized);
  97.         if (filename) break;
  98.         }
  99.       }
  100.     }
  101. }

  • 重新进行编译

    make clean
    make

  • 开始批量测试

    ./darknet detector test cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc_20000.weights

  • 输入Image Path(所有的测试文件的路径，可以复制voc.data中valid后边的路径):

    /home/learner/darknet/data/voc/2007_test.txt # 完整路径

  • 结果都保存在./data/out文件夹下

生成预测结果

生成预测结果

  • ./darknet detector valid <data_cfg> <test_cfg> <weights> <out_file>
  • yolov3-voc.cfg(cfg文件夹下)文件中batch和subdivisions两项必须为1。
  • 结果生成在<data_cfg>的results指定的目录下以<out_file>开头的若干文件中，若<data_cfg>没有指定results，那么默认为<darknet_root>/results。

  • 执行语句如下：在终端只返回用时，在./results/comp4_det_test_[类名].txt里保存测试结果

    ./darknet detector valid cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc_20000.weights

采用第三方compute_mAP

下载第三方库：

  1. git clone https://github.com/LianjiLi/yolo-compute-map.git

进行如下修改:

  • 修改darknet/examples/detector.c中validate_detector()

    1. char *valid_images = option_find_str(options, "valid", "./data/2007_test.txt");//改成自己的测试文件路径
    3. if(!outfile) outfile = "comp4_det_test_";
    4.         fps = calloc(classes, sizeof(FILE *));
    5.         for(j = 0; j < classes; ++j){
    6.             snprintf(buff, 1024, "%s/%s.txt", prefix, names[j]);//删除outfile参数以及对应的%s
    7.             fps[j] = fopen(buff, "w");

  • 重新编译

    1. make clean
    2. make

  • 运行valid

    1. darknet文件夹下运行./darknet detector valid cfg/voc.data cfg/yolov3-tiny.cfg backup/yolov3-tiny_164000.weights（改为自己的模型路径）
  • 在本文件夹下运行python compute_mAP.py
  • 说明：compute_mAP.py中的test.txt文件内容只有文件名字，不带绝对路径，不带后缀

Reference

YOLOv3目标检测总结

官方网站

思路整理自@zhaonan

代码改变世界

分类: 深度学习 专栏

标签: 经验, yolov3, 配置, map, test

好文要顶 关注我 收藏该文 icon_weibo_24.png wechat.png

20170903155022.png

pprp
关注 - 69
粉丝 - 21

+加关注

1

0

« 上一篇：CycleGAN 配置及其实现
» 下一篇：(转载)YOLO配置文件理解

posted @ 2018-08-23 18:22 pprp 阅读(7246) 评论(0) 编辑 收藏

发表评论

表情：
评论列表 （有 0 条评论，514人围观）

还没有评论，来说两句吧...

相关阅读