Tensorflow Object Detection API介绍和应用(mac版)——（3.加载预训练模型训练自定义模型）

拼搏现实的明天。 2021-12-14 02:05 401阅读 0赞

### 1、前言 ###

本篇用到的pre-trained model是coco（上一篇提到的），下载几个图片文件作为训练数据，迭代次数10次，训练差不都5分钟。只是给个例子，本机小破mac怕爆炸。看google建议是每个label是100张图片（能500+最好）。本人实测，一个label样式不是很多的logo类图像，40或50个训练数据，200k迭代，感觉效果已经不错了。  
上一篇：[https://blog.csdn.net/bjjoy2009/article/details/94962333][https_blog.csdn.net_bjjoy2009_article_details_94962333]

### 2、安装cocoapi ###

打开终端，按顺序执行下面操作  
（1）我的本地cocoapi下载路经  
cd /Users/bjjoy  
（2）git clone [https://github.com/cocodataset/cocoapi.git][https_github.com_cocodataset_cocoapi.git]  
（3）cd cocoapi/PythonAPI  
（4）make  
（5）将编译好的文件夹放到第一篇下载的models对应路径下  
cp -r pycocotools /Users/bjjoy/JupyterProjects/models/research/

### 3、安装LabelImg ###

该项目就是标记图像，打标签用的(python3以上)，直接终端安装  
（1）pip install labelImg  
（2）终端输入：labelImg，打开下图  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2Jqam95MjAwOQ_size_16_color_FFFFFF_t_70]

### 4、构建测试数据 ###

（1）在model路径下（~/JupyterProjects/models/research/object\_detection）创建3个文件夹  
![在这里插入图片描述][20190707154005492.png]  
my\_train：存放训练模型  
my\_test\_data: 训练和测试图像数据  
my\_image：需要标注的图像数据（这里用了6个huawei的logo，4训练2测试）  
（2）用labelImg标记图像（这里标签就用的huawei）  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2Jqam95MjAwOQ_size_16_color_FFFFFF_t_70 1]  
左侧绿框 Open Dir：下载的图片存放路径（这里是my\_images）  
左侧红框Create\\nRecBox：标记图像（右侧绿色框）  
输入标签名：点击ok  
保存（command+s)：存储为xml文件，每个图像一个xml文件

### 5、训练数据xml转为csv ###

参考：[https://github.com/datitran/raccoon\_dataset][https_github.com_datitran_raccoon_dataset]  
（1）xml ->csv  
用pycharm创建了一个xml转csv的项目，结构如下图  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2Jqam95MjAwOQ_size_16_color_FFFFFF_t_70 2]  
将labelImg生成的xml文件放到绿色框对应文件夹中  
红色框xml\_to\_csv.py程序如下，运行后csv文件放到data目录下

import os
    import glob
    import pandas as pd
    import xml.etree.ElementTree as ET
    
    def xml_to_csv(path):
        xml_list = []
        for xml_file in glob.glob(path + '/*.xml'):
            tree = ET.parse(xml_file)
            root = tree.getroot()
            for member in root.findall('object'):
                value = (root.find('filename').text,
                         int(root.find('size')[0].text),
                         int(root.find('size')[1].text),
                         member[0].text,
                         int(member[4][0].text),
                         int(member[4][1].text),
                         int(member[4][2].text),
                         int(member[4][3].text)
                         )
                xml_list.append(value)
        column_name = ['filename', 'width', 'height', 'class', 'xmin', 'ymin', 'xmax', 'ymax']
        xml_df = pd.DataFrame(xml_list, columns=column_name)
        return xml_df
    
    def main2():
        for directory in ['train', 'test']:
            image_path = os.path.join(os.getcwd(), '{}'.format(directory))
            xml_df = xml_to_csv(image_path)
            xml_df.to_csv('data/{}_labels.csv'.format(directory), index=None)
            print('Successfully converted xml to csv.')
    
    main2()

### 6、csv转为tfrecord（训练识别的文件类型） ###

（1）将上面的csv文件搬到models/research/object\_detection/my\_test\_data  
（2）在models/research/object\_detection 创建一个generate\_tfrecord.py，代码如下

from __future__ import division
    from __future__ import print_function
    from __future__ import absolute_import
    
    import os
    import io
    import pandas as pd
    import tensorflow as tf
    
    from PIL import Image
    from object_detection.utils import dataset_util
    from collections import namedtuple, OrderedDict
    
    flags = tf.app.flags
    flags.DEFINE_string('csv_input', '', 'Path to the CSV input')
    flags.DEFINE_string('output_path', '', 'Path to output TFRecord')
    flags.DEFINE_string('image_dir', '', 'Path to images')
    FLAGS = flags.FLAGS
    
    
    # TO-DO replace this with label map
    def class_text_to_int(row_label):
        if row_label == 'huawei':
            return 1
        else:
            None
    
    
    def split(df, group):
        data = namedtuple('data', ['filename', 'object'])
        gb = df.groupby(group)
        return [data(filename, gb.get_group(x)) for filename, x in zip(gb.groups.keys(), gb.groups)]
    
    
    def create_tf_example(group, path):
        with tf.gfile.GFile(os.path.join(path, '{}'.format(group.filename)), 'rb') as fid:
            encoded_jpg = fid.read()
        encoded_jpg_io = io.BytesIO(encoded_jpg)
        image = Image.open(encoded_jpg_io)
        width, height = image.size
    
        filename = group.filename.encode('utf8')
        image_format = b'jpg'
        xmins = []
        xmaxs = []
        ymins = []
        ymaxs = []
        classes_text = []
        classes = []
    
        for index, row in group.object.iterrows():
            xmins.append(row['xmin'] / width)
            xmaxs.append(row['xmax'] / width)
            ymins.append(row['ymin'] / height)
            ymaxs.append(row['ymax'] / height)
            classes_text.append(row['class'].encode('utf8'))
            classes.append(class_text_to_int(row['class']))
    
        tf_example = tf.train.Example(features=tf.train.Features(feature={ 
            'image/height': dataset_util.int64_feature(height),
            'image/width': dataset_util.int64_feature(width),
            'image/filename': dataset_util.bytes_feature(filename),
            'image/source_id': dataset_util.bytes_feature(filename),
            'image/encoded': dataset_util.bytes_feature(encoded_jpg),
            'image/format': dataset_util.bytes_feature(image_format),
            'image/object/bbox/xmin': dataset_util.float_list_feature(xmins),
            'image/object/bbox/xmax': dataset_util.float_list_feature(xmaxs),
            'image/object/bbox/ymin': dataset_util.float_list_feature(ymins),
            'image/object/bbox/ymax': dataset_util.float_list_feature(ymaxs),
            'image/object/class/text': dataset_util.bytes_list_feature(classes_text),
            'image/object/class/label': dataset_util.int64_list_feature(classes),
        }))
        return tf_example
    
    
    def main(_):
        writer = tf.python_io.TFRecordWriter(FLAGS.output_path)
        path = os.path.join(FLAGS.image_dir)
        examples = pd.read_csv(FLAGS.csv_input)
        grouped = split(examples, 'filename')
        for group in grouped:
            tf_example = create_tf_example(group, path)
            writer.write(tf_example.SerializeToString())
    
        writer.close()
        output_path = os.path.join(os.getcwd(), FLAGS.output_path)
        print('Successfully created the TFRecords: {}'.format(output_path))
    
    
    if __name__ == '__main__':
        tf.app.run()

（3）生成tfrecord  
i）打开终端，cd进入到models/research  
ii) 切换python项目路径：export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim  
iii)cd进入models/research/object\_detection，执行下面2条命令：

python generate\_tfrecord.py --csv\_input=my\_test\_data/test\_labels.csv --output\_path=my\_test\_data/test.record --image\_dir=my\_images/

python generate\_tfrecord.py --csv\_input=my\_test\_data/train\_labels.csv --output\_path=my\_test\_data/train.record --image\_dir=my\_images/

### 7、训练 ###

#### 7.1 生成标签文件 ####

在my\_test\_data文件夹创建huawei\_detetion.pbtxt，内容如下

item \{  
id: 1  
name: ‘huawei’  
\}

#### 7.2 解压coco模型内文件 ####

此处用的版本是ssd\_mobilenet\_v1\_coco\_2017\_11\_17.tar.gz，直接解压到object\_detection文件夹下即可。

#### 7.3 配置config ####

（1）在models/research/object\_detection/samples/configs，找到ssd\_mobilenet\_v1\_coco.config，复制到my\_test\_data中  
（2）修改config文件，主要修改如下  
迭代次数，num\_steps: 10  
减小 batch\_size: 24  
文件最下面，所有路径相关参数，例如 fine\_tune\_checkpoint

# SSD with Mobilenet v1 configuration for MSCOCO Dataset.
    # Users should configure the fine_tune_checkpoint field in the train config as
    # well as the label_map_path and input_path fields in the train_input_reader and
    # eval_input_reader. Search for "PATH_TO_BE_CONFIGURED" to find the fields that
    # should be configured.
    
    model { 
      ssd { 
        num_classes: 1
        box_coder { 
          faster_rcnn_box_coder { 
            y_scale: 10.0
            x_scale: 10.0
            height_scale: 5.0
            width_scale: 5.0
          }
        }
        matcher { 
          argmax_matcher { 
            matched_threshold: 0.5
            unmatched_threshold: 0.5
            ignore_thresholds: false
            negatives_lower_than_unmatched: true
            force_match_for_each_row: true
          }
        }
        similarity_calculator { 
          iou_similarity { 
          }
        }
        anchor_generator { 
          ssd_anchor_generator { 
            num_layers: 6
            min_scale: 0.2
            max_scale: 0.95
            aspect_ratios: 1.0
            aspect_ratios: 2.0
            aspect_ratios: 0.5
            aspect_ratios: 3.0
            aspect_ratios: 0.3333
          }
        }
        image_resizer { 
          fixed_shape_resizer { 
            height: 300
            width: 300
          }
        }
        box_predictor { 
          convolutional_box_predictor { 
            min_depth: 0
            max_depth: 0
            num_layers_before_predictor: 0
            use_dropout: false
            dropout_keep_probability: 0.8
            kernel_size: 1
            box_code_size: 4
            apply_sigmoid_to_scores: false
            conv_hyperparams { 
              activation: RELU_6,
              regularizer { 
                l2_regularizer { 
                  weight: 0.00004
                }
              }
              initializer { 
                truncated_normal_initializer { 
                  stddev: 0.03
                  mean: 0.0
                }
              }
              batch_norm { 
                train: true,
                scale: true,
                center: true,
                decay: 0.9997,
                epsilon: 0.001,
              }
            }
          }
        }
        feature_extractor { 
          type: 'ssd_mobilenet_v1'
          min_depth: 16
          depth_multiplier: 1.0
          conv_hyperparams { 
            activation: RELU_6,
            regularizer { 
              l2_regularizer { 
                weight: 0.00004
              }
            }
            initializer { 
              truncated_normal_initializer { 
                stddev: 0.03
                mean: 0.0
              }
            }
            batch_norm { 
              train: true,
              scale: true,
              center: true,
              decay: 0.9997,
              epsilon: 0.001,
            }
          }
        }
        loss { 
          classification_loss { 
            weighted_sigmoid { 
            }
          }
          localization_loss { 
            weighted_smooth_l1 { 
            }
          }
          hard_example_miner { 
            num_hard_examples: 3000
            iou_threshold: 0.99
            loss_type: CLASSIFICATION
            max_negatives_per_positive: 3
            min_negatives_per_image: 0
          }
          classification_weight: 1.0
          localization_weight: 1.0
        }
        normalize_loss_by_num_matches: true
        post_processing { 
          batch_non_max_suppression { 
            score_threshold: 1e-8
            iou_threshold: 0.6
            max_detections_per_class: 100
            max_total_detections: 100
          }
          score_converter: SIGMOID
        }
      }
    }
    
    train_config: { 
      batch_size: 24
      optimizer { 
        rms_prop_optimizer: { 
          learning_rate: { 
            exponential_decay_learning_rate { 
              initial_learning_rate: 0.004
              decay_steps: 800720
              decay_factor: 0.95
            }
          }
          momentum_optimizer_value: 0.9
          decay: 0.9
          epsilon: 1.0
        }
      }
      fine_tune_checkpoint: "object_detection/ssd_mobilenet_v1_coco_2017_11_17/model.ckpt"
      from_detection_checkpoint: true
      # Note: The below line limits the training process to 200K steps, which we
      # empirically found to be sufficient enough to train the pets dataset. This
      # effectively bypasses the learning rate schedule (the learning rate will
      # never decay). Remove the below line to train indefinitely.
      num_steps: 10
      data_augmentation_options { 
        random_horizontal_flip { 
        }
      }
      data_augmentation_options { 
        ssd_random_crop { 
        }
      }
    }
    
    train_input_reader: { 
      tf_record_input_reader { 
        input_path: "object_detection/my_test_data/train.record"
      }
      label_map_path: "object_detection/my_test_data/huawei_detetion.pbtxt"
    }
    
    eval_config: { 
      num_examples: 12
      # Note: The below line limits the evaluation process to 10 evaluations.
      # Remove the below line to evaluate indefinitely.
      max_evals: 10
    }
    
    eval_input_reader: { 
      tf_record_input_reader { 
        input_path: "object_detection/my_test_data/test.record"
      }
      label_map_path: "object_detection/my_test_data/huawei_detetion.pbtxt"
      shuffle: false
      num_readers: 1
    }

### 8、训练 ###

终端执行，等几分钟  
python object\_detection/model\_main.py --model\_dir=object\_detection/my\_train/ --pipeline\_config\_path=object\_detection/my\_test\_data/ssd\_mobilenet\_v1\_coco.config

### 9、tensorboard查看结果 ###

tensorboard --logdir=‘object\_detection/my\_train’

### 10、object\_detection\_tutorial.ipynb测试训练好的模型 ###

（1）模型转为pb文件  
python object\_detection/export\_inference\_graph.py --input\_type image\_tensor --pipeline\_config\_path object\_detection/my\_test\_data/ssd\_mobilenet\_v1\_coco.config --trained\_checkpoint\_prefix object\_detection/my\_train/model.ckpt-10 --output\_directory object\_detection/my\_test\_data/frozen\_inference\_graph  
（2）object\_detection\_tutorial.ipynb运行  
修改模型路径：

PATH_TO_FROZEN_GRAPH = 'my_test_data' + '/frozen_inference_graph.pb'
    PATH_TO_LABELS = os.path.join('my_test_data', 'huawei_detetion.pbtxt')

Download Model模块代码全部注释掉

# opener = urllib.request.URLopener()
    # opener.retrieve(DOWNLOAD_BASE + MODEL_FILE, MODEL_FILE)
    # tar_file = tarfile.open(MODEL_FILE)
    # for file in tar_file.getmembers():
    # file_name = os.path.basename(file.name)
    # if 'frozen_inference_graph.pb' in file_name:
    # tar_file.extract(file, os.getcwd())

测试图片参数

TEST_IMAGE_PATHS = ['my_images/hw8.jpg']

（3）运行结果，及其不准  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2Jqam95MjAwOQ_size_16_color_FFFFFF_t_70 3]

### 11、总结 ###

写的有点长，中间也不知道哪里会有问题了，有些是根据回忆。只是没想到原来想用现成的项目原来也这么麻烦啊。

[https_blog.csdn.net_bjjoy2009_article_details_94962333]: https://blog.csdn.net/bjjoy2009/article/details/94962333
[https_github.com_cocodataset_cocoapi.git]: https://github.com/cocodataset/cocoapi.git
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2Jqam95MjAwOQ_size_16_color_FFFFFF_t_70]: /images/20211213/b87cde0a50e745a185e826bd804bee17.png
[20190707154005492.png]: /images/20211213/bea48d0cb6544a7a94822abd1d1313c0.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2Jqam95MjAwOQ_size_16_color_FFFFFF_t_70 1]: /images/20211213/9d3accffe84043e78b0ad5ee43ac1da3.png
[https_github.com_datitran_raccoon_dataset]: https://github.com/datitran/raccoon_dataset
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2Jqam95MjAwOQ_size_16_color_FFFFFF_t_70 2]: /images/20211213/61c20839041c4b8bb5b136bad48a82db.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2Jqam95MjAwOQ_size_16_color_FFFFFF_t_70 3]: /images/20211213/03e9c596abe4439b86175fb1c1f643fa.png