fast-reid复现（Market1501）

比眉伴天荒 2023-02-16 15:29 1阅读 0赞

由于fast-reid变动太大，这里将我6月份下载的版本放到这里，也是这篇博客使用的版本：[https://download.csdn.net/download/qq\_35975447/12760996][https_download.csdn.net_download_qq_35975447_12760996]。

[https://github.com/JDAI-CV/fast-reid][https_github.com_JDAI-CV_fast-reid]

## 0.环境 ##

ubuntu16.04
    cuda9.0
    python3.6
    torch==1.1.0
    torchvision==0.3.0
    Cython
    yacs
    tensorboard
    future
    termcolor
    sklearn
    tqdm
    opencv-python==4.1.0.25
    matplotlib
    scikit-image
    numpy==1.16.4

pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt

## 1.下载与环境准备 ##

首先下载[face-reid-master][https_github.com_JDAI-CV_fast-reid]，[数据集Market-1501][Market-1501]。

下载的数据集，解压后放到datasets目录下：

![202006121311036.png][]

## 2.运行train\_net（Market1501-AGW\_R50.yml） ##

### 2.1 训练 ###

CUDA_VISIBLE_DEVICES=0 python ./tools/train_net.py --config-file='./configs/Market1501/AGW_R50.yml'

此处也使用了预训练模型。

出现错误：

urllib.error.URLError: <urlopen error \[SSL: CERTIFICATE\_VERIFY\_FAILED\] certificate verify failed (\_ssl.c:852)>

![20200611214625198.png][]

解决：

vim ./fastreid/modeling/backbones/resnet.py

插入下面两句话在模块引用部分：

import ssl
    ssl._create_default_https_context = ssl._create_unverified_context

成功后：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 1][]

### 2.2 测试 ###

CUDA_VISIBLE_DEVICES=0 python tools/train_net.py --config-file "logs/market1501/agw_R50/config.yaml" --eval-only  MODEL.WEIGHTS "logs/market1501/agw_R50/model_final.pth"

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 2][]

和作者的结果(下图)，还是有一定差距：

![20200612174826424.png][]

### 2.3 可视化 ###

CUDA_VISIBLE_DEVICES=0 python ./demo/visualize_result.py --config-file './configs/Market1501/AGW_R50.yml' --vis-label --dataset-name 'Market1501' --output 'logs/market1501/agw_R50/agw_market1501_vis' --opts MODEL.WEIGHTS "./logs/market1501/agw_R50/model_final.pth"

![20200615113318709.png][] 遇到问题：

Traceback (most recent call last):
      File "./demo/visualize_result.py", line 139, in <module>
        args.rank_sort, args.label_sort, args.max_rank)
      File "./fastreid/utils/visualizer.py", line 157, in vis_rank_list
        self.save_rank_result(query_indices, output, max_rank, vis_label, label_sort, actmap)
      File "./fastreid/utils/visualizer.py", line 69, in save_rank_result
        ax.set_title('{}/{:.2f}/cam{}'.format(query_name, self.all_ap[q_idx], cam_id))
    IndexError: invalid index to scalar variable.

（回避式）解决：

注释掉"fast-reid-master/fastreid/utils/visualizer.py"中的line69。

warning:

./fastreid/evaluation/rank.py:14: UserWarning: Cython evaluation (very fast so highly recommended) is unavailable, now use python evaluation.
      'Cython evaluation (very fast so highly recommended) is '

解决(./fastreid/evaluation/rank.py)：

注释掉：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 3][]

结果示例：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 4][]

## 3.运行train\_net（Market1501-AGW\_R101-ibn.yml） ##

### 3.1 训练 ###

CUDA_VISIBLE_DEVICES=0 python ./tools/train_net.py --config-file='./configs/Market1501/AGW_R101-ibn.yml'

出现错误：

FileNotFoundError: \[Errno 2\] No such file or directory: '/export/home/lxy/.cache/torch/checkpoints/resnet101\_ibn\_a.pth.tar'

解决----修改config中路径：

vim ./configs/Market1501/AGW_R101-ibn.yml

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 5][]

下载[resnet101\_ibn\_a.pth.tar][resnet101_ibn_a.pth.tar]，(有的预训练模型在[这里][Link 1])，然后将其放到/home/\*\*\*/.cache/torch/checkpoints'路径下。

然后再运行：

CUDA_VISIBLE_DEVICES=0 python ./tools/train_net.py --config-file='./configs/Market1501/AGW_R101-ibn.yml'

![20200612111517304.png][]

### 3.2 测试  ###

CUDA_VISIBLE_DEVICES=0 python tools/train_net.py --config-file "logs/market1501/agw_R101-ibn/config.yaml" --eval-only  MODEL.WEIGHTS "logs/market1501/agw_R101-ibn/model_final.pth"

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 6][]

![20200612175536518.png][]

### 3.3 可视化 ###

CUDA_VISIBLE_DEVICES=0 python ./demo/visualize_result.py --config-file './configs/Market1501/AGW_R101-ibn.yml' --vis-label --dataset-name 'Market1501' --output 'logs/market1501/agw_R101-ibn/agw_market1501_vis' --opts MODEL.WEIGHTS "./logs/market1501/agw_R101-ibn/model_final.pth"

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 7][]

### 3.4 demo.py ###

CUDA_VISIBLE_DEVICES=0 python ./demo/demo.py --config-file './configs/Market1501/AGW_R50.yml' --input '/*/*/fast-reid-master/demo/1502_c1s4_065128_00.jpg' --output '/*/*/fast-reid-master/logs/market1501/agw_R50/agw_market1501_vis/demo_output/' --opts MODEL.WEIGHTS "/*/*/fast-reid-master/logs/market1501/agw_R50/model_final.pth"

## 4.caffe部署 ##

参考：[https://github.com/JDAI-CV/fast-reid/tree/master/tools/deploy][https_github.com_JDAI-CV_fast-reid_tree_master_tools_deploy]

做以下修改：

### 4.1 修改preprocess\_image ###

![20200721112130106.png][]

def preprocess_image(self, batched_inputs):
        """
        Normalize and batch the input images.
        """
        # images = [x["images"] for x in batched_inputs]
        # images = batched_inputs["images"]
        images = batched_inputs
        images.sub_(self.pixel_mean).div_(self.pixel_std)
        return images

即注释line 71，取消注释line 72。

### 4.2 修改与运行run\_export.sh ###

要想在caffe下部署,这里只能用bagtricks\_R50 !!!!!这是痛的领悟~.~重新训练bagtricks\_R50模型吧.

修改run\_export.sh：

python caffe_export.py --config-file "/opt/faceReID/fast-reid-master/logs/market1501/bagtricks_R50/config.yaml" \
    --name "baseline_R50" \
    --output "/opt/faceReID/fast-reid-master/logs/market1501/bagtricks_R50/caffe_model" \
    --opts MODEL.WEIGHTS "/opt/faceReID/fast-reid-master/logs/market1501/bagtricks_R50/model_final.pth"

--config-file：训练时输出的模型的目录下有个config.yaml文件路径；

--name：输出的caffe文件命令；

--output：想要将转化后的caffe输出路径，可以写成绝对路径，尤其是在docker虚拟环境中；

--opts：训练时保存的最后一个模型。

进入/tools/deploy下运行：

sh ./run_export.sh

### 4.3 改变prorotxt文件中的三个部分 ###

（1）这个比较简单，注释掉ceil\_mode:false就可以了

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 8][]

（2） 找到bottom: "relu\_blob49"，将这层放到其之前，再将下一层的bottom内容换成avgpool\_blob1。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 9][]

（3） 这个较为简单，修改top为output就可以了。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 10][]

###  4.4 安装caffe ###

由于内容比较多，所以写在另外一篇：[https://blog.csdn.net/qq\_35975447/article/details/107507434][https_blog.csdn.net_qq_35975447_article_details_107507434]

###  4.5 在python中推理 ###

安装：scikit-image（通过pip安装就行）

在tools/deploy目录下修改：

# 在caffe_inference.py最前面加上：
    import sys
    caffe_root = "build caffe path"
    sys.path.insert(0, caffe_root + "python")
    import caffe

在tools/deploy目录下运行：

sh ./run_inference.sh

显示结果：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 11][]

## 参考 ##

[1.fast-reid(JDAI-CV)][https_github.com_JDAI-CV_fast-reid]

[2.FastReID: 一个面向学术界和工业界的 ReID Toolbox(官方介绍博客)][2.FastReID_ _ ReID Toolbox]

[3.python3中使用urlopen()报错的解决方法][3.python3_urlopen]

[4.(caffe)python 3.6编译libboost\_python3.so并编译cpu-caffe][https_blog.csdn.net_qq_35975447_article_details_107507434]

[https_download.csdn.net_download_qq_35975447_12760996]: https://download.csdn.net/download/qq_35975447/12760996
[https_github.com_JDAI-CV_fast-reid]: https://github.com/JDAI-CV/fast-reid
[Market-1501]: https://github.com/JDAI-CV/fast-reid/tree/master/datasets
[202006121311036.png]: https://img-blog.csdnimg.cn/202006121311036.png
[20200611214625198.png]: https://img-blog.csdnimg.cn/20200611214625198.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20200611215022740.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20200612101149876.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/2020061217473639.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[20200612174826424.png]: https://img-blog.csdnimg.cn/20200612174826424.png
[20200615113318709.png]: https://img-blog.csdnimg.cn/20200615113318709.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/20200612142848682.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 4]: https://img-blog.csdnimg.cn/20200615114454488.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 5]: https://img-blog.csdnimg.cn/20200612105042825.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[resnet101_ibn_a.pth.tar]: https://drive.google.com/file/d/11vx1h_ibJcHiPl8mX9B6TzUquX-LVmxn/view?usp=sharing
[Link 1]: https://drive.google.com/drive/folders/1thS2B8UOSBi_cJX6zRy6YYRwz_nVFI_S
[20200612111517304.png]: https://img-blog.csdnimg.cn/20200612111517304.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 6]: https://img-blog.csdnimg.cn/20200612175514234.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[20200612175536518.png]: https://img-blog.csdnimg.cn/20200612175536518.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 7]: https://img-blog.csdnimg.cn/20200615114700973.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[https_github.com_JDAI-CV_fast-reid_tree_master_tools_deploy]: https://github.com/JDAI-CV/fast-reid/tree/master/tools/deploy
[20200721112130106.png]: https://img-blog.csdnimg.cn/20200721112130106.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 8]: https://img-blog.csdnimg.cn/20200721113952659.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 9]: https://img-blog.csdnimg.cn/20200721114646592.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 10]: https://img-blog.csdnimg.cn/20200721114855858.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[https_blog.csdn.net_qq_35975447_article_details_107507434]: https://blog.csdn.net/qq_35975447/article/details/107507434
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70 11]: https://img-blog.csdnimg.cn/20200723093004411.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3,size_16,color_FFFFFF,t_70
[2.FastReID_ _ ReID Toolbox]: https://l1aoxingyu.github.io/blogpages/reid/2020/05/29/fastreid.html
[3.python3_urlopen]: https://blog.csdn.net/qq_25403205/article/details/81258327