人脸对齐：ICCV2017 How far are we from solving the 2D & 3D Face Alignment problem

浅浅的花香味﹌ 2021-11-16 15:22 245阅读 0赞

ICCV2017  
  
[https://www.adrianbulat.com/face-alignment][https_www.adrianbulat.com_face-alignment]  
  
Pytorch Code: [https://github.com/1adrianb/face-alignment][https_github.com_1adrianb_face-alignment]  
  
Torch7 Code: [https://github.com/1adrianb/2D-and-3D-face-alignment][https_github.com_1adrianb_2D-and-3D-face-alignment]

本文深入探讨了人脸对齐问题，文章题目起的很霸气啊！

facial landmark localization 也就是 face alignment

1 Introduction  
cascaded regression methods 在人脸对齐上取得不错的效果，但是当存在 large (and unfamiliar) facial poses（也就是一部分特征点 self-occluded landmarks or large in-plane rotations）cascaded regression methods 效果就不太好。近年来 fully Convolutional Neural Network architectures based on heatmap regression have revolutionized human pose estimation，于是沿着这个思路来做人脸对齐。

本文主要有以下几个 contributions:  
1) 针对人脸对齐，我们构建了一个很好的基准：通过结合一个最好的网络框架和一个最好的残差模块，在一个很大的2D数据库上训练，然后再其他2D数据库上测试（~230000张图像），分析我们离解决人脸对齐还有多远  
2)考虑到 3D 人脸对齐数据库很少，我们训练一个CNN将 2D 标记转为3D，基于此建立一个新的数据库 LS3D-W，得到一个最大的3D facial landmark dataset（~230000张图像）  
3） 基于 LS3D-W，我们训练了一个 3D 人脸对齐网络，并在这个数据库上评估了 3D 人脸对齐  
4）我们深入分析了影响人脸对齐的各个因素，并引入了一个新的因素：网络规模 the size of the network  
5） 我们发现不管是2D 人脸对齐网络还是3D 人脸对齐网络 在目前的数据库上性能都不错，可能接近目前数据库性能上的饱和。

2 Closely related work  
2D face alignment： 这里主要使用的是 cascaded regression 方法，基本解决可控人脸姿态的数据库 LFPW \[2\], Helen \[22\] and 300-W \[30\]

CNNs for face alignment：cascade CNN；multi-task learning；recurrent neural networks ；near-frontal faces of 300-W \[30\]  
large pose and 3D face alignment

Transferring landmark annotations 数据库的迁移学习

3 Datasets  
当前 2D 3D 人脸对齐数据库的一些情况  
![这里写图片描述][SouthEast]

3.3. Metrics  
一般使用的度量方法是 the metric used for face alignment is the point-to-point Euclidean distance normalized by the interocular distance  
这里我们改进了一下度量方式:normalize by the bounding box size. In particular, we used the Normalized Mean Error

4 Method  
4.1. 2D and 3D Face Alignment Networks  
Face Alignment Network (FAN) 基于 Hour-Glass (HG) network of \[23\]

![这里写图片描述][SouthEast 1]

we used 300W-LP-2D and 300W-LP-3D to train 2D-FAN and 3D-FAN

4.2. 2D-to-3D Face Alignment Network  
将2D 标记数据转为 3D 标记数据

4.3. Training 这要介绍了各个网络的训练

下面的网络的性能评估  
5 2D face alignment  
![这里写图片描述][SouthEast 2]

![这里写图片描述][SouthEast 3]

![这里写图片描述][SouthEast 4]

Conclusion： 2D-FAN achieves near saturating performance on the above 2D datasets

6 Large Scale 3D Faces in-the-Wild dataset  
2D-to-3D FAN  
![这里写图片描述][SouthEast 5]

2D 到3D 的转换引入一定的误差  
![这里写图片描述][SouthEast 6]

7 3D face alignment

![这里写图片描述][SouthEast 7]

Facial pose is not a major issue for 3D-FAN  
![这里写图片描述][SouthEast 8]

Resolution is not a major issue for 3D-FAN  
![这里写图片描述][SouthEast 9]

Initialization is not a major issue for 3D-FAN  
![这里写图片描述][SouthEast 10]

There is a moderate performance drop vs the number of parameters of 3D-FAN  
![这里写图片描述][SouthEast 11]

最后的结论是： 模型对于目前的数据基本已经达到性能饱和，对于一些不常见的姿态可以通过增加训练数据来提升网络的性能

[https_www.adrianbulat.com_face-alignment]: https://www.adrianbulat.com/face-alignment
[https_github.com_1adrianb_face-alignment]: https://github.com/1adrianb/face-alignment
[https_github.com_1adrianb_2D-and-3D-face-alignment]: https://github.com/1adrianb/2D-and-3D-face-alignment
[SouthEast]: /images/20211116/7df9b3d1c1ed448c98a166f999d8d76e.png
[SouthEast 1]: /images/20211116/cb5a6f1c56284e1e82ae227d6c835ebb.png
[SouthEast 2]: /images/20211116/941c69064fec43bebe5fb2153fa6dbb2.png
[SouthEast 3]: /images/20211116/d60caf15ea5f41bca6b2dbb11a62072f.png
[SouthEast 4]: /images/20211116/231016c30d7541e38d04d3b82b48eb42.png
[SouthEast 5]: /images/20211116/7b3f41364d174ce5b3c314b3eb5d67c1.png
[SouthEast 6]: /images/20211116/2e8fc6a859a7493998294f8828c191f3.png
[SouthEast 7]: /images/20211116/59ee537e0f6d491f9c8b769eec7eb084.png
[SouthEast 8]: /images/20211116/fccbb90144874bab9a663c0659f491c9.png
[SouthEast 9]: /images/20211116/7771c45764bc43229c1a88c0311566e3.png
[SouthEast 10]: /images/20211116/ddaa5d48302f45198887ea05058109fb.png
[SouthEast 11]: /images/20211116/7267ee73da7c4da397cb5a1d02eccafb.png