Current Issue Cover


摘 要
目的 人脸二维图像反映出来的纹理并非是三维人脸曲面真实的纹理,并且受光照和妆容的影响很大,因此探索三维局部纹理特征对于人脸识别任务有着重要的意义。方法 本文详细分析了一种新颖的三维局部纹理特征mesh-LBP对于人脸纹理的描述能力。首先,在特征提取和识别任务之前,进行一系列的预处理:人脸分割,异常点移除和孔洞填补。接着,在预处理后的人脸曲面上,提取原始mesh-LBP特征,以及基于阈值化策略的三种改进特征:mesh-tLBP,mesh-MBP和mesh-LTP。然后,对于上述提取的四种特征,采用不同的统计方法,包括整体直方图,局部分块直方图和整体编码图像,用作人脸纹理的特征描述。最后,针对CASIA3D数据集中不同表情和姿态变化的人脸,采用余弦相似度进行人脸的识别任务。结果 通过对比人脸曲面和普通物体曲面的纹理特征,发现人脸纹理完全不同于普通纹理,不规则并且难以描述;通过对比mesh-LBP两种变体,发现mesh-LBP(ɑ1)适用于姿态变化, 而mesh-LBP(ɑ2)适用于表情变化;通过对比原始mesh-LBP及其三种改进,发现mesh-tLBP对于人脸不同表情变化下的识别准确率最高有0.5%的提升;通过对比三种不同的统计方法,发现采用整体编码图像进行统计的特征尽管弱于局部分块直方图,但相比整体直方图,识别率在不同表情变化下最高有46.8%的提升。结论 Mesh-LBP特征是一种优良的三维局部纹理特征,未来将会在三维医学处理,三维地形起伏检测以及三维人脸识别中得到更多的应用。
Local texture features on the mesh for 3D face recognition

雷 超,张 海燕,詹 曙(College of Computer and information, HeFei University of Technology, HeFei 230601 China)

Objective Texture reflected by 2D facial image is not true for a 3D face surface, and this 2D texture is greatly affected by variations of illumination and make-up. These issues make the exploration of 3D local texture features important for face recognition tasks. The concept of 3D texture is completely different from 2D texture, which reflects the repeatable patterns of a 3D facial surface. Also, due to the flexibility of 3D mesh, in addition to geometric information, it also preserves the photometric information of the same individual. Therefore, it is of great importance to explore these two original 3D texture, i.e, 3D geometric texture and 3D photometric texture.Method In this paper, we investigate a novel framework, i.e, mesh-LBP for representing 3D facial texture in detail. Notice here that we mainly focus on the improvement and statistic of this operator, rather than the comparisons of final face recognition rate with the state of the art methods. First, because raw 3D facial data not only contain spikes and holes but also a large background area, a set of general preprocessing operations, including face detection, outlier removal and holes filling are performed before feature extraction and classification. Specifically, a facial surface is cropped at first by a common scheme, i.e., the point sets of raw face model located on a sphere, which is constructed by nose tip and fixed radius, are extracted as the detected facial area. Then we define the outlier of raw data as the point whose number of neighborhood points are lower than a threshold. When these outliers are detected, the mean filter is used to smooth facial surface. The outlier removal operation usually results in holes in 3D facial data, so we adopt bicubic interpolation to solve this problem. Second, the construction procedure of original mesh-LBP operator and three improved operators based on thresholding scheme, which we dubbed as mesh-tLBP, mesh-MBP and mesh-LTP, are developed. For mesh-tLBP, a small threshold is added to the calculation process of mesh-LBP. For mesh-MBP, value of a center facet on the mesh is replaced by the mean value of its neighborhood. For mesh-LTP, an additional coding unit is added for more subtle capture of code changes of mesh-LBP. The first two improvements are designed to improve the robustness of mesh-LBP to noise or changes of face, while the last one is to improve the power of mesh-LBP to capture facial details. Third, different statistical methods, including the na?ve holistic histogram, the spatially enhanced histogram and the holistic coded image are used to form the final facial representation. For the na?ve holistic histogram, we do not use any processing method and directly perform frequency statistics for the calculated LBP pattern. For the spatially enhanced histogram, we first block a 3D facial surface, then perform frequency statistics for each block, and concatenate them together to form the whole description of this face. For the holistic coded image, we use the calculated LBP pattern directly, but the number of patterns arising from different faces is different, so we first need to normalize them to the same size. Finally, based on a simple minimum distance classifier, we employ 615 neutral scans under different illumination condition from CASIA3D face database as the training set and evaluate the recognition performance on the 615 scans of expression variation and 1230 scans of pose.Result By comparing the texture features of facial surface and common object surface, we found that facial texture is completely different from ordinary texture, and it is irregular and difficult to describe. In addition, we noticed that the texture variations of 3D face are smaller than that of 2D face, and this discovery shows the superiority of 3D data. By the experiments of two variants of mesh-LBP, we found that mesh-LBP(ɑ1) is more robust to pose variations, while mesh-LBP(ɑ2) more robust to expression variations. By the experiments of original mesh-LBP and its three improvements, we observed that only mesh-tLBP brings a 0.5% improvement of recognition accuracy on different facial expression variations in the best case. And results of mesh-LTP are basically the same as mesh-LBP, while results of mesh-MBP become worse than mesh-LBP. Surprisingly, these improvements bring no great promotion for representing 3D facial texture, further pre-processing and parameter selection scheme may be required for the better results. By comparing the result of three statistical methods, we noticed that features based on the spatially enhanced histogram obtain the best recognition in two experimental scenarios. The description power of features based on holistic coded image is weaker than that of the spatially enhanced histogram, but its recognition rate is increased by 46.8% compared with the features based the na?ve holistic histogram on different expression variations in the best case. In addition, we found results of features based on holistic coded image on pose variations are the worst of all statistical methods, this is mainly due to the limitation of image.Conclusion Compared with other 3D local feature descriptors, mesh-LBP is an elegant and efficient framework that allows the extraction 3D local texture directly from a mesh manifold. The calculated patterns of mesh-LBP can use different statistical methods for 3D texture analysis of different type of object. For example, the simple mesh-hLBPH is suitable for ordinary 3D object, while the mesh-eLBPH for 3D face analysis. We believed that mesh-LBP will be widely used in 3D medical imaging, 3D terrain relief inspection and 3D face recognition in the near future. As our next work, there are several aspects worth to explore. First, the fusion of 3D geometry and 3D photometric appearance based on the mesh-LBP framework seems a promising direction to improve recognition. Second, optimizing further the size of mesh-LBP(ɑ2) ,increasing the discrimination power mesh-LBP(ɑ1) . Third, extending other scheme of 2D LBP, such as neighborhood topology and sampling to mesh-LBP for different application of 3D texture. Finally, integrating mesh-LBP with a robust matching algorithm would also be worthwhile to investigate.