|
|
- 2017
基于单目RGB摄像机的三维人脸表情跟踪算法
|
Abstract:
摘要： 利用单目摄像机对人脸表情进行三维跟踪有两个关键问题,一是从单幅图像中重建人脸的三维模型,二是建立不同三维模型之间稠密的对应关系。针对上述问题,提出一种有效的三维人脸表情跟踪算法。对输入的各帧人脸图像进行自动特征点检测,并根据2D特征点,利用Multilinear Model重建不同表情的三维模型。通过将三维模型重投影至图像空间,计算各帧图像间的Optic Flow与Sift Flow对应。根据图像间的对应关系,利用Laplacian网格变形对三维模型进行修正,建立不同表情之间三维的稠密对应。试验表明,该方法可以重建出较为真实的三维人脸表情模型,同时可以精确跟踪连续变化的三维表情序列。
Abstract: There were two key problems in tracking 3D facial expression using a monocular RGB camera. One was how to reconstruct 3D facial model from a single image, and the other was how to establish the dense correspondences between the different 3D models. To solve the above problems, an effective 3D facial expression tracking algorithm was proposed. The feature points were automatically detected on each input facial image frame, and the 3D models of different expressions were reconstructed by Multilinear Model according to 2D feature points. By re-projecting the 3D models into the image space, the Optic Flow and Sift Flow correspondences between images were calculated. According to the correspondences of the images, the 3D models were rectified by Laplacian mesh deformation, and the dense 3D correspondences between different expressions were established. The experimental results showed that the proposed method could create more realistic 3D expression models, and could accurately track the continuous change of the 3D expression sequence
| [1] | BLANZ V, SCHERBAUM K, VETTER T, et al. Exchanging faces in images[J].Computer Graphics Forum, 2004, 23(3): 669-676. |
| [2] | VLASIC D, BRAND M, PFISTER H, et al. Face transfer with multilinear models[J]. ACM Transactions on Graphics, 2005, 24(3): 426-433. |
| [3] | CAO Chen, WENG Yanlin, LIN Stephen, et al. 3D shape regression for real-time facial animation[J]. ACM Transactions on Graphics, 2013, 32(4): 41. |
| [4] | SUPASOM S, IRA K, SEITZ M S. Total moving face reconstruction[M] //Lecture Notes in Computer Science(including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). [S.l.] : Springer International Publishing,2014,8692(4): 796-812. |
| [5] | REN Shaoqing, CAO Xuding, WEI Yichen, et al. Face alignment at 3000 fps via regressing local binary features[C] //Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014: 1685-1692. |
| [6] | SHI Fuhao, WU Hsiangtao, TONG Xin, et al. Automatic acquisition of high-fidelity facial performances using monocular videos[J]. ACM Transactions on Graphics, 2014, 33(6): 222. |
| [7] | FYFFE G, JONES A, ALEXANDER O, et al. Driving high-resolution facial scans with video performance capture[J]. ACM Transactions on Graphics, 2014, 34(1): 8. |
| [8] | SIBBING D, HABBECKE M, KOBBELT L. Markerless reconstruction and synthesis of dynamic facial expressions[J]. Computer Vision and Image Understanding, 2011, 115(5): 668-680. |
| [9] | LI Hao, YU Jihun, YE Yuting, et al. Realtime facial animation with on-the-fly correctives[J]. ACM Transactions on Graphics, 2013, 32(4): 42. |
| [10] | 迟静, 屠长河, 陈雪. 基于法向保持的三维人脸表情跟踪[J]. 计算机辅助设计与图形学学报, 2011, 23(6): 1069-1077. CHI Jing, TU Changhe, CHEN Xue. 3D facial expression tracking based on normal preserving[J]. Journal of Computer-Aided Design & Computer Graphics, 2011, 23(6): 1069-1077. |
| [11] | WEISE T, BOUAZIZ S, LI Hao, et al. Realtime performance-based facial animation[J]. ACM Transactions on Graphics, 2011, 30(4): 77. |
| [12] | 何钦政, 王运巧. 基于Kinect的人脸表情捕捉及动画模拟系统研究[J]. 图学学报, 2016, 37(3): 290-295. HE Qinzheng, WANG Yunqiao. Research on system of facial expression capture and animation simulation based on Kinect[J]. Journal of Graphics, 2016, 37(3): 290-295. |
| [13] | 李俊龙, 章登义, 黄珺. Kinect驱动的人脸动画合成技术研究[J]. 计算机工程, 2015, 41(3):237-241. LI Junlong, ZHANG Dengyi, HUANG Jun. Research on synthesis technology of facial animation driven by Kinect[J]. Computer Engineering, 2015, 41(3): 237-241. |
| [14] | 栾勐. 基于拉普拉斯变形技术的形状分析方法研究[D].杭州: 浙江大学, 2011. LUAN Meng. Shape analysis based on Laplacian deformation[D]. Hangzhou: Zhejiang University, 2011. |
| [15] | SORKINE O, COHEN-OR D, LIPMAN Y, et al. Laplacian surface editing[C] //Proceedings of the 2004 Eurographics/ACM SIGGRAPH Symposium on Geometry processing. [S.l.] :ACM, 2004, 71:175-184. |
| [16] | ZHANG Li, SNAVELY N, CURLESS B, et al. Spacetime faces: high resolution capture for modeling and animation[J]. ACM Transactions on Graphics, 2004, 23(3): 548-558. |
| [17] | BEELER T, BICKEL B, BEARDSLEY P, et al. High-quality single-shot capture of facial geometry[J]. ACM Transactions on Graphics, 2010, 29(4): 157-166. |
| [18] | BEELER T, HAHN F, BRADLEY D, et al. High-quality passive facial performance capture using anchor frames[J]. ACM Transactions on Graphics, 2011, 30(4): 75. |
| [19] | CAO Chen, WENG Yanlin, ZHOU Shun, et al. Face-warehouse: a 3d facial expression database for visual computing[J]. IEEE Transactions on Visualization and Computer Graphics, 2014, 20(3): 413-425. |
| [20] | HUANG Haoda, CHAI Jinxiang, TONG Xin, et al. Leveraging motion capture and 3D scanning for high-fidelity facial performance acquisition[J]. ACM Transactions on Graphics, 2011, 30(4): 74. |
| [21] | BRADLEY D, HEIDRICH W, POPA T, et al. High resolution passive facial performance capture[J]. ACM Transactions on Graphics, 2010, 29(4): 41. |
| [22] | CHEN Yenlin, WU Hsiangtao, SHI Fuhao, et al. Accurate and robust 3D facial capture using a single RGBD camera[C] //Proceedings of the 2013 IEEE International Conference on Computer Vision. Sydney, Australia: IEEE, 2013: 3615-3622. |
