全部 标题 作者
关键词 摘要

OALib Journal期刊
ISSN: 2333-9721
费用:99美元

查看量下载量

相关文章

更多...

2维至3维图像/视频转换的深度图提取方法综述

DOI: 10.11834/jig.20141001

Keywords: 2D-3D,深度图提取,立体视觉,深度线索,机器学习

Full-Text   Cite this paper   Add to My Lib

Abstract:

目的深度图提取是计算机视觉领域的研究热点。随着3D显示设备的普及,2D-3D图像/视频转换的深度图提取研究受到越来越多国内外学者的关注。为此回顾深度图提取研究历程,并对已有成果进行分类、概括和评述。方法由于深度图提取方法的实现主要依赖于深度线索,不同方法存在人机交互程度上的差异。采用基于深度线索和基于人机交互程度的两种分类方法进行归纳评述。结果根据深度线索的不同,将深度图提取方法分为基于单目线索的方法、基于双目线索的方法和基于混合线索的3类方法。然后从人机交互的角度,将深度图提取方法分为人工法、半自动法和全自动法。介绍了这些方法的基本思想,比较归纳不同方法的优点与不足。最后,阐述了近年来热门的机器学习方法在深度图提取的应用。结论对深度图提取研究进行简要的总结和展望。指出深度图提取研究具有从研究热点中挖掘创新思路、引入新的深度线索等发展趋势。

References

[1]  Fehn C. Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV[C]//Proceedings of Electronic Imaging 2004. Bellingham: SPIE, 2004: 93-104. [DOI:10.1117/12.524762]
[2]  Kauff P, Atzpadin N, Fehn C, et al. Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability[J]. Signal Processing: Image Communication, 2007, 22(2): 217-234. [DOI: 10.1016/j.image.2006.11.013]
[3]  Zhang R, Tsai P S, Cryer J E, et al. Shape-from-shading: a survey[J]. IEEE Transactions on, Pattern Analysis and Machine Intelligence, 1999, 21(8): 690-706. [DOI: 10.1109/34.784284]
[4]  Krotkov E. Focusing[J]. International Journal of Computer Vision, 1988, 1(3): 223-237. [DOI: 10.1007/BF00127822]
[5]  Li P, Farin D, Gunnewiek R K, et al. On creating depth maps from monoscopic video using structure from motion[C]// Proceedings of the 27th Symposium on Information Theory in the Benelux. Benelux: IEEE, 2006: 508-515. [DOI: 10.1.1.129.7308]
[6]  Kindermann R, Snell J L. Markov Random Fields and their Applications[M]. Providence, RI: American Mathematical Society, 1980.
[7]  Zhou Y, Hu B, Zhang J. Occlusion Detection and Tracking Method Based on Bayesian Decision Theory[M]. Berlin Heidelberg: Springer, 2006: 474-482. [DOI: 10.1007/11949534_47]
[8]  Harman P V, Flack J, Fox S, et al. Rapid 2D-to-3D conversion[C]//Proceedings of SPIE. Bellingham: Society of Photo-Optical Instrumentntation Engineers Press, 2002: 78-86. [DOI: 10.1117/12.468020]
[9]  Zhang L, V?zquez C, Knorr S. 3D-TV content creation: automatic 2D-to-3D video conversion[J]. IEEE Transactions on, Broadcasting, 2011, 57(2): 372-383. [DOI: 10.1109/TBC.2011.2122930]
[10]  Helmholtz H. Treatise on Physiological Optics [M]. New York: Courier Dover Publications, 2005.
[11]  Grimson W E L. A computer implementation of a theory of human stereo vision[J]. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 1981: 217-253.
[12]  Ikeuchi K. Determining a depth map using a dual photometric stereo[J]. The International Journal of Robotics Research, 1987, 6(1): 15-31. [DOI: 10.1177/027836498700600102]
[13]  Shao M, Simchony T, Chellappa R. New algorithms from reconstruction of a 3-d depth map from one or more images[C]//Proceedings of CVPR\'88. Ann Arbor: IEEE, 1988: 530-535. [DOI: 10.1109/CVPR.1988.196286]
[14]  Matthies L, Kanade T, Szeliski R. Kalman filter-based algorithms for estimating depth from image sequences[J]. International Journal of Computer Vision, 1989, 3(3): 209-238. [DOI: 10.1007/BF00133032]
[15]  Matthies L, Szeliski R, Kanade T. Incremental estimation of dense depth maps from image sequences[C]//Proceedings of CVPR\'88. Ann Arbor: IEEE, 1988: 366-374. [DOI: 10.1109/CVPR.1988.196261]
[16]  Mori T, Yamamoto M. A dynamic depth extraction method[C]//Proceedings of Third International Conference on Computer Vision. Osaka: IEEE, 1990: 672-676. [DOI: 10.1109/ICCV.1990.139616]
[17]  Inoue H, Tachikawa T, Inaba M. Robot vision system with a correlation chip for real-time tracking, optical flow and depth map generation[C]//Proceedings of Robotics and Automation. Nice: IEEE, 1992: 1621-1626. [DOI: 10.1109/ROBOT.1992.220020]
[18]  Loomis J M. Looking down is looking up[J]. Nature, 2001, 414(6860): 155-156. [DOI: 10.1038/35102648]
[19]  Cozman F, Krotkov E. Depth from scattering[C] // Proceedings of CVPR\'97. San Juan: IEEE, 1997: 801-806. [DOI: 10.1109/CVPR.1997.609419]
[20]  Xiong Y, Shafer S A. Depth from focusing and defocusing[C]//Proceedings of CVPR\'93. New York: IEEE, 1993: 68-73. [DOI: 10.1109/CVPR.1993.340977]
[21]  更多...
[22]  Choe Y, Kashyap R. Shape from textured and shaded surface[C]//Proceedings of Pattern Recognition. Atlantic City: IEEE, 1990, 1: 294-296. [DOI: 10.1109/ICPR.1990.118117]
[23]  Battiato S, Capra A, Curti S, et al. 3D stereoscopic image pairs by depth-map generation[C]//Proceedings of 3D Data Processing, Visualization and Transmission. Los Alamitos: IEEE, 2004: 124-131. [DOI: 10.1109/TDPVT.2004.1335185]
[24]  Torralba A, Oliva A. Depth estimation from image structure[J]. IEEE Transactions on, Pattern Analysis and Machine Intelligence, 2002, 24(9): 1226-1238. [DOI: 10.1109/TPAMI.2002.1033214]
[25]  Huang Y S, Cheng F H, Liang Y H. Creating depth map from 2D scene classification[C]//Proceedings of ICICIC\'08. Dalian: IEEE, 2008: 69-69. [DOI: 10.1109/ICICIC.2008.205]
[26]  Saxena A, Schulte J, Ng A Y. Depth Estimation Using Monocular and Stereo Cues[C]//IJCAI California: AAAI, 2007, 7:2197-2203.
[27]  Wei Q. Converting 2d to 3d: a survey[C] //Proceedings of International Conference. Netherlands: Delft University of Technology, 2005, 7: 14.
[28]  Cao X, Bovik A C, Wang Y, et al. Converting 2D video to 3D: an efficient path to a 3D experience[J]. MultiMedia, IEEE, 2011, 18(4): 12-17. [DOI: 10.1109/MMUL.2011.65]
[29]  Comaniciu D, Meer P. Robust analysis of feature spaces: color image segmentation[C]//Proceedings of CVPR\'97. San Juan: IEEE, 1997: 750-755. [DOI: 10.1109/CVPR.1997.609410]
[30]  Yu F, Liu J, Ren Y, et al. Depth generation method for 2D to 3D conversion[C]//Proceedings of 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video (3DTV-CON). Antalya: IEEE, 2011: 1-4. [DOI: 10.1109/3DTV.2011.5877196]
[31]  Elder J H, Zucker S W. Local scale control for edge detection and blur estimation[J]. IEEE Transactions on, Pattern Analysis and Machine Intelligence, 1998, 20(7): 699-716. [DOI: 10.1109/34.689301]
[32]  Thoma R, Bierling M. Motion compensating interpolation considering covered and uncovered background[J]. Signal Processing: Image Communication, 1989, 1(2): 191-212. [DOI: 10.1016/0923-5965(89)90009-X]
[33]  Izquierdo E. Stereo matching for enhanced telepresence in three-dimensional video communications[J]. IEEE Transactions on, Circuits and Systems for Video Technology, 1997, 7(4): 629-643. [DOI: 10.1109/76.611174]
[34]  Horn B K P, Brooks M J. The variational approach to shape from shading[J]. Computer Vision, Graphics, and Image Processing, 1986, 33(2): 174-208. [DOI: 10.1016/0734-189X(86)90114-3]
[35]  Frankot R T, Chellappa R. A method for enforcing integrability in shape from shading algorithms[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1988, 10(4): 439-451. [DOI: 10.1109/34.3909]
[36]  Rouy E, Tourin A. A viscosity solutions approach to shape-from-shading[J]. SIAM Journal on Numerical Analysis, 1992, 29(3): 867-884. [DOI: 10.1137/0729053]
[37]  Pentland A P. Local shading analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1984 (2): 170-187. [DOI: 10.1109/TPAMI.1984.4767501]
[38]  Ping-Sing T, Shah M. Shape from shading using linear approximation[J]. Image and Vision Computing, 1994, 12(8): 487-498. [DOI: 10.1016/0262-8856(94)90002-7]
[39]  Clerc M, Mallat S. The texture gradient equation for recovering shape from texture[J]. IEEE Transactions on, Pattern Analysis and Machine Intelligence, 2002, 24(4): 536-549. [DOI: 10.1109/34.993560]
[40]  Forsyth D A. Shape from texture without boundaries[M]//Computer Vision―ECCV 2002. Berlin Heidelberg: Springer, 2002: 225-239. [DOI: 10.1007/3-540-47977-5_15]
[41]  Loh A M, Hartley R. Shape from non-homogeneous, non-stationary, anisotropic, perspective texture[C]//Proceedings of the British Machine Vision Conference 2005. Oxford: BMVC, 2005. [DOI: 10.5244/C.19.8]
[42]  Shimshoni I, Moses Y, Lindenbaum M. Shape reconstruction of 3D bilaterally symmetric surfaces[J]. International Journal of Computer Vision, 2000, 39(2): 97-110. [DOI: 10.1016/0734-189X(86)90114-3]
[43]  Jung Y J, Baik A, Kim J, et al. A novel 2D-to-3D conversion technique based on relative height-depth cue[C]//Proceedings of Stereoscopic Displays and Application XX. Bellingham: SPIE, 2009: 72371U-72371U-8. [DOI: 10.1117/12.806058]
[44]  Xu F, Er G, Xie X, et al. 2D-to-3D conversion based on motion and color mergence[C]//Proceedings of 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video. Istanbul: IEEE, 2008: 205-208. [DOI: 10.1109/3DTV.2008.4547844]
[45]  Kim J, Kim Y, Park J, et al. Stereoscopic conversion of two-dimensional movie encoded in MPEG-2[C]//Proceedings of the International Society for Optical Engineering. Bellingham: SPIE, 2006, 6311: 631105(1-8). [DOI:10.1117/12.680328]
[46]  Cancino-Suarez S, Klimaszewski K, Stankiewicz O, et al. Enhancement of stereoscopic depth estimation by the use of motion information[C]//Proceedings of the 44th Southeastern Symposium on System Theory (SSST), Jacksonville: IEEE, 2012: 94-98. [DOI: 10.1109/SSST.2012.6195119]
[47]  Li T. Depth map recovery for monocular and multi-view images[D]. Beijing: Tsinghua University, 2009.[李涛. 单目视图与多目视图的深度图恢复方法研究[D].北京:清华大学,2009.]
[48]  Ens J, Lawrence P. A matrix based method for determining depth from focus[C]//Proceedings of CVPR\'91. Maui: IEEE, 1991: 600-606. [DOI: 10.1109/CVPR.1991.139760]
[49]  Wong K T, Ernst F. Single image depth from defocus[D]. Netherlands: Delft university of Technology & Philips Natlab Research, Eindhoven, 2004.
[50]  Taketomi Y, Ikeoka H, Hamamoto T. Depth estimation based on defocus blur using a single image taken by a tilted lens optics camera[C]// International Symposium on Intelligent Signal Processing and Communications Systems (ISPACS ). Naha: IEEE, 2013: 403-408. [DOI: 10.1109/ISPACS.2013.6704583]
[51]  Shi G K, Wang Q H, Li D H, et al. Method for depth map extraction of defocused image based on segmentation[J]. Chinese Journal of Liquid Crystals and Displays, 2012,2:229-234. [史国凯,王琼华,李大海,等. 基于分割的离焦图像深度图提取方法[J]. 液晶与显示,2012,2:229-234.]
[52]  Zong C L. Research on disparity extraction algorithm based on binocular stereo matching[D]. Chengdu: Southwest Jiaotong University, 2013. [宗晨临. 基于双目立体匹配的视差提取算法研究[D].成都:西南交通大学,2013.]
[53]  Scharstein D, Szeliski R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[J]. International Journal of Computer Vision, 2002, 47(1-3): 7-42. [DOI: 10.1023/A:1014573219977]
[54]  Gong M, Yang R, Wang L, et al. A performance study on different cost aggregation approaches used in real-time stereo matching[J]. International Journal of Computer Vision, 2007, 75(2): 283-296. [DOI: 10.1007/s11263-006-0032-x]
[55]  Veksler O. Fast variable window for stereo correspondence using integral images[C]// Proceedings of CVPR\'03. Wiscosin: IEEE, 2003, 1: 556-561. [DOI: 10.1109/CVPR.2003.1211403]
[56]  Yoon K J, Kweon I S. Adaptive support-weight approach for correspondence search[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(4): 650-656
[57]  Scharstein D, Szeliski R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[J]. International Journal of Computer Vision, 2002, 47(1-3): 7-42. [DOI: 10.1023/A:1014573219977]
[58]  Boykov Y, Veksler O, Zabih R. Fast approximate energy minimization via graph cuts[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(11): 1222-1239. [DOI: 10.1109/34.969114]
[59]  Pearl J. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference[M]. San Francisco: Morgan Kaufmann, 1988.
[60]  Chen W M, Jhang S H. Improving graph cuts algorithm to transform sequence of stereo image to depth map[J]. Journal of Systems and Software, 2013, 86(1): 198-210
[61]  Dai Q H, Ma X, Suo J L, et al. Binocular video depth map calculating method: CN, 103258604 [P]. 2013-08-28. [戴琼海,马潇,索津莉,等. 双目视频深度图求取方法:中国,103258604 [P]. 2013-08-28.]
[62]  Verma R, Singh H S, Verma A K. Depth estimation from stereo images based on adaptive weight and segmentation[J]. Journal of The Institution of Engineers (India): Series B, 2012, 93(4): 223-229. [DOI: 10.1007/s40031-012-0030-z]
[63]  Bülthoff I, Bülthoff H, Sinha P. Top-down influences on stereoscopic depth-perception[J]. Nature Neuroscience, 1998, 1(3): 254-257. [DOI: 10.1038/699]
[64]  Ghaffar R, Jafri N, Khan S A. Depth extraction system using stereo pairs[M]//Image Analysis and Recognition. Berlin Heidelberg: Springer, 2004: 512-519. [DOI: 10.1007/978-3-540-30126-4_63]
[65]  Huang X, Wang L, Huang J, et al. A depth extraction method based on motion and geometry for 2D to 3D conversion[C]//Proceedings of IITA. Nanchang: IEEE, 2009, 3: 294-298. [DOI: 10.1109/IITA.2009.481]
[66]  Harman P. Home based 3D entertainment-an overview[C]//Proceedings of Image Processing. Vancouver: IEEE, 2000, 1: 1-4. [DOI: 10.1109/ICIP.2000.900877]
[67]  Yu F L. Research on depth generation methods for 2D-3D conversion[D]. Shandong: Shandong University, 2012. [于凤利. 2D-3D视频转换中深度图生成方法研究[D].山东:山东大学,2012.]
[68]  Chen J, Zhao J, Wang X, et al. A simple semi-automatic technique for 2D to 3D video conversion[M]//Artificial Intelligence and Computational Intelligence. Berlin Heidelberg: Springer, 2011: 336-343. [DOI: 10.1007/978-3-642-23887-1_42]
[69]  Lie W N, Chen C Y, Chen W C. 2D to 3D video conversion with key-frame depth propagation and trilateral filtering[J]. Electronics Letters, 2011, 47(5): 319-321. [DOI: 10.1049/el.2010.2912]
[70]  Yan X, Yang Y, Er G, et al. Depth map generation for 2d-to-3d conversion by limited user inputs and depth propagation[C]//Proceedings of 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video (3DTV-CON). Antalya: IEEE, 2011: 1-4. [DOI: 10.1109/3DTV.2011.5877167]
[71]  Jung J I, Ho Y S. Depth map estimation from single-view image using object classification based on Bayesian learning[C]//Proceedings of 3DTV-Conference. Tampere: IEEE, 2010: 1-4. [DOI: 10.1109/3DTV.2010.5506603]
[72]  Lee H, Jung C, Kim C. Depth map estimation based on geometric scene categorization[C]//Proceedings of the 19th Korea-Japan Joint Workshop on Frontiers of Computer Vision. Incheon: IEEE, 2013: 170-173. [DOI: 10.1109/FCV.2013.6485482]
[73]  Saxena A, Chung S H, Ng A Y. 3-d depth reconstruction from a single still image[J]. International Journal of Computer Vision, 2008, 76(1): 53-69. [DOI: 10.1007/s11263-007-0071-y]
[74]  Mitchell T M. Machine learning and data mining[J]. Communications of the ACM, 1999, 42(11): 30-36. [DOI: 10.1145/319382.319388]
[75]  Saxena A, Chung S H, Ng A. Learning depth from single monocular images[C]//Proceedings of Advances in Neural Information Processing Systems. Cambridge: The MIT Press,2005: 1161-1168. [DOI: 10.1.1.72.8799]
[76]  Saxena A, Chung S H, Ng A Y. 3-d depth reconstruction from a single still image[J]. International Journal of Computer Vision, 2008, 76(1): 53-69. [DOI: 10.1007/s11263-007-0071-y]
[77]  Saxena A, Sun M, Ng A Y. Make3D: depth perception from a single still image[C]//Proceedings of the 23rd National Conference on Artificial Intelligence-Volume 3. California: AAAI, 2008: 1571-1576.
[78]  Karsch K, Liu C, Kang S B. Depth extraction from video using non-parametric sampling [M]// Computer Vision-ECCV. Berlin Heidelberg: Springer, 2012: 775-788. [DOI: 10.1007/978-3-642-33715-4_56]
[79]  Konrad J, Wang M, Ishwar P. 2d-to-3d image conversion by learning depth from examples [C]// Proceedings of CVPRW. Providence, RI: IEEE, 2012: 16-22. [DOI: 10.1109/CVPRW.2012.6238903]
[80]  Liu C, Yuen J, Torralba A, et al. Sift flow: dense correspondence across different scenes[M]//Computer Vision-ECCV. Berlin Heidelberg: Springer, 2008: 28-42. [DOI: 10.1007/978-3-540-88690-7_3]
[81]  Liu C. Beyond pixels: exploring new representations and applications for motion analysis[D]. Cambridge, Massachusetts: Massachusetts Institute of Technology, 2009
[82]  Su C C, Cormack L K, Bovik A C. Depth estimation from monocular color images using natural scene statistics models[C]//Proceedings of IVMSP Workshop. Seoul: IEEE, 2013: 1-4. [DOI: 10.1109/IVMSPW.2013.6611900]
[83]  Li M, Fan Z, Lu Y. A New method of Image depth extraction based on SVM[C]//Proceedings of CCA. Harbin: SERSC, 2013: 219-222.
[84]  Han H, Lee G, Lee J, et al. A new method to create depth information based on lighting analysis for 2D/3D conversion[J]. Journal of Central South University, 2013, 20(10): 2715-2719. [DOI: 10.1007/s11771-013-1788-0]

Full-Text

Contact Us

service@oalib.com

QQ:3279437679

WhatsApp +8615387084133