|
|
基于深度学习的6D位姿估计方法最新研究
|
Abstract:
6D位姿估计技术在工业机器人、虚拟现实和餐饮服务等领域已经成为关键性的技术。它的发展逐渐由离线方式发展到端到端的方式。这主要因为随着深度学习技术的发展，促进了6D位姿估计理论不断地完善，进一步导致了对该方面的技术总结和时效性不强。因此，本文基于最新的6D位姿估计技术的方法进行调研。在本文中，我们首先介绍了6D位姿估计技术的评估指标、数据集和6D位姿估计方法。其中6D位姿估计方法以数据的输入方式进行划分，我们将方法划分为基于D、RGB和RGBD的方法。最后，我们就这些各类方法给出了我们的启发和未来可能的研究方向，希望给予相关人员一定的帮助。
6D pose estimation technology has become critical in industrial robotics, virtual reality, and food service areas. It has gradually evolved from an offline approach to an end-to-end approach. This is mainly due to the fact that with the development of deep learning techniques, it has facilitated the continuous improvement of 6D pose estimation theory, which has further led to a poor summary and timeliness of the techniques in this area. Therefore, this paper investigates the approach based on the latest 6D pose estimation techniques. In this paper, we first introduce the evaluation metrics, datasets, and 6D pose estimation methods for 6D pose estimation techniques. Where the 6D pose estimation methods are classified by the input method of the data, we classify the methods into D-based, RGB, and RGBD-based methods. Finally, we give our inspiration and possible future research directions on these various methods, which we hope can help related people.
| [1] | Kiran, B.R., Sobh, I., Talpaert, V., et al. (2021) Deep Reinforcement Learning for Autonomous Driving: A Survey. IEEE Transactions on Intelligent Transportation Systems, 23, 4909-4926. https://doi.org/10.1109/TITS.2021.3054625 |
| [2] | Zhu, Z. and Zhao, H. (2021) A Survey of Deep RL and IL for Autonomous Driving Policy Learning. |
| [3] | Bousmalis, K., Irpan, A., Wohlhart, P., et al. (2018) Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping. Proceedings of the IEEE International Conference on Robotics and Automation, Brisbane, 21-25 May 2018, 4243-4250. https://doi.org/10.1109/ICRA.2018.8460875 |
| [4] | James, S., Wohlhart, P., Kalakrishnan, M., et al. (2019) Sim-to-Real via Sim-to-Sim: Data-Efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Long Beach, 15-20 June 2019, 12627-12637.
https://doi.org/10.1109/CVPR.2019.01291 |
| [5] | Fang, W., Zheng, L. and Wu, X. (2017) Multi-Sensor Based Real-Time 6-DoF Pose Tracking for Wearable Augmented Reality. Computers in Industry, 92, 91-103. https://doi.org/10.1016/j.compind.2017.06.002 |
| [6] | Strobl, K.H., Mair, E., Bodenmuller, T., et al. (2018) Portable 3-D Modeling Using Visual Pose Tracking. Computers in Industry, 99, 53-68. https://doi.org/10.1016/j.compind.2018.03.009 |
| [7] | Hodaň, T., Zabulis, X., Lourakis, M., et al. (2015) Detection and Fine 3D Pose Estimation of Texture-Less Objects in RGB-D Images. IEEE International Conference on Intelligent Robots and Systems, Hamburg, 28 September-3 October 2015, 4421-4428. https://doi.org/10.1109/IROS.2015.7354005 |
| [8] | Hinterstoisser, S., Cagniart, C., Ilic, S., et al. (2012) Gradient Response Maps for Real-Time Detection of Textureless Objects. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34, 876-888.
https://doi.org/10.1109/TPAMI.2011.206 |
| [9] | Chen, J., Lei, B., Song, Q., et al. (2020) A Hierarchical Graph Network for 3D Object Detection on Point Clouds. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, 14-19 June 2020, 392-401. https://doi.org/10.1109/CVPR42600.2020.00047 |
| [10] | Shi, W. and Rajkumar, R. (2020) Point-gnn: Graph Neural Network for 3D Object Detection in a Point Cloud. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, 14-19 June 2020, 1711-1719. https://doi.org/10.1109/CVPR42600.2020.00178 |
| [11] | Zhu, C., Chen, F., Ahmed, U., et al. (2021) Semantic Relation Reasoning for Shot-Stable Few-Shot Object Detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Nashville, 20-25 June 2021, 8782-8791. https://doi.org/10.1109/CVPR46437.2021.00867 |
| [12] | Yang, C., Wu, Z., Zhou, B., et al. (2021) Instance Localization for Self-Supervised Detection Pretraining. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Nashville, 20-25 June 2021, 3987-3996. https://doi.org/10.1109/CVPR46437.2021.00398 |
| [13] | Sahin, C. and Kim, T.K. (2018) Recovering 6D Object Pose: A Review and Multi-Modal Analysis. Proceedings of the European Conference on Computer Vision, Munich, 8-14 September 2018, 15-31.
https://doi.org/10.1007/978-3-030-11024-6_2 |
| [14] | Patil, A.V. and Rabha, P. (2018) A Survey on Joint Object Detection and Pose Estimation Using Monocular Vision. MATEC Web of Conferences, 277, Article No. 02029. https://doi.org/10.1051/matecconf/201927702029 |
| [15] | Kleeberger, K., Bormann, R., Kraus, W., et al. (2020) A Survey on Learning-Based Robotic Grasping. Current Robotics Reports, 1, 239-249. https://doi.org/10.1007/s43154-020-00021-6 |
| [16] | Sahin, C., Garcia-Hernando, G., Sock, J., et al. (2020) A Review on Object Pose Recovery: From 3d Bounding Box Detectors to Full 6d Pose Estimators. Image and Vision Computing, 96, Article ID: 103898.
https://doi.org/10.1016/j.imavis.2020.103898 |
| [17] | Hinterstoisser, S., Lepetit, V., Ilic, S., et al. (2012) Model Based Training, Detection and Pose Estimation of Textureless 3D Objects in Heavily Cluttered Scenes. In: Asian Conference on Computer Vision, Springer, Berlin, 548-562. |
| [18] | He, Y., Wang, Y., Fan, H., et al. (2022) FS6D: Few-Shot 6D Pose Estimation of Novel Objects. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seoul, 27 October-2 November 2022, 6814-6824. |
| [19] | Di, Y., Manhardt, F., Wang, G., et al. (2021) So-Pose: Exploiting Self-Occlusion for Direct 6d Pose Estimation. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, 27 October-2 November 2021, 12396-12405.
https://doi.org/10.1007/978-3-319-49409-8_52 |
| [20] | Shotton, J., Glocker, B., Zach, C., et al. (2013) Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Portland, 23-28 June 2013, 2930-2937. https://doi.org/10.1109/CVPR.2013.377 |
| [21] | Hodaň, T., Matas, J. and Obdr？álek, ？. (2016) On Evaluation of 6D Object Pose Estimation. In: Hua, G. and Jégou, H., Eds., European Conference on Computer Vision, Springer, Cham, 606-619.
https://doi.org/10.1007/978-3-319-49409-8_52 |
| [22] | Brachmann, E., Krull, A., Michel, F., et al. (2014) Learning 6d Object Pose Estimation Using 3d Object Coordinates. In: Fleet, D., Pajdla, T., Schiele, B. and Tuytelaars, T., Eds., European Conference on Computer Vision, Springer, Cham, 536-551. https://doi.org/10.1007/978-3-319-10605-2_35 |
| [23] | Singh, A., Sha, J., Narayan, K.S., et al. (2014) BigBIRD: A Large-Scale 3D Database of Object Instances. Proceedings of the IEEE International Conference on Robotics and Automation, Hong Kong, 31 May-7 June 2014, 509-516.
https://doi.org/10.1109/ICRA.2014.6906903 |
| [24] | Calli, B., Singh, A., Walsman, A., et al. (2015) The ycb Object and Model Set: Towards Common Benchmarks for Manipulation Research. Proceedings of the IEEE International Conference on Robotics and Automation, Seattle, 26-30 May 2015, 510-517. https://doi.org/10.1109/ICAR.2015.7251504 |
| [25] | Xiang, Y., Schmidt, T., Narayanan, V., et al. (2017) Posecnn: A Convolutional Neural Network for 6d Object Pose Estimation in Cluttered Scenes. https://doi.org/10.15607/RSS.2018.XIV.019 |
| [26] | Wang, H., Sridhar, S., Huang, J., et al. (2019) Normalized Object Coordinate Space for Category-Level 6d Object Pose and Size Estimation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Long Beach, 15-20 June 2019, 2642-2651. https://doi.org/10.1109/CVPR.2019.00275 |
| [27] | Hodan, T., Haluza, P., Obdr？álek, ？., et al. (2017) T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-Less Objects. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Santa Rosa, 24-31 March 2017, 880-888. https://doi.org/10.1109/WACV.2017.103 |
| [28] | Doumanoglou, A., Kouskouridas, R., Malassiotis, S., et al. (2016) Recovering 6D Object Pose and Predicting Next-Best-View in the Crowd. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 3583-3592. https://doi.org/10.1109/CVPR.2016.390 |
| [29] | Brégier, R., Devernay, F., Leyrit, L., et al. (2017) Symmetry Aware Evaluation of 3d Object Detection and Pose Estimation in Scenes of Many Parts in Bulk. Proceedings of the IEEE International Conference on Computer Vision Workshops, Venice, 22-29 October 2017, 2209-2218. https://doi.org/10.1109/ICCVW.2017.258 |
| [30] | Kleeberger, K., Landgraf, C. and Huber, M.F. (2019) Large-Scale 6d Object Pose Estimation Dataset for Industrial Bin-Picking. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Macau, 3-8 November 2019, 2573-2578. https://doi.org/10.1109/IROS40897.2019.8967594 |
| [31] | Gao, G., Lauri, M., Hu, X., et al. (2021) CloudAAE: Learning 6D Object Pose Regression with On-Line Data Synthesis on Point Clouds. 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi’an, 30 May-5 June 2021, 11081-11087. https://doi.org/10.1109/ICRA48506.2021.9561475 |
| [32] | Hagelskj？r, F. and Buch, A.G. (2020) PointVoteNet: Accurate Object Detection and 6 DOF Pose Estimation in Point Clouds. Proceedings of the IEEE International Conference on Image Processing, Macau, 1-4 December 2020, 2641-2645. https://doi.org/10.1109/ICIP40778.2020.9191119 |
| [33] | Rad, M. and Lepetit, V. (2017) Bb8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3d Poses of Challenging Objects without Using Depth. Proceedings of the IEEE International Conference on Computer Vision, Venice, 22-29 October 2017, 3828-3836. https://doi.org/10.1109/ICCV.2017.413 |
| [34] | Tekin, B., Sinha, S.N. and Fua, P. (2018) Real-Time Seamless Single Shot 6d Object Pose Prediction. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-22 June 2018, 292-301.
https://doi.org/10.1109/CVPR.2018.00038 |
| [35] | Oberweger, M., Rad, M. and Lepetit, V. (2018) Making Deep Heatmaps Robust to Partial Occlusions for 3d Object Pose Estimation. Proceedings of the European Conference on Computer Vision, Munich, 8-14 September 2018, 119-134. https://doi.org/10.1007/978-3-030-01267-0_8 |
| [36] | Peng, S., Liu, Y., Huang, Q., et al. (2019) Pvnet: Pixel-Wise Voting Network for 6d of Pose Estimation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Long Beach, 15-20 June 2019, 4561-4570. https://doi.org/10.1109/CVPR.2019.00469 |
| [37] | Song, C., Song, J. and Huang, Q. (2020) Hybridpose: 6d Object Pose Estimation under Hybrid Representations. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, 14-19 June 2020, 431-440. https://doi.org/10.1109/CVPR42600.2020.00051 |
| [38] | Brachmann, E., Michel, F., Krull, A., et al. (2016) Uncertainty-Driven 6d Pose Estimation of Objects and Scenes from a Single RGB Image. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 3364-3372. https://doi.org/10.1109/CVPR.2016.366 |
| [39] | Li, Z., Wang, G. and Ji, X. (2019) Cdpn: Coordinates-Based Disentangled Pose Network for Real-Time RGB-Based 6d of Object Pose Estimation. Proceedings of the IEEE International Conference on Computer Vision, Seoul, 27 October-2 November 2019, 7678-7687. https://doi.org/10.1109/ICCV.2019.00777 |
| [40] | Park, K., Patten, T. and Vincze, M. (2019) Pix2pose: Pixel-Wise Coordinate Regression of Objects for 6d Pose Estimation. Proceedings of the IEEE International Conference on Computer Vision, Seoul, 27 October-2 November 2019, 7668-7677. https://doi.org/10.1109/ICCV.2019.00776 |
| [41] | Zakharov, S., Shugurov, I. and Ilic, S. (2019) Dpod: 6d Pose Object Detector and Refiner. Proceedings of the IEEE International Conference on Computer Vision, Seoul, 27 October-2 November 2019, 1941-1950.
https://doi.org/10.1109/ICCV.2019.00203 |
| [42] | Hodan, T., Barath, D. and Matas, J. (2020) Epos: Estimating 6d Pose of Objects with Symmetries. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, 14-19 June 2020, 11703-11712. https://doi.org/10.1109/CVPR42600.2020.01172 |
| [43] | Park, K., Mousavian, A., Xiang, Y., et al. (2020) Latentfusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, 14-19 June 2020, 10710-10719. https://doi.org/10.1109/CVPR42600.2020.01072 |
| [44] | Li, Y., Wang, G., Ji, X., et al. (2018) Deepim: Deep Iterative Matching for 6d Pose Estimation. Proceedings of the European Conference on Computer Vision, Munich, 8-14 September 2018, 683-698.
https://doi.org/10.1007/978-3-030-01231-1_42 |
| [45] | Labbé, Y., Carpentier, J., Aubry, M., et al. (2020) Cosypose: Consistent Multi-View Multi-Object 6d Pose Estimation. In: Vedaldi, A., Bischof, H., Brox, T. and Frahm, J.-M., Eds., Proceedings of the European Conference on Computer Vision, Springer, Cham, 574-591. https://doi.org/10.1007/978-3-030-58520-4_34 |
| [46] | He, Y., Sun, W., Huang, H., et al. (2020) Pvn3d: A Deep Point-Wise 3d Keypoints Voting Network for 6d of Pose Estimation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, 14-19 June 2020, 11632-11641. https://doi.org/10.1109/CVPR42600.2020.01165 |
| [47] | Chen, W., Duan, J., Basevi, H., et al. (2020) PointPoseNet: Point Pose Network for Robust 6D Object Pose Estimation. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Snowmass Village, 1-5 March 2020, 2824-2833. https://doi.org/10.1109/WACV45572.2020.9093272 |
| [48] | He, Y., Huang, H., Fan, H., et al. (2021) FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Nashville, 20-25 June 2021, 3003-3013. https://doi.org/10.1109/CVPR46437.2021.00302 |
| [49] | Wu, Y., Zand, M., Etemad, A., et al. (2021) Vote from the Center: 6 DoF Pose Estimation in RGB-D Images by Radial Keypoint Voting. 17th European Conference, Tel Aviv, 23-27 October 2022, 335-352.
https://doi.org/10.1007/978-3-031-20080-9_20 |
| [50] | Wang, C., Xu, D., Zhu, Y., et al. (2019) Densefusion: 6d Object Pose Estimation by Iterative Dense Fusion. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Long Beach, 15-20 June 2019, 3343-3352. https://doi.org/10.1109/CVPR.2019.00346 |
| [51] | Wada, K., Sucar, E., James, S., et al. (2020) Morefusion: Multi-Object Reasoning for 6d Pose Estimation from Volumetric Fusion. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, 14-19 June 2020, 14540-14549. https://doi.org/10.1109/CVPR42600.2020.01455 |
| [52] | 张磊, 徐孝彬, 曹晨飞, 等. 基于动态特征剔除图像与点云融合的机器人位姿估计方法[J/OL]. 中国激光, 2022, 49(6): 126-137. |
| [53] | Chen, W., Jia, X., Chang, H.J., et al. (2020) G2l-net: Global to Local Network for Real-Time 6d Pose Estimation with Embedding Vector Features. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, 14-19 June 2020, 4233-4242. https://doi.org/10.1109/CVPR42600.2020.00429 |
| [54] | Hua, W., Zhou, Z., Wu, J., et al. (2021) Rede: End-to-End Object 6d Pose Robust Estimation Using Differentiable Outliers Elimination. IEEE Robotics and Automation Letters, 6, 2886-2893.
https://doi.org/10.1109/LRA.2021.3062304 |
