|
|
多重注意力非局部特征融合的序列影像–激光点云深度补全
|
Abstract:
有效且准确的深度信息能够精确感知场景的几何结构。目前主要采用的是激光雷达传感器。但由于其扫描线数有限，深度信息的稠密度非常低。由此提出深度补全任务(Depth Completion)，基于给定的稀疏深度值来估计稠密深度信息。本文针对先前深度补全存在的边界模糊和混合深度的问题，提出一种多重注意力非局部特征融合的序列影像–激光点云深度补全模型。该模型通过融合人序列影像和激光点云，实现了多模态数据的优势互补，通过网络学习到更丰富的特征。实验表明该方法可以有效降低误差，提高深度补全的效果，极大地提高例如目标识别、目标跟踪、路径规划等任务的精度。
Effective and accurate depth information can accurately perceive the geometry of the scene. At present, LiDAR sensors are mainly used. But due to the limited number of scanning lines, the density of depth information is very low. Therefore, a depth completion task is proposed to estimate the dense depth information based on the given sparse depth map. In order to solve the problems of boundary blur and mixed depth in previous depth completion tasks, this paper proposes a depth completion network between single image and sparse LiDAR data with multiple attention and non-local feature. The fusion of image and LiDAR data realizes the complementary advantages of multi-modal data, and richer features can be learned through the network. Experiments show that this network can effectively reduce the error, improving the effect of depth completion, and greatly improve the accuracy of tasks such as target recognition, target tracking, path planning and so on.
| [1] | Cui, Z., Heng, L., Yeo, Y.C., Geiger, A., Pollefeys, M. and Sattler, T. (2019) Real-Time Dense Mapping for Self-Driving Vehicles Using Fisheye Cameras. International Conference on Robotics and Automation (ICRA), Montreal, 20-24 May 2019, 6087-6093. https://doi.org/10.1109/ICRA.2019.8793884 |
| [2] | Song, X., Dai, Y., Zhou, D., Liu, L., Li, W., Li, H. and Yang, R. (2020) Channel Attention Based Iterative Residual Learning for Depth Map Super-Resolution. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 13-19 June 2020, 5630-5639. https://doi.org/10.1109/CVPR42600.2020.00567 |
| [3] | Armbrüster, C., Wolter, M., Kuhlen, T., Spijkers, W. and Fimm, B. (2008) Depth Perception in Virtual Reality: Distance Estimations in Peri- and Extrapersonal Space. Cyberpsychology Behavior, 11, 9-15.
https://doi.org/10.1089/cpb.2007.9935 |
| [4] | Park, J., Joo, K., Hu, Z., Liu, C.-K. and So Kweon, I. (2020) Non-Local Spatial Propagation Network for Depth Completion. European Conference on Computer Vision (ECCV), 120-136. https://doi.org/10.1007/978-3-030-58601-0_8 |
| [5] | Uhrig, J., Schneider, N., Schneider, L., Franke, U., Brox, T. and Geiger, A. (2017) Sparsity Invariant CNNs. International Conference on 3D Vision (3DV), Qingdao, 10-12 October 2017, 11-20. https://doi.org/10.1109/3DV.2017.00012 |
| [6] | Chodosh, N., Wang, C. and Lucey, S. (2018) Deep Convolutional Compressed Sensing for Lidar Depth Completion. Asian Conference on Computer Vision, 499-513. https://doi.org/10.1007/978-3-030-20887-5_31 |
| [7] | Ma, F. and Karaman, S. (2018) Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image. International Conference on Robotics and Automation (ICRA), Brisbane, 21-25 May 2018, 4796-4803.
https://doi.org/10.1109/ICRA.2018.8460184 |
| [8] | Cheng, X., Wang, P. and Yang, R. (2018) Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network. European Conference on Computer Vision (ECCV), 103-119.
https://doi.org/10.1007/978-3-030-01270-0_7 |
| [9] | Liu, S., De Mello, S., Gu, J., Zhong, G., Yang, M.-H. and Kautz, J. (2017) Learning Affinity via Spatial Propagation Networks. Advances in Neural Information Processing Systems, 30. |
| [10] | Liu, J.-J., Hou, Q., Cheng, M.-M., Wang, C. and Feng, J. (2020) Improving Convolutional Networks with Self-Calibrated Convolutions. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 13-19 June 2020, 10093-10102. https://doi.org/10.1109/CVPR42600.2020.01011 |
| [11] | Woo, S., Park, J., Lee, J.-Y. and Kweon, I.S. (2018) CBAM: Convolutional Block Attention Module. European Conference on Computer Vision (ECCV), 3-19. https://doi.org/10.1007/978-3-030-01234-2_1 |
| [12] | Qiu, J., Cui, Z., Zhang, Y., Zhang, X., Liu, S., Zeng, B. and Pollefeys, M. (2019) DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse Lidar Data and Single Color Image. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 3313-3322.
https://doi.org/10.1109/CVPR.2019.00343 |
| [13] | Xu, Y., Zhu, X., Shi, J., Zhang, G., Bao, H. and Li, H. (2019) Depth Completion from Sparse Lidar Data with Depth-Normal Constraints. IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, 27 October-2 November 2019, 2811-2820. https://doi.org/10.1109/ICCV.2019.00290 |
