|
|
结构感知的自监督点云上采样网络
|
Abstract:
现有的多分支自监督点云上采样框架的设计机制通常没有关注到各分支间的结构一致性问题，特征表示能力不足。为解决上述问题，提出一种结构感知的自监督点云上采样网络。特征提取阶段，模型在低层特征处加入结构感知模块，重新组合特征点云，来提升结构感知的能力。同时添加自注意力机制，整合不同分支的特征。特征扩展阶段，提出使用插值点云来指导上采样过程，以提高保持原始点云形状的能力。在PUGAN数据集上进行实验，与基线模型SPU-Net相比，所提出模型在三个指标上均有所提升，其中P2F和HD两个指标提升显著。实验结果表明，模型可以生成具有较多几何细节、较高质量的密集点云。
The design mechanisms of existing multi-branch self-supervised point cloud upsampling frameworks usually do not pay attention to the structural consistency problem among branches, and the feature representation capability is insufficient. To solve the above problem, a structure-aware self-supervised point cloud upsampling network is proposed. In the feature extraction stage, the model adds a structure-aware module at the low-level features to recombine the feature point cloud to improve the ability of structure-awareness. At the same time, a self-attention mechanism is added to integrate features from different branches. In the feature expansion stage, the use of interpolated point clouds is proposed to guide the upsampling process to improve the ability to maintain the shape of the original point cloud. Experiments are conducted on the PUGAN dataset. Compared to the baseline model SPU-Net, the proposed model improved on all three metrics. Two of the metrics, P2F and HD, improved significantly. Experimental results show that the model can generate dense point clouds with more geometric details and higher quality.
| [1] | Cai, Q., Pan, Y., Yao, T. and Mei, T. (2022) 3D Cascade RCNN: High Quality Object Detection in Point Clouds. IEEE Transactions on Image Processing, 31, 5706-5719. https://doi.org/10.1109/tip.2022.3201469 |
| [2] | An, P., Liang, J., Ma, T., Chen, Y., Wang, L. and Ma, J. (2023) ProUDA: Progressive Unsupervised Data Augmentation for Semi-Supervised 3D Object Detection on Point Cloud. Pattern Recognition Letters, 170, 64-69. https://doi.org/10.1016/j.patrec.2023.04.002 |
| [3] | Xie, G., Li, Y., Wang, Y., Li, Z. and Qu, H. (2023) 3D Point Cloud Object Detection Algorithm Based on Temporal Information Fusion and Uncertainty Estimation. Remote Sensing, 15, Article No. 2986. https://doi.org/10.3390/rs15122986 |
| [4] | Zhao, Y., Hui, L. and Xie, J. (2021) SSPU-Net: Self-Supervised Point Cloud Upsampling via Differentiable Rendering. Proceedings of the 29th ACM International Conference on Multimedia, 20-24 October 2021, 2214-2223. https://doi.org/10.1145/3474085.3475381 |
| [5] | Liu, X., Liu, X., Liu, Y. and Han, Z. (2022) Spu-Net: Self-Supervised Point Cloud Upsampling by Coarse-to-Fine Reconstruction with Self-Projection Optimization. IEEE Transactions on Image Processing, 31, 4213-4226. https://doi.org/10.1109/tip.2022.3182266 |
| [6] | Charles, R.Q., Su, H., Kaichun, M. and Guibas, L.J. (2017) PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 1-6 October 2017, 77-85. https://doi.org/10.1109/cvpr.2017.16 |
| [7] | Charles, R.Q., Yi, L., Su, H. and Guibas, L.J. (2017) PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. International Conference on Neural Information Processing Systems (NIPS), Long Beach, 4-9 December 2017, 5099-5108. |
| [8] | Yu, L., Li, X., Fu, C., Cohen-Or, D. and Heng, P. (2018) PU-Net: Point Cloud Upsampling Network. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-22 June 2018, 2790-2799. https://doi.org/10.1109/cvpr.2018.00295 |
| [9] | Yifan, W., Wu, S., Huang, H., Cohen-Or, D. and Sorkine-Hornung, O. (2019) Patch-Based Progressive 3D Point Set Upsampling. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 5951-5960. https://doi.org/10.1109/cvpr.2019.00611 |
| [10] | Li, R., Li, X., Fu, C., Cohen-Or, D. and Heng, P. (2019) PU-GAN: A Point Cloud Upsampling Adversarial Network. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, 27 October-2 November 2019, 7202-7211. https://doi.org/10.1109/iccv.2019.00730 |
| [11] | Qian, G., Abualshour, A., Li, G., Thabet, A. and Ghanem, B. (2021) PU-GCN: Point Cloud Upsampling Using Graph Convolutional Networks. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, 19-25 June 2021, 11678-11687. https://doi.org/10.1109/cvpr46437.2021.01151 |
| [12] | Liu, H., Yuan, H., Hou, J., Hamzaoui, R. and Gao, W. (2022) PUFA-GAN: A Frequency-Aware Generative Adversarial Network for 3D Point Cloud Upsampling. IEEE Transactions on Image Processing, 31, 7389-7402. https://doi.org/10.1109/tip.2022.3222918 |
| [13] | Ye, S., Chen, D., Han, S., Wan, Z. and Liao, J. (2022) Meta-PU: An Arbitrary-Scale Upsampling Network for Point Cloud. IEEE Transactions on Visualization and Computer Graphics, 28, 3206-3218. https://doi.org/10.1109/tvcg.2021.3058311 |
| [14] | Feng, W., Li, J., Cai, H., Luo, X. and Zhang, J. (2022) Neural Points: Point Cloud Representation with Neural Fields for Arbitrary Upsampling. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, 18-24 June 2022, 18612-18621. https://doi.org/10.1109/cvpr52688.2022.01808 |
| [15] | He, Y., Tang, D., Zhang, Y., Xue, X. and Fu, Y. (2023) Grad-PU: Arbitrary-Scale Point Cloud Upsampling via Gradient Descent with Learned Distance Functions. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, 18-22 June 2023, 5354-5363. https://doi.org/10.1109/cvpr52729.2023.00518 |
| [16] | Huang, H., Wu, S., Gong, M., Cohen-Or, D., Ascher, U. and Zhang, H. (2013) Edge-Aware Point Set Resampling. ACM Transactions on Graphics, 32, 1-12. https://doi.org/10.1145/2421636.2421645 |
| [17] | Liu, X., Han, Z., Wen, X., Liu, Y. and Zwicker, M. (2019) L2G Auto-Encoder: Understanding Point Clouds by Local-to-Global Reconstruction with Hierarchical Self-Attention. Proceedings of the 27th ACM International Conference on Multimedia, Nice, 21-25 October 2019, 989-997. https://doi.org/10.1145/3343031.3350960 |
| [18] | Zhao, W., Liu, X., Zhong, Z., Jiang, J., Gao, W., Li, G., et al. (2022) Self-Supervised Arbitrary-Scale Point Clouds Upsampling via Implicit Neural Representation. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, 18-24 June 2022, 1989-1997. https://doi.org/10.1109/cvpr52688.2022.00204 |
| [19] | Zhao, W., Liu, X., Zhai, D., Jiang, J. and Ji, X. (2023) Self-Supervised Arbitrary-Scale Implicit Point Clouds Upsampling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45, 12394-12407. https://doi.org/10.1109/tpami.2023.3287628 |
| [20] | Zou, Y.L., Zhang, Z.Z., Zhang, H., et al. (2020) PseudoSeg: Designing Pseudo Labels for Semantic Segmentation. Proceedings of International Conference on Learning Representations (ICLR). https://openreview.net/forum?id=-TwO99rbVRu |
| [21] | Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M. and Solomon, J.M. (2019) Dynamic Graph CNN for Learning on Point Clouds. ACM Transactions on Graphics, 38, 1-12. https://doi.org/10.1145/3326362 |
| [22] | Nguyen, T., Pham, Q., Le, T., Pham, T., Ho, N. and Hua, B. (2021) Point-Set Distances for Learning Representations of 3D Point Clouds. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, 10-17 October 2021, 10458-10467. https://doi.org/10.1109/iccv48922.2021.01031 |
| [23] | Jesorsky, O., Kirchberg, K.J. and Frischholz, R.W. (2001) Robust Face Detection Using the Hausdorff Distance. 3rd International Conference, AVBPA 2001, Halmstad, 6-8 June 2001, 90-95. https://doi.org/10.1007/3-540-45344-x_14 |
| [24] | Liu, Y., Chen, R., Li, Y., Li, Y. and Tan, X. (2024) SPU-PMD: Self-Supervised Point Cloud Upsampling via Progressive Mesh Deformation. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 17-21 June 2024, 5188-5197. https://doi.org/10.1109/cvpr52733.2024.00496 |
