As a work exploring the existing trade-off between accuracy and efficiency in the context of point cloud processing, Point Transformer V3 (PTV3) has made significant advancements in computational efficiency through its innovative point cloud serialization strategy. However, this optimization for computational efficiency comes at the cost of sacrificing high-fidelity perception of fine-grained local geometric structures, thereby introducing a limitation termed “geometric information vacuum” in the model. To address this issue, our work proposes a coordinate-aware feature activation module, which enhances the model’s sensitivity to spatial locations by dynamically calibrating feature channel responses using the 3D absolute coordinates of points during the encoder stage. Furthermore, our work designs a position-aware upsampling mechanism that accurately restores the geometric details smoothed out during downsampling by learning a feature compensation term associated with the relative positions of points within voxels during the decoder stage. Experiments on 3D point cloud segmentation on S3DIS and ScanNe t v2 show that the Geo-PT model proposed in this study achieves better performance than PTv3 with negligible additional computational cost.
References
[1]
Qi, C.R., Su, H., Mo, K. and Guibas, L.J. (2017) PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. arXiv: 1612.00593.
[2]
Guo, M., Cai, J., Liu, Z., Mu, T., Martin, R.R. and Hu, S. (2021) PCT: Point Cloud Transformer. ComputationalVisualMedia, 7, 187-199. https://doi.org/10.1007/s41095-021-0229-5
[3]
Wang, P. (2023) OctFormer: Octree-Based Transformers for 3D Point Clouds. ACMTransactionsonGraphics, 42, 1-11. https://doi.org/10.1145/3592131
[4]
Wu, X., Jiang, L., Wang, P., Liu, Z., Liu, X., Qiao, Y., etal. (2024) Point Transformer V3: Simpler, Faster, Stronger. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 16-22 June 2024, 4840-4851. https://doi.org/10.1109/cvpr52733.2024.00463
[5]
Wu, X., Lao, Y., Jiang, L., Liu, X. and Zhao, H. (2022) Point Transformer V2: Grouped Vector Attention and Partition-Based Pooling. arXiv: 2210.05666.
[6]
Zhao, H., Jiang, L., Jia, J., Torr, P. and Koltun, V. (2021) Point Transformer. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, 10-17 October 2021, 16239-16248. https://doi.org/10.1109/iccv48922.2021.01595
[7]
Vafeiadis, T., Kolokas, N., Dimitriou, N., Zacharaki, A., Yildirim, M., Selvi, H.G., etal. (2022) A Comparison of 2DCNN Network Architectures and Boosting Techniques for Regression-Based Textile Whiteness Estimation. SimulationModellingPracticeandTheory, 114, Article ID: 102400. https://doi.org/10.1016/j.simpat.2021.102400
[8]
Su, H., Maji, S., Kalogerakis, E. and Learned-Miller, E. (2015) Multi-View Convolutional Neural Networks for 3D Shape Recognition. 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, 7-13 December 2015, 945-953. https://doi.org/10.1109/iccv.2015.114
[9]
Kanezaki, A., Matsushita, Y. and Nishida, Y. (2018) RotationNet: Joint Object Categorization and Pose Estimation Using MultiViews from Unsupervised Viewpoints. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-23 June 2018, 5010-5019. https://doi.org/10.1109/cvpr.2018.00526
[10]
Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J. and Beijbom, O. (2019) PointPillars: Fast Encoders for Object Detection from Point Clouds. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 12689-12697. https://doi.org/10.1109/cvpr.2019.01298
[11]
Riegler, G., Ulusoy, A.O. and Geiger, A. (2017) OctNet: Learning Deep 3D Representations at High Resolutions. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 21-26 July 2017, 6620-6629 https://doi.org/10.1109/cvpr.2017.701
[12]
Choy, C., Gwak, J. and Savarese, S. (2019) 4D Spatio-Temporal Convnets: Minkowski Convolutional Neural Networks. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 3070-3079. https://doi.org/10.1109/cvpr.2019.00319
[13]
Li, Y., Bu, R., Sun, M., Wu, W., Di, X. and Chen, B. (2018) PointCNN: Convolution On X-Transformed Points. arXiv: 1801.07791.
[14]
Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M. and Solomon, J.M. (2019) Dynamic Graph CNN for Learning on Point Clouds. ACMTransactionsonGraphics, 38, 1-12. https://doi.org/10.1145/3326362
[15]
Thomas, H., Qi, C.R., Deschaud, J., Marcotegui, B., Goulette, F. and Guibas, L. (2019) KPConv: Flexible and Deformable Convolution for Point Clouds. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, 27 October-2 November 2019, 6410-6419. https://doi.org/10.1109/iccv.2019.00651
[16]
Pang, Y., Wang, W., Tay, F.E.H., Liu, W., Tian, Y. and Yuan, L. (2022) Masked Autoencoders for Point Cloud Self-Supervised Learning. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M. and Hassner, T., Eds., Computer Vision—ECCV 2022, Springer, 604-621. https://doi.org/10.1007/978-3-031-20086-1_35
[17]
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., etal. (2021) Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, 10-17 October 2021, 9992-10002. https://doi.org/10.1109/iccv48922.2021.00986
[18]
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., et al. (2021) An Image Is Worth 16 × 16 Words: Transformers for Image Recognition at Scale. arXiv: 2010.11929.
[19]
Liu, Z., Yang, X., Tang, H., Yang, S. and Han, S. (2023) FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformer. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, 17-24 June 2023, 1200-1211. https://doi.org/10.1109/cvpr52729.2023.00122
[20]
Zhao, H., Jiang, L., Fu, C. and Jia, J. (2019) PointWeb: Enhancing Local Neighborhood Features for Point Cloud Processing. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 5560-5568. https://doi.org/10.1109/cvpr.2019.00571
[21]
Huang, H., Zhang, Y. and Ren, P. (2025) KernelDNA: Dynamic Kernel Sharing via Decoupled Naive Adapters. arXiv: 2503.23379.
[22]
Komarichev, A., Zhong, Z. and Hua, J. (2019) A-CNN: Annularly Convolutional Neural Networks on Point Clouds. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 7413-7422. https://doi.org/10.1109/cvpr.2019.00760
[23]
Ran, H., Liu, J. and Wang, C. (2022) Surface Representation for Point Clouds. arXiv: 2205.05740.
[24]
Li, R., Li, X., Fu, C., Cohen-Or, D. and Heng, P. (2019) PU-GAN: A Point Cloud Upsampling Adversarial Network. 2019 IEEE/CVFInternationalConferenceonComputerVision (ICCV), Seoul, 27 October-2 November 2019, 7202-7211. https://doi.org/10.1109/iccv.2019.00730
[25]
Yu, L., Li, X., Fu, C., Cohen-Or, D. and Heng, P. (2018) PU-Net: Point Cloud Upsampling Network. 2018 IEEE/CVFConferenceonComputerVisionandPatternRecognition, Salt Lake City, 18-23 June 2018, 2790-2799. https://doi.org/10.1109/cvpr.2018.00295
[26]
Liu, R., Lehman, J., Molino, P., Such, F.P., Frank, E., Sergeev, A. and Yosinski, J. (2018) An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution. arXiv: 1807.03247.
[27]
Wang, L., Huang, Y., Hou, Y., Zhang, S. and Shan, J. (2019) Graph Attention Convolution for Point Cloud Semantic Segmentation. 2019 IEEE/CVFConferenceonComputerVisionandPatternRecognition (CVPR), Long Beach, 15-20 June 2019, 10288-10297. https://doi.org/10.1109/cvpr.2019.01054
[28]
Hu, J., Shen, L., Albanie, S., Sun, G. and Wu, E. (2019) Squeeze-and-Excitation Networks. arXiv: 1709.01507.
[29]
Qi, C.R., Yi, L., Su, H. and Guibas, L.J. (2017) PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. arXiv: 1706.02413.
[30]
Qian, G., Li, Y., Peng, H., Mai, J., Hammoud, H.A.A.K., Elhoseiny, M. and Ghanem, B. (2022) PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies. arXiv: 2206.04670.
[31]
Dai, A., Chang, A.X., Savva, M., Halber, M., Funkhouser, T. and Niessner, M. (2017) ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes. 2017 IEEEConferenceonComputerVisionandPatternRecognition (CVPR), Honolulu, 21-26 July 2017, 2432-2443. https://doi.org/10.1109/cvpr.2017.261
[32]
Armeni, I., Sener, O., Zamir, A.R., Jiang, H., Brilakis, I., Fischer, M., etal. (2016) 3D Semantic Parsing of Large-Scale Indoor Spaces. 2016 IEEEConferenceonComputerVisionandPatternRecognition (CVPR), Las Vegas, 27-30 June 2016, 1534-1543. https://doi.org/10.1109/cvpr.2016.170
[33]
Tchapmi, L., Choy, C., Armeni, I., Gwak, J. and Savarese, S. (2017) SEGCloud: Semantic Segmentation of 3D Point Clouds. 2017 InternationalConferenceon3DVision (3DV), Qingdao, 10-12 October 2017, 537-547. https://doi.org/10.1109/3dv.2017.00067
[34]
Rozenberszki, D., Litany, O. and Dai, A. (2022) Language-Grounded Indoor 3D Semantic Segmentation in the Wild. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M. and Hassner, T., Eds., Computer Vision—ECCV 2022, Springer, 125-141. https://doi.org/10.1007/978-3-031-19827-4_8
[35]
Lai, X., Liu, J., Jiang, L., Wang, L., Zhao, H., Liu, S., etal. (2022) Stratified Transformer for 3D Point Cloud Segmentation. 2022 IEEE/CVFConferenceonComputerVisionandPatternRecognition (CVPR), New Orleans, 18-24 June 2022, 8490-8499. https://doi.org/10.1109/cvpr52688.2022.00831
[36]
Yang, Y.Q., Guo, Y.X., Xiong, J.Y., Liu, Y., Pan, H., Wang, P.S., Tong, X. and Guo, B. (2023) Swin3D: A Pretrained Transformer Backbone for 3D Indoor Scene Understanding. arXiv: 2304.06906.
[37]
Dai, Z., Liu, H., Le, Q.V. and Tan, M. (2021) CoAtNet: Marrying Convolution and Attention for All Data Sizes. arXiv: 2106.04803.
[38]
Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N. and Liang, J. (2018) UNet++: A Nested U-Net Architecture for Medical Image Segmentation. In: Stoyanov, D., et al., Eds., Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer, 3-11. https://doi.org/10.1007/978-3-030-00889-5_1
[39]
Huang, H., Lin, L., Tong, R., Hu, H., Zhang, Q., Iwamoto, Y., etal. (2020) UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation. ICASSP 2020—2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, 4-8 May 2020, 1055-1059. https://doi.org/10.1109/icassp40776.2020.9053405
