|
|
卫星可见光多视角影像的表面重构综述
|
Abstract:
在卫星多视角影像三维重构领域的关键技术领域中，主流的方法主要为基于立体匹配的点云生成与表面重构方法，包括三角化方法、模板先验、建模先验及深度学习等方法。而对于新兴的重构技术神经辐射场，特别探讨了其原理与工作流程，及其在稀疏视角和复杂光照条件下的优势与研究进展。最后，文章总结了未来的发展趋势，并提出了未来的研究方向，涵盖了3DGS大规模场景重建技术、语义感知辅助建模等方面。
This paper reviews key technologies in satellite image 3D reconstruction, focusing on stereo matching-based point cloud generation, triangulation, modeling priors, and deep learning methods. It discusses the principles and advantages of Neural Radiance Fields (NeRF) in sparse views and complex lighting. The role of geometric information extraction in surface reconstruction and the current research status of traditional and deep learning methods are also highlighted. Future research directions include 3DGS for large-scale scene reconstruction and so on.
| [1] | Xu, C., Xu, J., Huang, T., Zhang, H., Mei, L., Zhang, X., et al. (2023) Progressive Matching Method of Aerial-Ground Remote Sensing Image via Multi-Scale Context Feature Coding. International Journal of Remote Sensing, 44, 5876-5895. https://doi.org/10.1080/01431161.2023.2255352 |
| [2] | Zhao, L., Wang, H., Zhu, Y. and Song, M. (2023) A Review of 3D Reconstruction from High-Resolution Urban Satellite Images. International Journal of Remote Sensing, 44, 713-748. https://doi.org/10.1080/01431161.2023.2169844 |
| [3] | Penczek, P.A. (2010) Fundamentals of Three-Dimensional Reconstruction from Projections. In: Jensen, G.J., Ed., Methods in Enzymology, Elsevier, 1-33. https://doi.org/10.1016/s0076-6879(10)82001-4 |
| [4] | Xu, W., Dai, W., Xue, D., Zheng, Z., Li, C., Zou, J. and Xiong, H. (2024) Point Cloud Denoising with Fine-Granularity Dynamic Graph Convolutional Networks. https://doi.org/10.48550/arXiv.2411.14158 |
| [5] | Berger, M., Tagliasacchi, A., Seversky, L., Alliez, P., Levine, J., et al. (2014) State of the Art in Surface Reconstruction from Point Clouds. Eurographics 2014-State of the Art Reports, Strasbourg, April 2014, 161-185. |
| [6] | Khatamian, A. and Arabnia, H.R. (2016) Survey on 3D Surface Reconstruction. Journal of Information Processing Systems, 12, 338-357. https://doi.org/10.3745/JIPS.01.0010 |
| [7] | Elharrouss, O., Hassine, K., Zayyan, A., et al. (2023) 3D Point Cloud for Objects and Scenes Classification, Recognition, Segmentation, and Reconstruction: A Review. Cloud Computing and Data Science, 4, 134-160. https://doi.org/10.37256/ccds.4220232722 |
| [8] | Huang, Z., Wen, Y., Wang, Z., Ren, J. and Jia, K. (2024) Surface Reconstruction from Point Clouds: A Survey and a Benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46, 9727-9748. https://doi.org/10.1109/tpami.2024.3429209 |
| [9] | Sulzer, R., Marlet, R., Vallet, B. and Landrieu, L. (2025) A Survey and Benchmark of Automatic Surface Reconstruction from Point Clouds. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47, 2000-2019. https://doi.org/10.1109/tpami.2024.3510932 |
| [10] | Cohen-Steiner, D. and Da, F. (2004) A Greedy Delaunay-Based Surface Reconstruction Algorithm. The Visual Computer, 20, 4-16. https://doi.org/10.1007/s00371-003-0217-z |
| [11] | Bernardini, F., Mittleman, J., Rushmeier, H., Silva, C. and Taubin, G. (1999) The Ball-Pivoting Algorithm for Surface Reconstruction. IEEE Transactions on Visualization and Computer Graphics, 5, 349-359. https://doi.org/10.1109/2945.817351 |
| [12] | Sharp, N. and Ovsjanikov, M. (2020) PointTriNet: Learned Triangulation of 3D Point Sets. 16th European Conference Computer Vision—ECCV 2020, Glasgow, 23-28 August 2020, 762-778. https://doi.org/10.1007/978-3-030-58592-1_45 |
| [13] | Luo, Y., Mi, Z. and Tao, W. (2021) Deepdt: Learning Geometry from Delaunay Triangulation for Surface Reconstruction. Proceedings of the AAAI Conference on Artificial Intelligence, 35, 2277-2285. https://doi.org/10.1609/aaai.v35i3.16327 |
| [14] | Zhang, C., Yuan, G. and Tao, W. (2023) Dmnet: Delaunay Meshing Network for 3D Shape Representation. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, 1-6 October 2023, 14372-14382. https://doi.org/10.1109/iccv51070.2023.01326 |
| [15] | Schnabel, R., Wahl, R. and Klein, R. (2007) Efficient RANSAC for Point‐Cloud Shape Detection. Computer Graphics Forum, 26, 214-226. https://doi.org/10.1111/j.1467-8659.2007.01016.x |
| [16] | Nan, L. and Wonka, P. (2017) Polyfit: Polygonal Surface Reconstruction from Point Clouds. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 22-29 October 2017, 2353-2361. https://doi.org/10.1109/iccv.2017.258 |
| [17] | Zhu, Z., Zheng, A., Iglesias, A., Wang, S., Xia, Y., Chaudhry, E., et al. (2022) PDE Patch-Based Surface Reconstruction from Point Clouds. Journal of Computational Science, 61, Article ID: 101647. https://doi.org/10.1016/j.jocs.2022.101647 |
| [18] | Wei, M., Wu, Q.Y., Zheng, J.M., Rezatofighi, H. and Cai, J.F. (2024) Normal-GS: 3D Gaussian Splatting with Normal-Involved Rendering. |
| [19] | Pauly, M., Mitra, N.J., Giesen, J., Gross, M.H. and Guibas, L.J. (2005) Example-Based 3D Scan Completion. Proceedings Symposium on Geometry Processing, Vienna, 4-6 July 2005, 23-32. |
| [20] | Shen, C., Fu, H., Chen, K. and Hu, S. (2012) Structure Recovery by Part Assembly. ACM Transactions on Graphics, 31, Article No. 180. https://doi.org/10.1145/2366145.2366199 |
| [21] | Kuo, W., Angelova, A., Lin, T. and Dai, A. (2021) Patch2CAD: Patchwise Embedding Learning for In-the-Wild Shape Retrieval from a Single Image. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, 10-17 October 2021, 12569-12579. https://doi.org/10.1109/iccv48922.2021.01236 |
| [22] | Siddiqui, Y., Thies, J., Ma, F., Shan, Q., Niesner, M. and Dai, A. (2021) RetrievalFuse: Neural 3D Scene Reconstruction with a Database. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 11-17 October 2021, 12548-12557. https://doi.org/10.1109/iccv48922.2021.01234 |
| [23] | Kazhdan, M., Bolitho, M. and Hoppe, H. (2006) Poisson Surface Reconstruction. In: Polthier, K. and Sheffer, A., Eds., Eurographics Symposium on Geometry Processing, The Eurographics Association, 12569-12579. |
| [24] | Petrov, D., Gadelha, M., Měch, R. and Kalogerakis, E. (2024) ANISE: Assembly-Based Neural Implicit Surface Reconstruction. IEEE Transactions on Visualization and Computer Graphics, 30, 4514-4526. https://doi.org/10.1109/tvcg.2023.3265306 |
| [25] | Ren, X., Cao, C., Fu, Y. and Xue, X. (2024) Improving Neural Surface Reconstruction with Feature Priors from Multi-View Images. 18th European Conference Computer Vision—ECCV 2024, Milan, 29 September-4 October 2024, 445-463. https://doi.org/10.1007/978-3-031-73636-0_26 |
| [26] | Park, J.J., Florence, P., Straub, J., Newcombe, R. and Lovegrove, S. (2019) DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 165-174. https://doi.org/10.1109/cvpr.2019.00025 |
| [27] | Beauchamp, M., Fablet, R., Benaichouche, S., Tandeo, P., Desassis, N., et al. (2024) Neural Variational Data Assimilation with Uncertainty Quantification Using SPDE Priors. |
| [28] | Zhang, Y., Wang, Z., Li, X., Yuan, Y., Zhang, C., Sun, X., Zhong, Z. and Wang, J. (2024) DiffBody: Human body Restoration by Imagining with Generative Diffusion Prior. https://arxiv.org/abs/2404.03642 |
| [29] | Charles, R.Q., Su, H., Kaichun, M. and Guibas, L.J. (2017) PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 21-26 July 2017, 77-85. https://doi.org/10.1109/cvpr.2017.16 |
| [30] | Chen, Z. and Zhang, H. (2019) Learning Implicit Fields for Generative Shape Modeling. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 5932-5941. https://doi.org/10.1109/cvpr.2019.00609 |
| [31] | Zhao, H., Jiang, L., Jia, J., Torr, P. and Koltun, V. (2021) Point Transformer. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, 10-17 October 2021, 16239-16248. https://doi.org/10.1109/iccv48922.2021.01595 |
| [32] | Wu, X., Lao, Y., Jiang, L., Liu, X. and Zhao, H. (2022) Point Transformer V2: Grouped Vector Attention and Parti-tion-based Pooling (No. arXiv:2210.05666). arXiv. https://doi.org/10.48550/arXiv.2210.05666 |
| [33] | Ma, X., Qin, C., You, H., Ran, H. and Fu, Y. (2022). Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework. https://doi.org/10.48550/arXiv.2202.07123 |
| [34] | Lai, X., Liu, J., Jiang, L., Wang, L., Zhao, H., Liu, S., et al. (2022) Stratified Transformer for 3D Point Cloud Segmentation. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, 18-24 June 2022, 8490-8499. https://doi.org/10.1109/cvpr52688.2022.00831 |
| [35] | Lai, X., Liu, J., Jiang, L., Wang, L., Zhao, H., Liu, S., et al. (2022) Stratified Transformer for 3D Point Cloud Segmentation. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, 18-24 June 2022, 8490-8499. https://doi.org/10.1109/cvpr52688.2022.00831 |
| [36] | Qi, C.R., Yi, L., Su, H. and Guibas, L.J. (2017) PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, Curran Associates Inc., 5105-5114. |
| [37] | Thomas, H., Qi, C.R., Deschaud, J., Marcotegui, B., Goulette, F. and Guibas, L. (2019) KPConv: Flexible and Deformable Convolution for Point Clouds. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, 27 October-2 November 2019, 6410-6419. https://doi.org/10.1109/iccv.2019.00651 |
| [38] | Peng, S., Niemeyer, M., Mescheder, L., Pollefeys, M. and Geiger, A. (2020) Convolutional Occupancy Networks. Computer Vision—ECCV 2020 16th European Conference, Glasgow, 23-28 August 2020, 523-540. https://doi.org/10.1007/978-3-030-58580-8_31 |
| [39] | Chabra, R., Lenssen, J.E., Ilg, E., Schmidt, T., Straub, J., Lovegrove, S., et al. (2020) Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction. Computer Vision—ECCV 2020 16th European Conference, Glasgow, 23-28 August 2020, 608-625. https://doi.org/10.1007/978-3-030-58526-6_36 |
| [40] | Jiang, C., Sud, A., Makadia, A., Huang, J., NieBner, M. and Funkhouser, T. (2020) Local Implicit Grid Representations for 3D Scenes. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 14-19 June 2020, 6000-6009. https://doi.org/10.1109/cvpr42600.2020.00604 |
| [41] | Erler, P., Guerrero, P., Ohrhallinger, S., Mitra, N.J. and Wimmer, M. (2020) Points2Surf Learning Implicit Surfaces from Point Clouds. Computer Vision—ECCV 2020 16th European Conference, Glasgow, 23-28 August 2020, 108-124. https://doi.org/10.1007/978-3-030-58558-7_7 |
| [42] | Qian, G., Li, Y., Peng, H., Mai, J., Hammoud, H.A.A.K., Elhoseiny, M. and Ghanem, B. (2022) PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies. https://doi.org/10.48550/arXiv.2206.04670 |
| [43] | Boulch, A. and Marlet, R. (2022) POCO: Point Convolution for Surface Reconstruction. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, 18-24 June 2022, 6292-6304. https://doi.org/10.1109/cvpr52688.2022.00620 |
| [44] | Yuan, G., Fu, Q., Mi, Z., Luo, Y. and Tao, W. (2023) SSRNet: Scalable 3D Surface Reconstruction Network. IEEE Transactions on Visualization and Computer Graphics, 29, 4906-4919. https://doi.org/10.1109/tvcg.2022.3193406 |
| [45] | Ouasfi, A. and Boukhayma, A. (2023) Mixing-Denoising Generalizable Occupancy Networks. https://doi.org/10.48550/arXiv.2311.12125 |
| [46] | Ouasfi, A. and Boukhayma, A. (2024) Unsupervised Occupancy Learning from Sparse Point Cloud. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 16-22 June 2024, 21729-21739. https://doi.org/10.1109/cvpr52733.2024.02053 |
| [47] | Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R. and Ng, R. (2020) NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. Computer Vision—ECCV 2020 16th European Conference, Glasgow, 23-28 August 2020, 405-421. https://doi.org/10.1007/978-3-030-58452-8_24 |
| [48] | Xie, S., Zhang, L., Jeon, G. and Yang, X. (2023) Remote Sensing Neural Radiance Fields for Multi-View Satellite Photogrammetry. Remote Sensing, 15, Article No. 3808. https://doi.org/10.3390/rs15153808 |
| [49] | Zhou, X., Wang, Y., Lin, D., Cao, Z., Li, B. and Liu, J. (2024) SatelliteRF: Accelerating 3D Reconstruction in Multi-View Satellite Images with Efficient Neural Radiance Fields. Applied Sciences, 14, Article No. 2729. https://doi.org/10.3390/app14072729 |
| [50] | Behari, N., Dave, A., Tiwary, K., Yang, W. and Raskar, R. (2024) SUNDIAL: 3D Satellite Understanding through Direct, Ambient, and Complex Lighting Decomposition. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, 17-18 June 2024, 522-532. https://doi.org/10.1109/cvprw63382.2024.00057 |
| [51] | Derksen, D. and Izzo, D. (2021) Shadow Neural Radiance Fields for Multi-View Satellite Photogrammetry. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Nashville, 19-25 June 2021, 1152-1161. https://doi.org/10.1109/cvprw53098.2021.00126 |
| [52] | Adams, C., Lopez-Francos, I., Kints, E.V. and Hammond, A. (n.d.) A Summary of Neural Radiance Fields for Shadow Removal and Relighting of Satellite Imagery. https://digitalcommons.usu.edu/smallsat/ |
| [53] | Sun, W., Lu, Y., Zhang, Y., Zhu, Y., Zhou, Z., Wang, J., et al. (2024) Neural Radiance Fields for Multi-View Satellite Photogrammetry Leveraging Intrinsic Decomposition. IGARSS 2024-2024 IEEE International Geoscience and Remote Sensing Symposium, Athens, 7-12 July 2024, 4631-4635. https://doi.org/10.1109/igarss53475.2024.10641455 |
