Cloud detection is a critical preprocessing step in remote sensing image processing, as the presence of clouds significantly affects the accuracy of remote sensing data and limits its applicability across various domains. This study presents an enhanced cloud detection method based on the U-Net architecture, designed to address the challenges of multi-scale cloud features and long-range dependencies inherent in remote sensing imagery. A Multi-Scale Dilated Attention (MSDA) module is introduced to effectively integrate multi-scale information and model long-range dependencies across different scales, enhancing the model’s ability to detect clouds of varying sizes. Additionally, a Multi-Head Self-Attention (MHSA) mechanism is incorporated to improve the model’s capacity for capturing finer details, particularly in distinguishing thin clouds from surface features. A multi-path supervision mechanism is also devised to ensure the model learns cloud features at multiple scales, further boosting the accuracy and robustness of cloud mask generation. Experimental results demonstrate that the enhanced model achieves superior performance compared to other benchmarked methods in complex scenarios. It significantly improves cloud detection accuracy, highlighting its strong potential for practical applications in cloud detection tasks.
References
[1]
Zhang, Y., Rossow, W.B., Lacis, A.A., Oinas, V. and Mishchenko, M.I. (2004) Calculation of Radiative Fluxes from the Surface to Top of Atmosphere Based on ISCCP and Other Global Data Sets: Refinements of the Radiative Transfer Model and the Input Data. Journal of Geophysical Research: Atmospheres, 109, D19105. https://doi.org/10.1029/2003jd004457
[2]
Irish, R.R., Barker, J.L., Goward, S.N. and Arvidson, T. (2006) Characterization of the Landsat-7 ETM+ Automated Cloud-Cover Assessment (ACCA) Algorithm. Photogrammetric Engineering & Remote Sensing, 72, 1179-1188. https://doi.org/10.14358/pers.72.10.1179
[3]
Zhu, Z., Wang, S. and Woodcock, C.E. (2015) Improvement and Expansion of the Fmask Algorithm: Cloud, Cloud Shadow, and Snow Detection for Landsats 4-7, 8, and Sentinel 2 Images. Remote Sensing of Environment, 159, 269-277. https://doi.org/10.1016/j.rse.2014.12.014
[4]
Francis, A., Sidiropoulos, P. and Muller, J. (2019) CloudFCN: Accurate and Robust Cloud Detection for Satellite Imagery with Deep Learning. Remote Sensing, 11, Article 2312. https://doi.org/10.3390/rs11192312
[5]
Jeppesen, J.H., Jacobsen, R.H., Inceoglu, F. and Toftegaard, T.S. (2019) A Cloud Detection Algorithm for Satellite Imagery Based on Deep Learning. Remote Sensing of Environment, 229, 247-259. https://doi.org/10.1016/j.rse.2019.03.039
[6]
Li, W., Zou, Z. and Shi, Z. (2020) Deep Matting for Cloud Detection in Remote Sensing Images. IEEE Transactions on Geoscience and Remote Sensing, 58, 8490-8502. https://doi.org/10.1109/tgrs.2020.2988265
[7]
Wu, X., Shi, Z. and Zou, Z. (2021) A Geographic Information-Driven Method and a New Large Scale Dataset for Remote Sensing Cloud/Snow Detection. ISPRS Journal of Photogrammetry and Remote Sensing, 174, 87-104. https://doi.org/10.1016/j.isprsjprs.2021.01.023
[8]
Zhang, J., Wang, H., Wang, Y., Zhou, Q. and Li, Y. (2021) Deep Network Based on up and down Blocks Using Wavelet Transform and Successive Multi-Scale Spatial Attention for Cloud Detection. Remote Sensing of Environment, 261, Article ID: 112483. https://doi.org/10.1016/j.rse.2021.112483
[9]
Yang, J., Guo, J., Yue, H., Liu, Z., Hu, H. and Li, K. (2019) CDnet: CNN-Based Cloud Detection for Remote Sensing Imagery. IEEE Transactions on Geoscience and Remote Sensing, 57, 6195-6211. https://doi.org/10.1109/tgrs.2019.2904868
[10]
Guo, J., Yang, J., Yue, H., Tan, H., Hou, C. and Li, K. (2021) CDnetV2: CNN-Based Cloud Detection for Remote Sensing Imagery with Cloud-Snow Coexistence. IEEE Transactions on Geoscience and Remote Sensing, 59, 700-713. https://doi.org/10.1109/tgrs.2020.2991398
[11]
Li, Z., Shen, H., Cheng, Q., Liu, Y., You, S. and He, Z. (2019) Deep Learning Based Cloud Detection for Medium and High Resolution Remote Sensing Images of Different Sensors. ISPRS Journal of Photogrammetry and Remote Sensing, 150, 197-212. https://doi.org/10.1016/j.isprsjprs.2019.02.017
[12]
Li, X., Yang, X., Li, X., Lu, S., Ye, Y. and Ban, Y. (2022) GCDB-UNet: A Novel Robust Cloud Detection Approach for Remote Sensing Images. Knowledge-Based Systems, 238, Article ID: 107890. https://doi.org/10.1016/j.knosys.2021.107890
[13]
Wu, K., Xu, Z., Lyu, X. and Ren, P. (2022) Cloud Detection with Boundary Nets. ISPRS Journal of Photogrammetry and Remote Sensing, 186, 218-231. https://doi.org/10.1016/j.isprsjprs.2022.02.010
[14]
Zhai, H. and Xue, L. (2024) AMCD-Net: An Effective Attention-Aided Multi-level Cloud Detection Network for Optical Satellite Imagery. IEEE Transactions on Geoscience and Remote Sensing, 62, 1-21. https://doi.org/10.1109/tgrs.2024.3372589
[15]
Martins, B.J., Arrais, J.M., Cerentini, A., Wangenheim, A.v., Neto, G.P.R. and Mantelli, S. (2023) Segmentation and Classification of Individual Clouds in Images Captured with Horizon-Aimed Cameras for Nowcasting of Solar Irradiance Absorption. American Journal of Climate Change, 12, 628-654. https://doi.org/10.4236/ajcc.2023.124027
[16]
Matsunobu, L.M., Pedro, H.T.C. and Coimbra, C.F.M. (2021) Cloud Detection Using Convolutional Neural Networks on Remote Sensing Images. Solar Energy, 230, 1020-1032. https://doi.org/10.1016/j.solener.2021.10.065
[17]
Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., et al. (2023) Swin-Unet: Unet-Like Pure Transformer for Medical Image Segmentation. In: Karlinsky, L., Michaeli, T. and Nishino, K., Eds., Computer Vision—ECCV 2022 Workshops, Springer Nature Switzerland, 205-218. https://doi.org/10.1007/978-3-031-25066-8_9
[18]
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J.M. and Luo, P. (2021) Seg-Former: Simple and Efficient Design for Semantic Segmentation with Transformers. Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS’21), Vancouver, 6-14 December 2021, 12077-12090.
[19]
Liu, X., Peng, H., Zheng, N., Yang, Y., Hu, H. and Yuan, Y. (2023) EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, 17-24 June 2023, 14420-14430. https://doi.org/10.1109/cvpr52729.2023.01386
[20]
Yeom, S.K., and von Klitzing, J. (2023) U-MixFormer: UNet-Like Transformer with Mix-Attention for Efficient Semantic Segmentation. arXiv: 2312.06272.
[21]
Ma, J., Li, F., and Wang, B. (2024) U-Mamba: Enhancing Long-Range Dependency for Biomedical Image Segmentation. arXiv: 2401.04722.
[22]
Liu, M., Dan, J., Lu, Z., Yu, Y., Li, Y., and Li, X. (2024) CM-UNet: Hybrid CNN-Mamba UNet for Remote Sensing Image Semantic Segmentation. arXiv: 2405.10530.
[23]
Jiao, J., Tang, Y., Lin, K., Gao, Y., Ma, A.J., Wang, Y., et al. (2023) Dilateformer: Multi-Scale Dilated Transformer for Visual Recognition. IEEE Transactions on Multimedia, 25, 8906-8919. https://doi.org/10.1109/tmm.2023.3243616
[24]
Vaswani, A., Shazeer, N.M., Parmar, N., Uszkoreit, J., Jones, L., et al. (2017) Attention Is All You Need. Proceedings of the 31st Neural Information Processing Systems (NeurIPS), Long Beach, 4-9 December 2017, 5998-6008.
[25]
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., et al. (2021) An Image is Worth 16 × 16 Words: Transformers for Image Recognition at Scale. Proceedings of the International Conference on Learning Representations (ICLR).
[26]
Chu, X., Tian, Z., Zhang, B., Wang, X. and Shen, C. (2021) Conditional Positional Encodings for Vision Transformers. Proceedings of the International Conference on Learning Representations (ICLR).
[27]
de Boer, P., Kroese, D.P., Mannor, S. and Rubinstein, R.Y. (2005) A Tutorial on the Cross-Entropy Method. Annals of Operations Research, 134, 19-67. https://doi.org/10.1007/s10479-005-5724-z
[28]
Li, Z., Shen, H., Li, H., Xia, G., Gamba, P. and Zhang, L. (2017) Multi-Feature Combined Cloud and Cloud Shadow Detection in Gaofen-1 Wide Field of View Imagery. Remote Sensing of Environment, 191, 342-358. https://doi.org/10.1016/j.rse.2017.01.026
[29]
Hughes, M. and Hayes, D. (2014) Automated Detection of Cloud and Cloud Shadow in Single-Date Landsat Imagery Using Neural Networks and Spatial Post-Processing. Remote Sensing, 6, 4907-4926. https://doi.org/10.3390/rs6064907
[30]
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., et al. (2019) PyTorch: An Imperative Style, High-Performance Deep Learning Library. Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, 8-14 December 2019, 8026-8037.
[31]
Loshchilov, I., and Hutter, F. (2019) Decoupled Weight Decay Regularization. Proceedings of the International Conference on Learning Representations (ICLR).
[32]
Long, J., Shelhamer, E. and Darrell, T. (2015) Fully Convolutional Networks for Semantic Segmentation. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, 7-12 June 2015, 3431-3440. https://doi.org/10.1109/cvpr.2015.7298965
[33]
Powers, D.M.W. (2020) Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness and Correlation. arXiv: 2010.16061.
