This study evaluates the reliability of North American Multi-Model Ensemble (NMME) models in forecasting monthly rainfall over the Kafue River Basin using a well-selected multi-model ensemble approach. Gridded monthly rainfall forecasts were derived from global NMME models and validated against satellite-based rainfall products (SRPs) over the basin. To establish a reliable gridded rainfall dataset, three SRPs—TAMSAT, CHIRPS, and ARC2—were assessed against observed station data. Historical data were divided into a calibration period (1983-2003) at the station level and a validation period (2004-2022) using gridded datasets. The NMME models—CMC2 CANSIPSv2, NASA-GEOSS2S, CANCM4i, GFDL-CM2p1, GFDL-CM2p5-FLOR-B01, GFDL-CM2p5, NCEP-CFSv2, and COLA-RSMAS-CCSM—were downscaled using the Canonical Correlation Analysis (CCA) algorithm and evaluated using Spearman’s correlation coefficient, mean bias, and root mean square error (RMSE). The Anomaly Correlation Coefficient (ACC) was used to assess forecast reliability. Results show that CHIRPS outperformed TAMSAT and ARC2 in representing observed rainfall and was used to generate a gridded time-series dataset. NMME model performance improved when validated against gridded datasets rather than station-based point data. The ensemble forecasting approach demonstrated reliable monthly rainfall predictions for December, January, and March (2004-2022). However, caution is advised when using NMME models for October and February, as these months exhibited negative ACC values (？1) over much of the basin. The study highlights spatial and temporal variability in the reliability of individual NMME models, emphasizing the importance of understanding model strengths and limitations for effective climate adaptation and water resource management.
References
[1]
Bi, K.F., Xie, L.X., Zhang, H.H., Cheng, X., Gu, X.T. and Tian, Q. (2022) Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast. arXiv: 2211.02556. https://doi.org/10.48550/arXiv.2211.02556
[2]
Jaseena, K.U. and Kovoor, B.C. (2022) Deterministic Weather Forecasting Models Based on Intelligent Predictors: A Survey. JournalofKingSaudUniversity—ComputerandInformationSciences, 34, 3393-3412. https://doi.org/10.1016/j.jksuci.2020.09.009
[3]
WMO-No 1246 (2020) Guidance on Operational Practices for Objective Seasonal Forecasting. WMO Secretariat.
Dinku, T., Faniriantsoa, R., Islam, S., Nsengiyumva, G. and Grossi, A. (2022) The Climate Data Tool: Enhancing Climate Services across Africa. FrontiersinClimate, 3, Article 787519. https://doi.org/10.3389/fclim.2021.787519
[6]
Wang, Q., Xia, J., She, D., Zhang, X., Liu, J. and Zhang, Y. (2021) Assessment of Four Latest Long-Term Satellite-Based Precipitation Products in Capturing the Extreme Precipitation and Streamflow across a Humid Region of Southern China. AtmosphericResearch, 257, Article ID: 105554. https://doi.org/10.1016/j.atmosres.2021.105554
[7]
IPCC (2021) Climate Change Widespread, Rapid, and Intensifying. IPCC.
[8]
Yuan, F., Zhang, L., Soe, K.M.W., Ren, L., Zhao, C., Zhu, Y., et al. (2019) Applications of TRMM-and GPM-Era Multiple-Satellite Precipitation Products for Flood Simulations at Sub-Daily Scales in a Sparsely Gauged Watershed in Myanmar. RemoteSensing, 11, Article 140. https://doi.org/10.3390/rs11020140
[9]
Water Resource Management Authority (2022) WARMA Strategic Plan 2022 to 2026. WARMA.
[10]
Petropoulos, F., Apiletti, D., Assimakopoulos, V., Babai, M.Z., Barrow, D.K., Ben Taieb, S., et al. (2022) Forecasting: Theory and Practice. InternationalJournalofForecasting, 38, 705-871. https://doi.org/10.1016/j.ijforecast.2021.11.001
[11]
Hunziker, S., Brönnimann, S., Calle, J., Moreno, I., Andrade, M., Ticona, L., et al. (2018) Effects of Undetected Data Quality Issues on Climatological Analyses. ClimateofthePast, 14, 1-20. https://doi.org/10.5194/cp-14-1-2018
[12]
Faiz, M.A., Liu, D., Fu, Q., Li, M., Baig, F., Tahir, A.A., et al. (2018) Performance Evaluation of Hydrological Models Using Ensemble of General Circulation Models in the Northeastern China. JournalofHydrology, 565, 599-613. https://doi.org/10.1016/j.jhydrol.2018.08.057
[13]
Le Roy, B., Lemonsu, A. and Schoetter, R. (2021) A Statistical-Dynamical Downscaling Methodology for the Urban Heat Island Applied to the EURO-CORDEX Ensemble. ClimateDynamics, 56, 2487-2508. https://doi.org/10.1007/s00382-020-05600-z
[14]
Trzaska, S. and Schnarr, E. (2014) A Review of Downscaling Methods for Climate Change. United States Agency for International Development by Tetra Tech ARD, 1-42.
[15]
Qu, B., Fang, Y. and Li, L. (2022) Analysis of Statistical Post-Processing Methods for Multi-Model Ensemble Runoff Forecasts in Flood Seasons. IOPConferenceSeries: EarthandEnvironmentalScience, 1087, Article ID: 012052. https://doi.org/10.1088/1755-1315/1087/1/012052
[16]
Panofsky, H.A. and Brier, G.W. (1958) Some Applications of Statistics to Meteorology. Mineral Industries Extension Services, Penn State University.
[17]
Yan, G., Liu, Y. and Chen, X. (2018) Evaluating Satellite-Based Precipitation Products in Monitoring Drought Events in Southwest China. InternationalJournalofRemoteSensing, 39, 3186-3214. https://doi.org/10.1080/01431161.2018.1433892
[18]
JWGFVR Joint Working Group on Forecast Verification Research (2017) Types of Forecasts and Verifications. CAWCR.
[19]
WMO (2018) WMO Guidance on Verification of Operational Seasonal Climate Forecasts. WMO Secretariat.
[20]
Gutierrez, J.M., San-Martin, D., Cofino, A.S., Herrera, S., Manzanas, R.G. and Frias, M.D. (2011) User Guide of the ENSEMBLES Downscaling Portal. Santander Meteorology Group.
[21]
Hodson, T. (2022) Root Mean Square Error (RMSE) or Mean Absolute Error (MAE): When to Use Them or Not 1U.S. Geological Survey Central Midwest Water Science Center.
[22]
Kim, S. and Kim, H. (2016) A New Metric of Absolute Percentage Error for Intermittent Demand Forecasts. InternationalJournalofForecasting, 32, 669-679.
[23]
Meloun, M. and Militký, J. (2011) Statistical Analysis of Multivariate Data. In: Meloun, M. and Militký, J., Eds., StatisticalDataAnalysis, Elsevier, 151-403. https://doi.org/10.1533/9780857097200.151
[24]
Cannizzaro, D., Aliberti, A., Bottaccioli, L., Macii, E., Acquaviva, A. and Patti, E. (2021) Solar Radiation Forecasting Based on Convolutional Neural Network and Ensemble Learning. ExpertSystemswithApplications, 181, Article ID: 115167. https://doi.org/10.1016/j.eswa.2021.115167
[25]
Montaño Moreno, J., Palmer Pol, A., Sesé Abad, A. and Cajal Blasco, B. (2013) Using the R-MAPE Index as a Resistant Measure of Forecast Accuracy. Psicothema, 4, 500-506. https://doi.org/10.7334/psicothema2013.23
[26]
Han, L., Chen, M., Chen, K., Chen, H., Zhang, Y., Lu, B., et al. (2021) A Deep Learning Method for Bias Correction of ECMWF 24-240 H Forecasts. AdvancesinAtmosphericSciences, 38, 1444-1459. https://doi.org/10.1007/s00376-021-0215-y
[27]
WMO No 1203 (2017) WMO Guidelines on the Calculation of Climate Normals. WMO Secretariat.
[28]
Wang, B., Zheng, L., Liu, D.L., Ji, F., Clark, A. and Yu, Q. (2018) Using Multi-Model Ensembles of CMIP5 Global Climate Models to Reproduce Observed Monthly Rainfall and Temperature with Machine Learning Methods in Australia. InternationalJournalofClimatology, 38, 4891-4902. https://doi.org/10.1002/joc.5705
[29]
WMO (2021) WMO Guidelines on the Quality Control of Surface Climatological Da-ta. WMO Secretariat.
[30]
Baez-Villanueva, O.M., Zambrano-Bigiarini, M., Ribbe, L., Nauditt, A., Giraldo-Osorio, J.D. and Thinh, N.X. (2018) Temporal and Spatial Evaluation of Satellite Rainfall Estimates over Different Regions in Latin-America. AtmosphericResearch, 213, 34-50. https://doi.org/10.1016/j.atmosres.2018.05.011
[31]
Libanda, B., Mie, Z., Nyasa, L. and Chilekana, N. (2020) Deciphering the Performance of Satellite-Based Daily Rainfall Products over Zambia. ActaGeophysica, 68, 903-919. https://doi.org/10.1007/s11600-020-00429-w
[32]
Omondi, C.K. (2017) Assessment of Bias Corrected Satellite Rainfall Products for Stream Flow Simulation: A Top Model Application in the Kabompo River Basin Zambia. University of Twente.
[33]
Saha, U., Singh, T., Sharma, P., Gupta, M.D. and Prasad, V.S. (2020) Deciphering the Extreme Rainfall Scenario over Indian Landmass Using Satellite Observations, Reanalysis and Model Forecast: Case Studies. AtmosphericResearch, 240, Article ID: 104943. https://doi.org/10.1016/j.atmosres.2020.104943
[34]
Schober, P., Boer, C. and Schwarte, L.A. (2018) Correlation Coefficients: Appropriate Use and Interpretation. Anesthesia&Analgesia, 126, 1763-1768. https://doi.org/10.1213/ane.0000000000002864
[35]
Hagedorn, R., Doblas-Reyes, F.J. and Palmer, T.N. (2005) The Rationale behind the Success of Multi-Model Ensembles in Seasonal Forecasting—I. Basic Concept. TellusA, 57, 219-233. https://doi.org/10.1111/j.1600-0870.2005.00103.x
[36]
Ahmed, K., Sachindra, D.A., Shahid, S., Iqbal, Z., Nawaz, N. and Khan, N. (2020) Multi-Model Ensemble Predictions of Precipitation and Temperature Using Machine Learning Algorithms. AtmosphericResearch, 236, Article ID: 104806. https://doi.org/10.1016/j.atmosres.2019.104806
