Sa’adi et al. (2026) Optimizing category-based statistical metrics for selecting global climate models in rainfall projections for Peninsular Malaysia

Identification

• Journal: Theoretical and Applied Climatology
• Year: 2026
• Date: 2026-01-09
• Authors: Zulfaqar Sa’adi, Dauda Pius Awhari, Ricky Anak Kemarau, Zainura Zainon Noor, Zhang Taining, Mohamad Faizal Ahmad, Abdalmaged Salem, Gunarangini Muniandy, Mohammed Faisal Mohammed Dafalla, Wimbi Apriwanda Nursiwan, Muhammad Reza Agraha Maha, Ariani Dwi Astuti, Salam Aied Al-Husban
• DOI: 10.1007/s00704-025-05952-8

Research Groups

• Centre for Environmental Sustainability and Water Security (IPASA), Research Institute for Sustainable Environment (RISE), Universiti Teknologi Malaysia (UTM), Johor Bahru, Malaysia
• Department of Agricultural and Bioresources Engineering, Taraba State University, Jalingo, Nigeria
• Department of Water & Environmental Engineering, School of Civil Engineering, Faculty of Engineering, Universiti Teknologi Malaysia, Johor Bahru, Malaysia
• Earth Observation Centre, Institute of Climate Change, Universiti Kebangsaan Malaysia, Bangi, Selangor, Malaysia
• Faculty of Chemical and Energy Engineering, Universiti Teknologi Malaysia, Johor Bahru, Johor, Malaysia
• Department of Bioscience, Faculty of Science, Universiti Teknologi Malaysia, Johor Bahru, Johor, Malaysia
• Faculty of Civil Engineering, Universiti Teknologi Malaysia (UTM), Johor Bahru, Johor, Malaysia
• Chemistry Education, Faculty of Educational Sciences and Technology, Universiti Technology Malaysia, Johor, Malaysia
• Department of Geotechnics & Transportation, Faculty of Civil Engineering, Universiti Teknologi Malaysia, Johor, Malaysia
• Department of Environmental Engineering, Faculty of Landscape Architecture and Environmental Technology, Universitas Trisakti, Jakarta Barat, DKI Jakarta, Indonesia

Short Summary

This study optimized category-based statistical metrics to reduce redundancy and identify the most reliable Global Climate Models (GCMs) for accurate rainfall projections in Peninsular Malaysia, ultimately identifying five top-performing GCMs, notably TaiESM1 and CMCC-ESM2, that best replicate observed rainfall characteristics.

Objective

• To optimize category-based statistical metrics (error, efficiency, correlation, composite, and others) to reduce redundancy and identify the most reliable Global Climate Models (GCMs) for improving rainfall projection accuracy in Peninsular Malaysia.

Study Configuration

• Spatial Scale: Peninsular Malaysia (PM), covering an area of 131,598 square kilometers, analyzed at a 0.25-degree spatial resolution (178 grid points).
• Temporal Scale: Historical period from 1981 to 2014 for GCM evaluation against observational data. Observational data (CHIRPS) spans 1981 to 2023.

Methodology and Data

• Models used: 23 Global Climate Models (GCMs) from the Coupled Model Intercomparison Project Phase 6 (CMIP6). Statistical methods included a multi-filtration approach, Principal Component Analysis (PCA), Compromise Programming Index (CPI), Fisher-Jenks natural breaks classification, and Probability Distribution Function (PDF) analysis.
• Data sources: Climate Hazards Group InfraRed Precipitation with Station data (CHIRPS) daily rainfall data (observational benchmark). CMIP6 GCM outputs for historical simulations and Shared Socioeconomic Pathways (SSPs) scenarios (SSP1-2.6, SSP2-4.5, SSP3-7.0, SSP5-8.5).

Main Results

• A systematic selection process reduced the initial 22 GCM evaluation metrics by up to 64%, retaining eight key metrics: Unbiased Root Mean Squared Error (ubRMSE), Percent Bias (PBIAS), Nash-Sutcliffe Efficiency (NSE), Modified Nash-Sutcliffe Efficiency (mNSE), Coefficient of Determination (R²), Kling-Gupta Efficiency (KGE), Modified Index of Agreement (MD), and Revised Index of Agreement (DR).
• Principal Component Analysis (PCA) confirmed these eight metrics as the most representative, with the first two principal components capturing 95.5–100% of the total variance across different metric categories.
• Using the Compromise Programming Index (CPI) and Fisher-Jenks classification, five top-performing GCMs were identified: TaiESM1 (CPI = 0.5507), CMCC-ESM2 (CPI = 0.6753), MRI-ESM2-0 (CPI = 0.7473), CMCC-CM2-SR5 (CPI = 0.8402), and EC-Earth3-Veg (CPI = 1.0233).
• Validation using Probability Distribution Functions (PDFs) showed that TaiESM1 (Overlap Efficiency, OVL = 0.96) and CMCC-ESM2 (OVL = 0.91) most accurately replicated CHIRPS monthly rainfall, closely matching observed mean, standard deviation, skewness, and kurtosis.

Contributions

• Developed an innovative methodology for optimizing GCM selection by categorizing statistical metrics and systematically reducing redundancy, leading to a 64% reduction in metrics while retaining critical information.
• Significantly enhanced the robustness, reproducibility, and transferability of GCM performance assessments by reducing biases associated with subjective and imbalanced metric selection.
• Identified an optimized, non-redundant set of eight metrics that comprehensively assess multiple dimensions of model performance, addressing a gap in existing literature that often relies on limited or excessively large, unoptimized metric sets.
• Provided a replicable framework for improving GCM selection, particularly valuable for regions with complex climate dynamics, thereby advancing climate modeling for sustainable development planning and supporting informed decision-making in climate adaptation and water resource management.

Funding

• The Ministry of Higher Education Malaysia
• Universiti Teknologi Malaysia
• UTM Professional Development Research University Special (Grant No. R.J130000.7113.07E66)

Citation

@article{Saadi2026Optimizing,
  author = {Sa’adi, Zulfaqar and Awhari, Dauda Pius and Kemarau, Ricky Anak and Noor, Zainura Zainon and Taining, Zhang and Ahmad, Mohamad Faizal and Salem, Abdalmaged and Muniandy, Gunarangini and Dafalla, Mohammed Faisal Mohammed and Nursiwan, Wimbi Apriwanda and Maha, Muhammad Reza Agraha and Astuti, Ariani Dwi and Al-Husban, Salam Aied},
  title = {Optimizing category-based statistical metrics for selecting global climate models in rainfall projections for Peninsular Malaysia},
  journal = {Theoretical and Applied Climatology},
  year = {2026},
  doi = {10.1007/s00704-025-05952-8},
  url = {https://doi.org/10.1007/s00704-025-05952-8}
}