Uncertainty maps for model-based global climate classification systems

Abstract

Climate classification systems (CCSs) are emerging as essential tools in climate change science for mitigation and adaptation. However, their limitations are often misunderstood by non-specialists. This situation is especially acute when the CCSs are derived from Global Climate Model outputs (GCMs). We present a set of uncertainty maps of four widely used schemes -Whittaker-Ricklefs, Holdridge, Thornthwaite-Feddema and Köppen- for present (1980–2014) and future (2015–2100) climate based on 52 models from the Coupled Intercomparison Model Project Phase six (CMIP6). Together with the classification maps, the uncertainty maps provide essential guidance on where the models perform within limits, and where sources of error lie. We share a digital resource that can be readily and freely integrated into mitigation and adaptation studies and which is helpful for scientists and practitioners using climate classifications, minimizing the risk of pitfalls or unsubstantiated conclusions.

Background & Summary

Climate classification systems (CCSs) are valuable tools for societal and environmental applications^1,2,3. They simplify complex, multidimensional climate data by transforming continuous variables, such as temperature and precipitation, into discrete categories that are meaningful for ecological purposes. This process, known as dimension reduction, creates a user-friendly format that facilitates the identification of broad patterns between climate drivers and the spatial distribution of biota. Used together with climate model outputs, CCSs provide a neat description of the past and future climate change^4,5,6,7.

A significant limitation of CCSs, however, is that their ability to accurately define climate zones depends on the quality of the input data. If these are uncertain, the resulting product is compromised. End users assume these uncertainties are monitored by the developers of CCSs and that their impact on the final product is limited, giving a false sense of confidence⁸.

Most CCSs are built using monthly and/or yearly data from two primary variables: temperature (TS) and precipitation (PR). TS is a smooth, highly spatially autocorrelated climate field. In contrast, PR exhibits high spatial variability, making it challenging to measure and predict. Indeed, errors exceeding 100% are not uncommon for precipitation data⁹. Given the many difficulties in capturing the true nature of precipitation, it has unsurprisingly become the primary source of uncertainty and bias in CCSs, potentially limiting the reliability of the analysis derived from these datasets¹⁰.

Our goal is to provide both practitioners and researchers from a variety of disciplines -e.g., biologists, geographers, and ecologists- with the tools and datasets they need for accurate environmental analysis based on the current knowledge of state-of-the-art Global Climate Models (GCMs). A key element is that of consensus maps, as defined by Navarro et al.¹⁰. These maps, along with their corresponding classification maps, synthesize information from multiple models to reveal areas of agreement (or disagreement) according to present and future climate types (Fig. 1). This comprehensive view allows users to navigate potential pitfalls in their analyses and ensure robust research.

Fig. 1

Global distribution of climate zones for the four CCSs for the SSP5-8.5 future scenario (a,c,e,g). Consensus maps of climate zones for the top-10 ensemble mean (b,d,f,h). Colors represent the degree of confidence: dark blue (≥80%), light blue (79-60%), yellow (59-40%), light red (39-20%), and dark red (<20%). The stripes show the regions with low class agreement.

This analysis leverages output from top-ranked GCMs to explore future climate conditions through the lens of four well-known climate classification systems: Whittaker-Ricklefs biomes, Köppen’s climate types, Thornthwaite-Feddema climate classification, and Holdridge’s life zones. While Köppen is dominant^11,12,13, the other CCSs offer valuable and often more nuanced perspectives on the complex relationship between climate and environment, each tailored to the needs of specific audiences^14,15,16.

Methods

Climate Classification Systems

The main four CCSs used in the literature are included: 1. Holdridge’s life zones¹⁷; 2. Köppen climate types¹⁸; 3. Thornthwaite’s classification¹⁹; 4. Whittaker’s biomes²⁰.

Holdridge life zones are defined by three measurements: annual precipitation (mm·year⁻¹), biotemperature (°C) and potential evapotranspiration (PET) ratio. Annual precipitation is calculated from monthly precipitation data. Mean annual biotemperature is derived from monthly average temperature. Those months with a mean temperature above 30.0 °C and below 0.0 °C are considered as 30.0 °C and 0.0 °C, respectively. PET ratio is defined as the mean annual biotemperature multiplied by a constant value (58.93) and divided by annual precipitation. We assign a class to each grid cell by computing the minimum Euclidean distance between each pixel and the geometric centroids of life zones, as defined in Sisneros et al.²¹. The 33 classes are then grouped into 13 categories as shown in Table 1.

Table 1 Holdridge life zones and their geometric centroids, as distributed in 13 major biomes.

For the Köppen scheme, we followed the work of Lohman et al.²², which is based on Köppen’s early work. In this seminal publication, ecoregions are divided into five climate types, four thermal types (A, C, D, E), and one hydrologic type (B). In addition, the scheme includes three subtypes (f, s, w) related to the annual cycle of precipitation. Operationally, the classification algorithm works as follows: first, it evaluates the precipitation threshold. Second, it processes climate types A, C, D and E, respectively. This workflow ensures a consistent identification of arid climates²³. Otherwise, discrepancies may arise. While this is not a problem when only models are compared (as in Tapiador et al.²⁴), it will certainly yield paradoxes if observations are used. The defining criteria of each climate type are shown in Table 2.

Table 2 Climate types and defining criteria for Köppen.

We used the revised version of Thornthwaite’s classification proposed by Feddema²⁵, which has become standard practice in the field. Similar to the original work, it centers on the concept of water availability. However, this revision incorporates a simplified version of the moisture state, as defined by Willmott and Feddema²⁶. Moisture Index I_m ranges from −1 to 1 and is defined as follows: ${I}_{m}=\left\{\begin{array}{c}\left(r/{PE}\right)-1,r < {PE}\\ 1-\left({PE}/r\right),r\ge {PE}\end{array}\right.$ where r is the annual rainfall and PE is potential evapotranspiration. The thermal component remains faithful to the classic definition of potential evapotranspiration developed by Thornthwaite. Table 3 shows the criteria defining the 36 climate types.

Table 3 Description of thermal and moisture types according to Feddema’s method.

We followed the delineation of Whittaker’s biomes as described by Ricklefs²⁷. The scheme divides the Earth into nine biomes according to annual precipitation (cm·year⁻¹) and annual mean temperature (^oC). The biome distribution criteria are shown in Fig. 2.

Fig. 2

Whittaker’s Biome Diagram. Whittaker’s scheme uses climatologies of precipitation and temperature to shape the spatial pattern of major biomes. This is a modified version of Whittaker’s original work based on Ricklefs’s diagram.

Classification for this case was performed using the R package plotbiomes²⁸. Outliers were assigned the climate type of the closest biome.

Climate data

Climate classification maps and consensus maps were constructed using monthly precipitation and temperature from 52 models of the Coupled Model Intercomparison Project phase 6 (CMIP6)²⁹. CMIP6 data are publicly available and have been downloaded from the Earth System Grid Federation node (https://aims2.llnl.gov/search). The dataset included both the current climate (1980–2014) and future projections (2021–2050, 2051–2100, 2015–2100). To explore a range of future scenarios, we incorporated data from three Shared Socioeconomic Pathways (SSPs)³⁰: SSP1-2.6, SSP2-4.5, and SSP5-8.5. The three choices are intended to be comprehensive and cover the vast majority of needs of adaptation and mitigation studies. Finally, to ensure consistency, all datasets were interpolated to a common horizontal resolution of 1° × 1°, using bilinear interpolation. This choice aligns with the native resolution of most CMIP6 models and minimizes potential uncertainties introduced by downscaling to finer resolutions, which is critical for developing consensus maps. The analysis was for land-only, excluding Antarctica. The full list of models can be found in Supplementary Information (Table S1).

Model rank and top-10 ensemble members

We used Cohen’s kappa coefficient³¹ to quantify the ability of each model to reproduce the distribution of climate categories. This standard method is widely used to gauge model quality. Present reference climate is also the standard choice: it is calculated using the Climate Research Unit Time Series (CRU, version 4.04³²) observations for each CCS for the historical climatological period (1980–2014). The kappa coefficient (κ) is defined as follows: ${\rm{\kappa }}=\frac{{P}_{0}-{P}_{e}}{1-{P}_{e}}$, where P₀ is the proportion of units with agreement and P_e is the hypothetical probability of chance agreement. The kappa statistic ranges from 0 to 1, with 0 meaning no agreement and 1 meaning perfect agreement.

Models were also ranked in terms of joint agreement with precipitation and temperature observations. The metric used for these quantitative variables was the coefficient of determination R². The reason for using this metric is twofold: first, it has been shown to be useful for comparing large-scale means of climate data, such as those used by most climate classification schemes¹⁵. Second, it condenses information into a single, easy-to-interpret value, thus facilitating comparisons between models. Grid boxes were weighted by area in both metrics. We then identified the best performing models by focusing on the upper right quadrant of a 2D plane (Fig. 3). This quadrant represents models with both high class agreement (κ) and high precipitation scores (R²). The median values of κ and R² (precipitation) were used to define the boundaries. Finally, we selected the top-10 models within this quadrant based on the highest κ scores.

Fig. 3

Scores of CMIP6 models for the four classification schemes: (a) Holdridge, (b) Köppen, (c) Thornthwaite-Feddema, (d) Whittaker-Ricklefs. Dots are individual CMIP6 models (52 models + 2 ensemble means). The y axis is the R² score for global precipitation (mm·year⁻¹), x axis is the kappa coefficient of each CCS and colors are the R² score for global mean temperature. Dotted grey lines are the median. Zoomed areas indicate the region of best performance, and models within this area are then selected to create the top-10 ensemble mean (T10). Numbers in black are the top 10 models, while the other models are in grey. The red number is the T10 ensemble mean, and the blue one is the 52-model ensemble mean. The analysis was performed for the 1980–2014 period.

Table 4 shows the scores for the top-10 models for Whittaker’s scheme. For the remaining CCSs, see Tables S2–S4 in the Supplementary Information document.

Table 4 Scores of top-10 models, (52) ensemble, and (T10) ensemble for Whittaker’s biomes.

Uncertainty analysis

Model uncertainties are embedded in the so-called consensus maps (Fig. 4). Consensus maps provide a qualitative description of model uncertainties, highlighting regions where models disagree. Quantitatively, these maps are built on two key metrics derived from pixel-by-pixel comparisons: percent confidence and inter-model class agreement.

Fig. 4

Graphical description of the uncertainty estimation of distributed climate zones. (a) Maps of climate types for top-10 models and reference data. (b) The performance of individual models was evaluated against the reference data in a pixel-by-pixel comparison, and the degree of agreement between reference and models (in percent) was then calculated. Each pixel is colorized in the consensus map according to the level of confidence: dark blue (≥80%), light blue (79-60%), yellow (59-40%), light red (39-20%), and dark red (<20%). (c) Inter-model class agreement, which represents the number of different climate types identified by the top-10 models for each pixel. Two or fewer climate types indicates low inter-model variability, while three or more indicates high variability (striped regions in the consensus map). The analysis was performed for the four climate classification schemes. Reference data is CRU (for present) and T10 ensemble mean (for future scenarios).

The first metric (Fig. 4b) quantifies the accuracy of individual models relative to observed climate types. The formulation is as follows: ${{\rm{Confidence}}}_{i,j}=\frac{{{coincident\; models}}_{i,j}}{{{Nmodels}}_{i,j}}\cdot 100$, where the fraction represents the proportion of models, for each pixel (i,j), that predicted a climate class matching the reference data. Here, coincident models refers to the number of models with matching climate type, and Nmodels represents the total number of models considered. Confidence levels were discretized into five categories: very high (≥80%), high (79%- 60%) moderate (59%-40%), low (39%-20%), and very low (<20%). The second metric evaluates inter-model variability, i.e. how many climate types are identified by different models for the same point in space. For example, Fig. 4c shows that for a given pixel at (i,j), four models defined the region as woodland, three as temperate desert, and the other three as boreal forest. This results in three different climate types, which implies low inter-model agreement. Both metrics are complementary, the first metric shows the accuracy while the second measures the dispersion (precision). Uncertainty maps were constructed for each CCS for the present and the three SSPs, using the CRU and the top-10 ensemble, respectively, as reference.

Data Records

Global datasets of the four CCSs and their associated consensus maps for present and future climates are available at Figshare³³. The data are provided in three different file formats to meet the needs of a wide range of users. All datasets have a spatial resolution of 1°, which corresponds to 180 × 360 pixels. For ease of download, individual files within the same format are combined into a single tar.gz archive.

• 1. GeoTIFF files are for general users who want an immediate image of the distribution of climate types and consensus maps. The compressed file contains 120 GeoTIFF files accompanied by a legend (legend.txt) that defines the code categories used in each climate classification system (13 for Holdridge, 11 for Köppen, 36 for revised Thornthwaite and 9 for Whittaker’s biomes). The filename structure is: classification_varname_scenario_period.tif (e.g koppen_confidence_historical_1980-2014.tif, whittaker_class_ssp126_2051-2100.tif, and koppen_modvar_ssp585_2021-2050.tif). Class refers to climate classification, confidence is percent confidence, and modvar refers to inter-model variability.

• 2. NetCDF files are for more advanced users. They can be directly read in a Geographic Information System (GIS) such as QGIS. A major advantage is that they are self-describing files, meaning the data and its metadata are stored together in a single file. The tar.gz includes 80 nc files. The files including climate classification maps also include their respective key variables, except for Köppen. For example, the file whittaker_class_historical_1980-2014.nc contains data for annual mean temperature (AT, in ^oC), annual precipitation (APP, in cm·year^-1), and Whittaker’s climate classification itself. Similarly, holdridge_class_ssp245_2051–2100.nc includes biotemperature (ABT, in ^oC), annual precipitation (APP, in mm·year⁻¹), potential evapotranspiration ratio (PER) and Holdridge’s life zones (HLZ) for the future scenario SSP2-4.5 and the period 2051-2100. Files of consensus maps, like holdridge_consensus_ssp585_2021-2050.nc, store the following variables: confidence and modvar. Codes for the classification categories are embedded within the file’s attributes for easy reference.

• 3. BIL, pure binary files are suited for fast reading and efficient computation. The compressed file contains 120 bil files, along with the corresponding header files (hdr) and a legend file (legend.txt). They can also be directly read in a GIS. The dimensions of each bil file are 180 × 360. Filenames follow the same naming convention as tiff files.

The data are based on the top-10 ensemble mean, although we performed the calculations for the 52 CMIP6 models. Data for individual models and the MME (52) are available upon request to the corresponding author.

Technical Validation

Model validation

Climate classifications derived from GCMs were validated against observations from the Climate Research Unit (CRU at the University of East Anglia) Time Series version 4.04³². The CRU dataset has 0.5° spatial resolution but was aggregated to 1°, following the same procedure as for the CMIP6 models. The validation was performed for three variables: annual precipitation, mean annual temperature, and climate classes. Figure 3 shows the scores of individual models, as well as ensemble means -MME (52) and top-10 ensemble- for the 1980–2014 period. Alternative ensemble sizes (top-20, top-30 and top-40) were also evaluated, but their performance did not show consistent improvement over top-10 and MME. As illustrated in the figure, global performance varies depending on the climate classification scheme. Whittaker’s biomes achieve the highest overall scores (κ = 0.74), while Thornthwaite’s shows the lowest scores (κ = 0.42) for the MME (52). Eight individual GCMs/ESMs consistently achieve the highest scores across all CCSs: CESM2, EC-Earth-Veg-LR, HadGEM3-GC31-LL, HadGEM3-GC31-MM, MRI-ESM2-0, NorESM2-MM, TaiESM1 and UKESM1-0-LL. This is likely because they perform better at simulating precipitation, a key source of bias in CCSs¹⁰. For mean temperature, all models obtained scores above R² = 0.95.

Uncertainty analysis of reference data

Observational data often have differences that can affect the representation of climate zones. To assess this sensitivity, we evaluated climate classifications using alternative datasets in addition to CRU. These included station-based data from the University of Delaware (UDEL version 5.01³⁴) and the Global Precipitation Climatology Centre (GPCC) Full Data Monthly Product version 2022^35,36, as well as the ECMWF’s reanalysis data^37,38 (ERA5).

A classification scheme that is insensitive to these variations is considered robust to observational uncertainties³⁹. Figure 5 illustrates this concept, showing small changes in the global distribution of climate types.

Fig. 5

Percentage of continental area covered by each climate type for the four climate classification schemes (Köppen, Holdridge, Thornthwaite-Feddema, and Whittaker-Ricklefs) and the four observational datasets (CRU, GPCC, UDEL and ERA5). Classification schemes for GPCC are a combination of data from the GPCC (precipitation) and CRU (temperature).

CRU and GPCC show the greatest agreement (1.3% to 1.6% of the discrepancy in total land area) for Holdridge, Köppen, and Whittaker classifications. UDEL also has relatively small discrepancies (1.5% to 3.1%), while ERA5 displays slightly larger differences (3.6% to 7.3%). Thornthwaite’s scheme shows the greatest sensitivity to data source. Here, ERA5 predicts a shift in high-latitude climates, with cold-wet types becoming more common at the expense of cold-moist types.

Usage Notes

The dataset offers a comprehensive description of current and future global climate, based on four major classification schemes. We provide maps for Holdridge, Köppen, Thornthwaite-Feddema, and Whittaker-Ricklefs climate classifications along with their corresponding consensus maps. Future scenarios included are SSP1-2.6, SSP2-4.5 and SSP5-8.5 (three time-periods: 2021-2050, 2051-2100, and 2015-2100). The data are derived from an ensemble of the top-10 CMIP6 models (out of 52), selected based on their kappa statistic and their R² score for precipitation. Top-10 models differ by classification scheme. Validation with observations (CRU) indicates that model outputs are reasonable. However, there are certain caveats that users may consider before using the dataset:

• 1. We assume that current criteria for defining climate types are the same for future climate, leaving no room for new equilibrium conditions or new climate types. This is a common issue in any rule-based classification scheme^40,41.

• 2. Our maps have a 1° spatial resolution, which is the native resolution of most CMIP6 models. While it might be deemed low for certain regional applications, it is worth noting that kilometer or even hectometer resolutions are the result of interpolations well below the resolution of the original empirical data. Indeed, there is no limit for downscaling a scalar field; the only question is what is sacrificed in the mathematical process. By keeping the spatial resolution at the nominal model grid size, we avoid introducing additional uncertainties associated with downscaling techniques, and ensure that the final product reflects the inherent resolution of the underlying climate models.

• 3. Model performance is contingent to the classification scheme. As shown in Fig. 3, individual models and ensemble means presented different scores, according to the CCS. While models performed well for Köppen (11 cats), Holdridge (13 cats), and Whittaker (9 cats), the performance for Thornthwaite (36 cats) was moderate, meaning that their maps should be used with caution.

• 4. All four CCSs are primarily based on precipitation and temperature. However, each CCS was developed with distinct goals and criteria, leading to variations in how they identify regions undergoing climate change.

• 5. Consensus maps depict the confidence of model predictions pinpointing regions where consensus is lacking. While many of such regions overlap across CCSs, some disagreements arise, specifically at the boundary of climate types, as defined by each unique classification scheme. Additionally, the degree of consensus may vary depending on the socio-economic scenario chosen.