Background & Summary

Essential Climate Variables (ECVs) are a group of bio-physical variables that are critical for understanding and characterizing Earth’s climate1. Therefore, this relatively limited set of variables are vital “to guide mitigation and adaptation measures, to assess risks and enable attribution of climate events to underlying causes, and to underpin climate services”2. Originally conceptualized by the Global Climate Observing System (GCOS) in the 1990s, the ECVs framework has gained widespread adoption in scientific and policy domains. Various climate agencies and organizations, including the United Nations Framework Convention on Climate Change (UNFCCC), the World Meteorological Organization (WMO) and the Intergovernmental Panel on Climate Change (IPCC), rely on ECVs as guiding metrics. GCOS currently identifies 55 ECVs, classifying them into three main categories: atmospheric, terrestrial and oceanic variables3.

In order to evaluate large-scale climatic patterns, a consistent, extensive and continuous estimation of ECVs is needed. Such estimations can be achieved by merging observations, that are limited in time and space, with computational models, that can consistently fill the gaps between observations. Although various reanalysis products estimate atmospheric and oceanic ECVs4,5, important terrestrial ECVs such as the terrestrial water cycle components6 have yet to be treated in an integrated manner by the Earth Observation (EO) water cycle community and Land Surface and Hydrologic Model (LSM/HM) communities7. For instance, in estimating evapotranspiration, Surface Vegetation-Atmosphere Transfer (SVAT) schemes do not necessarily close the water balance in gauged basins8. Likewise, in reconstructing streamflow at a regional or continental scale, little or no input from EO systems is generally used9,10,11. Consequently, the EO and LSM/HM communities have yet to treat the water cycle as a unique system consistently.

To advance the generation of high-resolution terrestrial ECVs, the European Space Agency (ESA) initiated the “Hyper-resolution Earth observations and land-surface modeling for a better understanding of the water cycle” project, referred to as 4DHydro (4D refers to time and space). This initiative aims to facilitate extensive collaboration between the EO and LSM/HM communities, striving for better integration between innovative high-resolution satellite products and hyper-resolution modeling of the hydrological cycle. For example, from an EO perspective, the project will explore the potential benefits of LSM/HM simulations in enhancing the downscaling of satellite products and achieving water balance closure. In contrast, from an LSM/HM perspective, the project will explore the added value of assimilating EO products, through data assimilation and calibration, on high-resolution hydrological simulations. An essential first step of 4DHydro, presented here, involves a comprehensive assessment of the uncertainty inherent in existing LSM/HM datasets concerning key hydrological variables. Such an assessment will give a better understanding of the limitations in generating high-resolution terrestrial ECVs and will be used as a baseline for LSM/HM performance under EO integration. To that end, we introduce a new hydrological reference dataset.

Our hydrological reference dataset comprises 19 existing LSM/HM simulations from previous studies and represents the current state-of-the-art of our LSM/HMs. The dataset covers four study regions, namely continental Europe, the European Po and Rhine river basins and the Tugela river basin in Africa, and includes four hydrological variables on a daily time step, namely discharge, evapotranspiration, soil moisture content and total water storage (Data Records Section). Our dataset covers a diverse range of LSM/HM simulations with 8 different land-surface and hydrological models from the 4DHydro project and 7 different meteorological input datasets (Methods Section), thereby capturing both systemic and input uncertainties. Despite this diversity, simulations in our hydrological reference dataset are uniformly formatted following our storage protocol (see Data Records Section), which allows for seamless comparisons across simulations. Additionally, we present a performance benchmark of our LSM/HM simulations (Technical Validation Section). The benchmark comprehensively validates the simulations of our hydrological dataset using observations from discharge gauges (from GRDC, LamaH-CE, CAMELS-GB, CAMELS-CH and various local datasets), evapotranspiration towers (from FLUXNET), soil moisture stations (from ISMN) and total water storage anomaly satellite products (from GRACE and GRACE-FO).

LSM/HMs simulations of our hydrological reference dataset are publicly available through the open science catalog at 4dhydro.eu/catalog (Usage Notes section) to enable open collaboration with end-users, including the scientific community and the general public. Our hydrological reference dataset serves dual purposes: (1) the dataset provides valuable and publicly accessible information to support policy development in the regions of interest and (2) the dataset functions as a benchmark for generating high-resolution terrestrial ECVs, through the integration of EO products, during subsequent stages of the 4DHydro initiative. In these next stages, LSM/HM simulations will be improved through calibration, assimilation and validation with high-resolution EO products. These improved simulations, and the high-resolution EO products, will also be made available through the open science catalog. Considering that the main aim of the 4DHydro initiative is to facilitate collaboration, we welcome the scientific community to contribute to our hydrological reference dataset following the approach outlined here.

Methods

The 19 simulations in the reference dataset are derived from combinations of eight existing Land-Surface Model and Hydrological Models (LSM/HMs) and seven meteorological datasets. This section provides a concise description of the LSM/HMs, their main references and parameterization, and the meteorological datasets.

Land-Surface and Hydrological Models (LSM/HMs)

The reference dataset includes eight LSM/HMs: Community Land Model (CLM), GEOframe, mesoscale Hydrologic Model (mHM), Parflow-CLM, PCRaster Global Balance (PCR-GLOBWB), TETIS, Terrestrial Systems Modeling Platform (TSMP) and wflow_sbm. Table 1 presents the used data sets for the model parameterization.

Table 1 Summary of data sources used as input for each land surface and hydrological model.
Full size table

These models represent a diverse range of model structures, conceptualizations and implementations. For example, most models are distributed models, which simulate hydrology on a grid, except for the GEOframe model which is a semi-distributed model that simulates hydrology per sub-basin. Additionally, components such as lateral groundwater flows are implemented in Parflow-CLM, TSMP and wflow_sbm, whereas human impacts are implemented in PCR-GLOBWB (water use and reservoir operations) and wflow_sbm (reservoir operations). Lastly, several models rely on calibration, either directly (GEOframe and TETIS) or through transfer functions (mHM), whereas the other models only spin-up their initial conditions. Although all these model differences carry their benefits and uncertainties, all models are included in our reference dataset to capture both systemic and input uncertainties.

Community Land Model (CLM)

CLM simulates land surface processes in global climate studies as part of the Community Climate System Model (CCSM3)12,13,14. These processes encompass biophysics, radiation interactions, heat transfer, hydrology and vegetation characteristics. The model accounts for plant functional types and soil properties15. CLM version 3.5, the version used here, introduces significant changes in its hydrological cycle simulation. These include updated surface datasets16, an improved canopy integration scheme17, scaled canopy interception18, a simplified TOPMODEL-based runoff model19, a groundwater model that considers water exchange in the vertical direction20 and a new frozen soil scheme21. CLM has been used to simulate and assimilate tECVs in various regions such as Europe22 and China23. Although later versions of CLM (4.0 and 4.5) exhibit a better parametrization, previous studies indicate that the differences between CLM3.5 and later versions are small when comparing soil moisture variability to observations24. For the CLM simulations in the reference dataset, no calibration was required. Instead, CLM considers a spin-up of initial conditions and states up to a dynamic equilibrium.

GEOframe

GEOframe is a versatile computer-based hydrology system. GEOframe offers diverse modeling solutions with components built on the Object Modeling system framework25, which accommodates various modeling paradigms, including process-based26, lumped27 and machine learning28 paradigms. These components cover rainfall-runoff29, snow30, evaporation, transpiration31, infiltration26, terrain analysis32, interpolation33, calibration25 and more. The system is designed for adaptability and stability and handles spatially disjoint catchments. GEOframe has been used to simulate tECVs at different scales from point to large catchments34 and in various regions35. For the GEOframe simulations in the reference dataset, all crucial process-based modules necessary for geomorphological analysis, water budget calculation (i.e. canopy, snow, soil moisture and groundwater) and river discharge routing are included. Furthermore, 18 parameters were optimized (in terms of Kling-Gupta Efficiency) using the calibration process of the GEOframe system framework36.

mesoscale Hydrologic Model (mHM)

mHM is a gridded distributed hydrological model developed by Helmholtz Centre for Environmental Research GmbH (UFZ)37,38,39,40. The model covers processes like canopy interception, snow accumulation and melt, soil moisture dynamics, infiltration, surface runoff, evapotranspiration, deep percolation, subsurface storage, baseflow, discharge generation, discharge attenuation and flood routing. The model source code is highly modular and is written in Fortran. A particular feature of mHM is the original implementation of the multiscale parameter regionalization scheme. This scheme allows model parameters to be transferred across scales and locations with great computational efficiency and a minimum decrement in performance41. mHM has been used to simulate tECVs in over 1 000 European basins and globally over 5 000+ GRDC basins of varying sizes and diverse spatial resolutions (from 1 by 1 km to 100 by100 km)42,43,44,45,46 and is currently operational in the ULYSSES C3S global seasonal forecasting system and in the German Drought Monitor47. For the mHM simulations in the reference dataset, two different global parameter sets were used in this study: the default parameterization for the Rhine, Po and Tugela river basins48 and a compromise solution, constrained over six randomly selected European basins, for continental Europe. This random selection of six basins was repeated a hundred times. Then, the best-performing parameter cross-evaluated over 958 European basins (in terms of the median Kling-Gupta Efficiency) was selected49.

Parflow-CLM

Parflow is an integrated subsurface and surface hydrological model that simulates 3D variably saturated groundwater flow using the Richards equation and incorporates a 2D overland flow as a free surface boundary condition50,51. As the standalone ParFlow cannot account for land surface processes (e.g., evapotranspiration and snow water equivalent), this model is generally coupled to the Common Land Model (CLM), which is a modified version of the original Common Land Model52. Note that the Common Land Model (CLM) is not the same land surface model as the community land model (see CLM section), which is the land component of the Community Earth System Model (CESM). CLM provides the sources and sinks for soil moisture to Parflow whereas, to calculate the land surface water and energy balances, CLM receives from ParFlow spatially distributed soil moisture and soil matric potential. Parflow-CLM has been applied to simulate tECVs from test sites to the continental scale, from sub-daily to climate time scales, and range from scientific research to quasi-operational use around the world50,53,54,55. The Parflow-CLM simulations in the reference dataset include the so-called DE0655 domain which extends over the Rhine basin. Here, the model considers a spin-up of initial conditions and states up to a dynamic equilibrium, so no calibration is required.

PCRaster Global Water Balance (PCR-GLOBWB)

PCR-GLOBWB is a grid-based global hydrology and water resources model developed at the Department of Physical Geography, Faculty of Geosciences, Utrecht University, the Netherlands56,57. PCR-GLOBWB describes the terrestrial part of the hydrological cycle. For each grid cell and each day, PCR-GLOBWB simulates moisture storage as well as the water exchange between the soil, atmosphere and underlying groundwater reservoir. River discharge is calculated by accumulating and routing specific runoff along the drainage network. A particular feature of PCR-GLOBWB is that human water use is fully integrated with the hydrological cycle. Livestock, industry and households water demands are estimated based on socio-economic developments whereas irrigation water demands are dynamically calculated. These demands are subsequently translated into actual withdrawals from groundwater and surface water (rivers, lakes and reservoirs) subject to water availability and the maximum groundwater pumping capacity58. Since the model was first introduced, PCR-GLOBWB has been applied extensively to simulate tECVs in global water resource assessments ranging from 30 arc-minutes (50 by 50 km at the equator) to 30 arc-seconds (1 by 1 km at the equator) spatial resolutions59,60,61,62,63,64,65,66,67. For the PCR-GLOBWB simulations in the reference dataset, the default model parameterization57 without any calibration is used.

TETIS

TETIS is a spatially distributed eco-hydrological model68,69, that has been under development by the Instituto de Ingeniería del Agua y Medio Ambiente (IIAMA) of Universitat Politècnica de València (UPV) since 1995. The model accommodates various spatial scales, from under 1 km2 to 60 000 km2, spatial resolutions and multiple climatic conditions worldwide. TETIS accounts for spatial heterogeneity in its inputs, parameters and state variables and offers sub-modules such as reservoir operation, snow accumulation and melting, sediment erosion, transport and deposition, nitrogen cycle, natural dynamic vegetation, flood and drip irrigation and crop production70,71,72,73,74. This version of TETIS employs the traditional vertical conceptualization first introduced in Tetis V9.1, which is concerned with the vertical movement of water between different tanks. Additionally, it incorporates the horizontal structure regarding the movement of water across the drainage network, as derived from the mHM model. TETIS has been employed to evaluation Sentinel-1, SMAP and SMOS surface soil moisture products for distributed eco-hydrological modeling75,76. For the TETIS simulations on the reference data set, the default geologic units of mHM were employed. The nine corrector factors of the Tetis model were calibrated against daily observed data from discharge gauges using the Shuffled Complex Evolution (SCE) algorithm to optimize the Kling-Gupta Efficiency.

Terrestrial Systems Modeling Platform (TSMP)

TSMP is a fully coupled terrestrial model closing the water and energy balance from groundwater across the land surface to the top of the atmosphere77,78. TSMP integrates Parflow-CLM (see Parflow-CLM subsection) to represent the land-surface and the Consortium for Small-scale Modeling (COSMO)79, the former weather prediction system of the German Weather Service, to represent atmospheric processes. The land-surface and atmospheric models are coupled via the external coupler OASIS3-MCT80, allowing TSMP to run in fully coupled mode (COSMO - ParFlow-CLM), where the lower boundary information for COSMO is provided by ParFlow-CLM. For more details, including the implementation over the European CORDEX domain, the reader is referred to Furusho-Percot et al.81. TSMP has been used to simulate fully coupled tECVs in Europe82,83. For the TSMP simulations in the reference dataset, no calibration was required. Instead, TSMP considers a spin-up of initial conditions and states up to a dynamic equilibrium.

wflow_sbm

wflow_sbm is an open-source distributed hydrological model developed by Deltares and written in Julia84. The soil setup of the model was originally based on the topog_sbm (soil bucket model) concept85. Wflow_sbm tries to balance low-resolution, low-complexity (conceptual) and high-resolution, high-complexity (physics-based) hydrological models. This is attained by having parameters based on physical characteristics or processes while keeping relatively fast run times. The wflow_sbm model includes hydrological processes such as glacier and snow processes, lakes and reservoirs (and their operations), evapotranspiration processes, unsaturated zone dynamics, lateral subsurface flow processes and multiple surface flow routing modules. Wflow_sbm models can be easily derived and parameterized for any catchment with the Python tool HydroMT-Wflow86. This tool uses globally available datasets and derives the river morphology (river network, flow directions, river width, etc.)87 and subsequently derives parameter values through the use of (pedo)transfer functions and suitable upscaling rules88. wflow_sbm has been applied to simulate tECVs around the world at various spatial resolutions89,90,91,92. For the wflow_sbm simulations in the reference dataset, the default parameter derivation has been applied without any further calibration.

Meteorological Datasets

The land-surface and hydrological models described above are forced, in total, with seven meteorological datasets, namely the E-OBS, EMO-1, ERA-5, ERA-I, HRES, COSMO-REA6 and measurement datasets. All models except for CLM, ParFlow-CLM and TSMP require precipitation, temperature and potential evapotranspiration to run. Potential evapotranspiration is calculated using Priestley Taylor93 for GEOframe, Hargreaves Samani94 for mHM and TETIS, Penman-Monteith95 for PCR-GLOBWB and and de Bruin96 for wflow_sbm. CLM, ParFlow-CLM and TSMP additionally require relative humidity, wind speed, surface pressure and surface radiation. TSMP uses these variables only as a boundary condition and dynamically simulates meteorology (see TSMP section).

E-OBS97,98 is a gridded ensemble dataset based on the interpolation of meteorological station observations from the European Climate Assessment & Dataset (ECA&D)99,100. The ensemble comes from stochastic simulations that produce an ensemble of realizations based on the residuals of the initial (deterministic) interpolation. E-OBS provides daily estimates of precipitation, temperature, sea level pressure, global radiation and wind speed for Europe. Data is available from 1950s until near real-time, at two spatial resolutions (0.1 by 0.1 degrees and 0.25 by 0.25 degrees).

European Meteorological Observations (EMO-1) is a gridded ensemble dataset based on the interpolation of a historical and real-time meteorological observations from a range of data providers101. EMO-1 provides (sub-)daily estimates of precipitation, temperatures (minimum and maximum), wind speed, solar radiation and water vapour pressure for Europe. Data is available from 1990 until 2022 for the newest EMO version, at 1 by 1 arc-minutes spatial resolution (EMO-1), which replaces the previous version at 5 by 5km spatial resolution (EMO-5).

ECMWF ReAnalysis 5 (ERA-5)102 is the latest generation of atmospheric reanalysis by the European Centre for Medium-Range Weather Forecasts (ECMWF). ERA-5 provides gridded hourly estimates of a wide range of meteorological variables throughout the atmosphere and at the land surface around the world. Data is available from 1940 until near real-time at 30 by 30km spatial resolution. ERA-5 replaces its predecessor ECMWF ReAnalysis Interim (ERA-I)103 which provided the same variables from 1979 to 2019 at a 80 by 80km spatial resolution.

ECMWF High Resolution (HRES) is a deterministic medium-range forecast104 based on the European Centre for Medium-Range Weather Forecasts (ECMWF) high-resolution atmospheric model. Each day, HRES offers a single gridded forecast that describes the evolution of the weather out to 10 days ahead at a spatial resolution of 0.1 by 0.1 degrees. Data is available from 2010-09 until near real-time.

COSMO ReAnalysis 6 (COSMO-REA6)105, is an atmospheric reanalysis using the Consortium for Small-scale Modeling (COSMO) model. COSMO-REA6 provides gridded hourly estimates of a wide range of meteorological variables throughout the atmosphere and at the land surface for Continental Europe. Data is available from 1995 until 2019 at 0.055 by 0.055 degrees spatial resolution.

measurements include ground-based daily precipitation and temperature observations over the Aosta Valley (the upper part of the Po River basin) for the period of 2005-2021106.

Data Records

Our hydrological reference dataset consists of land-surface and hydrological simulations with various configurations (Table 2). Simulations are available in Network Common Data Form (NetCDF) files. Files are named following their respective simulation in the format: model, meteorological forcing, variable, region, spatial resolution, temporal resolution, start date and end date, all lowercase, separated by underscores (_) and appended by the NetCDF extension (.nc). For example: pcrglobwb_ERA-I_et_europe_05min_daily_19900101_19901231.nc.

Table 2 Available simulations by model, meteorology, resolution, regions and variables.
Full size table

NetCDF files are consistently formatted following our storage protocol (see Supplementary Information). All latitudes and longitudes are reported using the World Geodetic System 84 (WGS 84) with the same bounds for each region. Most simulations start in the 1990s and end in the 2010s. Simulations are split into multiple yearly or monthly files to reduce the file size if necessary (e.g. for higher spatial resolutions). Four key hydrological variables are provided at a daily timestep, namely discharge (q; m3 s-1), evapotranspiration (et; kg m2 s-1), soil moisture content (sm; % of soil volume) and total water storage (tws; kg m2). Note that soil moisture can be reported for multiple soil layers with varying depths. Besides the hydrological variables, a catchment mask and upstream area are provided for each region to aid in the validation.

Technical Validation

Although the LSM/HM conceptualization, implementation and setup differ substantially within our hydrological reference dataset, they aim to simulate the same hydrological processes. Moreover, simulations are uniformly formatted following our storage protocol (see Data Records Section). Therefore, a comprehensive validation is warranted.

Validation Data

To validate our hydrological reference dataset, simulations are compared with observed data collected from discharge gauges, evapotranspiration towers, soil moisture stations and total water storage anomaly satellites (Fig. 1). Specifically, discharge data originate from the Global Runoff Data Centre dataset (GRDC)107, large-sample data for hydrology and environmental sciences in central Europe dataset (LamaH-CE)108, catchment attributes and meteorology for large-sample studies in Great Britain dataset (CAMELS-GB)109, catchment attributes and meteorology for large-sample studies in Switzerland dataset (CAMELS-CH)110 and various local datasets106,111,112,113,114. Evapotranspiration data are obtained from the FLUXNET 2015 dataset (FLUXNET2015)115, while soil moisture observations are taken from the International Soil Moisture Network dataset (ISMN)116,117. Total water storage anomaly data originate from the Gravity Recovery and Climate Experiment (GRACE) and its follow-on satellite mission (GRACE-FO)118. Four GRACE solutions are included: the Jet Propulsion Laboratory (JPL) spherical119,120 and mascon solution121, the University of Texas Center for Space Research (CSR) spherical solution122,123 and the GeoForschungsZentrum (GFZ) spherical solution124,125.

Fig. 1
figure 1

Discharge (blue), evapotranspiration (red) and soil moisture (cyan) observation locations for the four regions of interest: Europe, including the Rhine and Po basin, and the Tugela basin in Africa.

Full size image

Validation Approach

Evapotranspiration and soil moisture observations are compared directly to the simulated grid cell closest to the reported observation location. Note that these comparisons are inherently limited due to the nature of our data. Grid-cell outputs represent an average value across the entire grid cell, while point-observation outputs correspond to a specific location, in the case of soil moisture content, or a relatively small area in the case of evapotranspiration. Particularly important is that variations in soil parameters, such as soil porosity, between point observations and grid-cell averages can introduce significant biases in the simulated soil moisture content. Therefore, soil moisture observations are compared as anomalies by subtracting the mean over the comparison period.

A different approach was taken for the evaluation of the discharge observations. Rather than comparing discharge directly to the grid cell closest to the reported observation location, observations are matched to a neighbouring grid cell that best corresponds to the reported upstream area. The nearest grid cells within a quarter arc degree from the reported station location where the upstream area difference is smaller than a tenth (or the minimum) is selected. Some local discharge observation datasets did not provide information about the station’s upstream area. In such cases, we used the reported average discharge difference for the selection.

Model evaluations of total water storage anomalies are spatially averaged per region, leading to a single time series per region. We used this approach because satellite-based terrestrial water storage anomaly signals undergo spatial smoothing and filtering. This process can result in signals from neighbouring areas affecting the observations, also known as “leakage”126. Spatial averaging of the observation-based products helps minimise the leakage signals’ influence.

Simulation Selection

For our comparison, we made a sub-selection of the available observation data, considering the common simulation period and the common simulation geographical domain of each region and variable (Table 3). This sub-selection process ensures that every simulation is compared to the same set of observations in both temporal and spatial dimensions. The only exception is the soil moisture anomaly observations, as some simulations estimated soil moisture anomalies at greater depths than others. Therefore, more soil observations at depth could be included in these simulations, resulting in varying sets of soil moisture anomaly observations. Observations that contained less than two years of data during the common simulation period are excluded.

Table 3 Number of observations and their average duration, per variable, region and period, used in the validation comparison.
Full size table

Not all simulations could be consistently combined in the comparison, as the common simulation period or geographical domain would have been too small (see Table 2). The Geoframe model simulations are only available for the upper Po basin, resulting in a common geographical domain of the upper Po. Additionally, the Geoframe and Parflow-CLM model simulations are only available after 2015 and 2013, respectively, whereas the PCR-GLOBWB model simulations are only available until 2015, resulting in a common simulation period of a year. Therefore, two additional comparisons are made with their own sub-selections (Table 3). First, a discharge comparison is conducted for the upper Po basin that includes the GEOframe model simulations and excludes the PCR-GLOBWB model simulations. Second, a soil moisture and total water storage anomaly comparison is conducted for the Rhine that includes the Parflow-CLM model simulations and excludes the PCR-GLOBWB model simulations.

Performance Metrics

Performance is evaluated using the Kling-Gupta Efficiency (KGE)127 metric and its components. The KGE is calculated following Equation (1), where r is the correlation component (Equation (2)), α is the variability ratio component (Equation (3)) and β is the bias ratio component (Equation (4)). Note that o denotes the observations and s denotes the simulations

$$KGE=1-\sqrt{{\left(r-1\right)}^{2}+{\left(\alpha -1\right)}^{2}+{\left(\beta -1\right)}^{2}}$$
(1)
$$r=\frac{cov(o,s)}{{\sigma }_{o}{\sigma }_{s}}$$
(2)
$$\alpha =\frac{{\sigma }_{s}}{{\sigma }_{o}}$$
(3)
$$\beta =\frac{{\mu }_{s}}{{\mu }_{o}}$$
(4)

Validation Results

Figures 2 and 3 summarize the simulation performance across four regions and four variables. Additionally, maps of the spatial performance distribution and figures for the monthly deseasonalized performance are available in the Supplementary Information. In general, the simulations perform well compared to the observed data, although the performance differs between variables. The median Kling-Gupta Efficiency (KGE) falls within the range of 0.15 to 0.76 for discharge, soil moisture anomaly, evapotranspiration and total water storage anomaly, in that respective order. Moreover, the correlation between simulations and observations is good, with median correlations larger than 0.6 over all variables and regions.

Fig. 2
figure 2

Simulated daily discharge, evapotranspiration, soil moisture anomaly and monthly total water storage anomaly performance for each region. Performance is measured by the Kling-Gupta efficiency (KGE) and its components, as shown on the x-axis. The KGE bias ratio is omitted for the anomalies, as the bias is zero by definition. The dashed horizontal line indicates the optimal performance value, whereas the box color indicates the performance range. Colored points indicate the median performance for each model. Note that some models include multiple simulations with different meteorology and resolution settings.

Full size image
Fig. 3
figure 3

Additional simulated daily discharge, soil moisture anomaly and monthly total water storage anomaly performance for each region. Additional performances have a different spatiotemporal subselection (see Technical Validation Section). Performance is measured by the Kling-Gupta efficiency (KGE) and its components, as shown on the x-axis. The KGE bias ratio is omitted for the anomalies, as the bias is zero by definition. The dashed horizontal line indicates the optimal performance value, whereas the box color indicates the performance range. Colored points indicate the median performance for each model. Note that some models include multiple simulations with different meteorology and resolution settings.

Full size image

Over all variables and regions, the median bias ratio (0.97) is generally better than the median variability ratio (0.84), indicating that the simulations are better at capturing the mean than the deviation in the observations. Additionally, over all variables, the median KGE in the Po and Rhine river basins (0.32 and 0.36, respectively) is generally better than in continental Europe and the Tugela river basin (0.23 and 0.17, respectively). This regional difference is likely related to the availability of observations and the subsequent higher-quality input data (e.g. meteorology and soil characteristics) for these regions.

Where the median values indicate the simulation performance, their variability indicates the uncertainty in the simulations. Here, we assess the uncertainty introduced by the model selection and region selection separately. For discharge, total water storage anomaly and soil moisture anomaly outputs, the model selection introduces the most variability. These variables show a larger median KGE interquartile range (1th to 3rd quantile) for each region encompassing all models (0.73, 0.18 and 0.34, respectively) than for each model containing all regions (0.71, 0.12 and 0.31, respectively). For discharge, the Tugela basin exhibits the most model-induced variability (0.9 interquartile range), whereas for total water storage and soil moisture anomaly, the Po basin exhibits the model-induced variability (0.4 and 0.6 interquartile range, respectively). We speculate that the reason for this model uncertainty is the presence or absence of lateral groundwater flow and human impact modules in the models. However, a harmonized model validation experiment is needed to confirm these speculations.

Conversely, for evapotranspiration outputs, the region selection introduces the most variability. This variable shows a larger median KGE interquartile range for each model encompassing all regions (0.26) than for each region containing all models (0.15). The same holds for the deseasonalized performance (see Supplementary Information). The reason for this regional uncertainty is likely because most models implement a similar evapotranspiration scheme that is strongly linked to meteorology. Therefore, regional uncertainties in meteorology are the main driver for uncertainties in evapotranspiration simulations.

Usage Notes

Although each modeling group has stored their simulations on a public repository, all simulations are also referenced following the SpatioTemporal Asset Catalog (STAC) protocol on the 4DHydro open science catalog: 4dhydro.eu/catalog/. Simulations can be found under products and then under working package 2. This catalog ensures simulations are discoverable by end-users, both the scientific community and the general public.