Introduction

Studies of past climate during the last two millennia depend on proxy reconstructions, such as tree rings, corals, ice cores, ocean and lake sediments1,2, and earth system model simulations3,4. Combing advantages from proxy records and model simulations, paleoclimate data assimilation (PDA) uses climate states simulated by models to infer new information from proxy records, and spatially propagate that information through all climate variables that are dynamically constrained by the climate model5.

For the last two millennia, Hakim et al.5 develop a flexible PDA system and generate the Last Millennium Reanalysis (LMR), with significant skill over tropical regions but less skill over Northern Hemisphere continental areas. Steiger et al.6 reconstruct annual to seasonal temperature and other climate variables over the Common Era through PDA, and examine the causes of climate variability. Based on LMR version 1 (LMR1)5, Tardif et al.7 use an updated proxy database and forward models based on seasonal regression to improve the reconstructed thermodynamic and hydroclimatic variables in mid-latitudes, known as LMR version 2 (LMR2). In addition to surface air temperature anomalies7,8,9, PDA has been used to reconstruct geopotential height fields7, El Niño index10,11, sea ice extent12,13, sea level pressure and wind12,14, and hydroclimatic variables7. Based on the improved data treatment strategy and a new method to assimilate short time series, Valler et al.15 also generate a new Modern Era Reanalysis (ModE-RA) including global monthly temperature, pressure, precipitation, geopotential height and wind reconstructions for the period 1421-2008. Beyond the last two millennia, the PDA is also applied to reconstruct the sea surface temperature evolution since the Last Glacial Maximum16,17. Among these variables, the skill of temperature and circulation reconstructions have been widely assessed, however, there are only a few previous studies7,15 assessing the skill of precipitation reconstructions.

In PDA system, important components that impact the quality of the reconstructions are the proxy records, model simulations serving as the prior18,19, the forward model that maps the variables from climate model output to proxy measurement (a.k.a., proxy system model, PSM), and the data assimilation method. Because one major aim of this study is to investigate the sensitivities of reconstruction performances to the number and location of proxy records and the assimilation methods, progress towards these two aspects are briefly reviewed in the following. In terms of proxy records, two proxy data sets2,20 are commonly used in the early PDA during the last two millennia, and updated proxy data sets, e.g., CoralHydro2k21, are also included in recent PDA.

In terms of assimilation methods, the most commonly used PDAs include the offline ensemble Kalman filter (OEnKF)5,6,7,22, and offline particle filter23. There are modified PDA methods, such as particle filter assimilation based on sequential importance resampling23,24, and offline paleoclimate data assimilation method combining analog assimilation methods and Kalman filters25. Recently, two improved OEnKF methods have been developed with the ability to capture some aspects of the “state-dependent” background error covariance, i.e., Analog Offline EnKF (AOEnKF)26 and Hybrid Gain Analog Offline EnKF (HGAOEnKF)27, besides the previous online data assimilations28,29. These methods could produce statistically significant error reductions due to more accurate prior ensemble mean, especially for larger sample sizes, greater observation uncertainties and sparser observing networks26,27.

Many studies have conducted PDA to reconstruct the climate in the last two millennia using a single PDA method and proxy record data set, but the influences from different assimilation methods and the number and locations of proxy records on the reconstructions need to be further assessed despite recent progress30. Therefore, the aim of this study is to systemically assess the performances of the OEnKF and the HGAOEnKF with three different proxy record databases (Fig. 1), especially for precipitation, and then reconstruct temperature and precipitation over the last two millennia. This new reanalysis product (NNU-2ka Reanalysis) is generated based on HGAOEnKF with an extended proxy database. The sensitivities of the reconstructions to numbers and locations of proxy records are also evaluated.

Fig. 1: Proxy data networks and temporal availability.
figure 1

Locations (a, c, e) and temporal distributions (b, d, f) of proxy records available for assimilation. Panels a, b are for PAGES2k database (602). Panels (c, d) are for PAGES2k + M08 database (834). Panels d, e are for the comprehensive database (1195). The orange, green, and blue symbols represent proxy records from the PAGES2k dataset, the M08 dataset, and the other datasets, respectively.

Full size image

Results

Validating against instrumental reanalyses

The reconstructed global mean temperature (GMT) based on the two assimilation methods with three proxy databases are first compared with the instrumental reanalyses (details in Table 1) for the period 1880-2000 (Fig. 2). All reconstructed GMT have similar characteristics, with a significant warming trend and decadal variations, consistent with the instrumental reanalyses. The correlation coefficients between the reconstructions and instrumental reanalyses are around 0.9. The consistent, high skill indicates that the reconstructions are relatively robust to the choice of proxy database or assimilation method.

Table 1 Gridded data sets used for the calibration of proxy system models (GISTEMP and GPCC) and for the verification of reconstructed results
Full size table
Fig. 2: Validation of reconstructed GMT time series with the instrumental reanalyses for the period 1880-2000.
figure 2

Comparisons between temperature reconstructions using PAGES2k (a), PAGES2k + M08 (b), and the comprehensive database (c) and instrumental reanalyses for the period 1880–2000. The correlation coefficients from OEnKF are in blue, and correlation coefficients from HGOEnKF are in black.

Full size image

In order to further compare the skill of different methods for reconstructing temperature, the spatial patterns of RMSE from reconstructed temperature relative to instrumental reanalyses are examined. The results based on the Berkeley Earth Surface Temperatures (BE) data set (Rohde et al., 2013) are shown in Fig. 3 as examples, and results based on five other data sets are similar (Figs. S1-S5). Reconstructions from OEnKF show higher RMSEs at high latitudes and lower RMSEs at middle to low latitudes. When the HGAOEnKF method is applied, the RMSE at high latitudes of the northern hemisphere is reduced by 0.04 k to 0.12 k, with larger improvements with PAGES2k + M08 and the comprehensive data bases (Fig. 3j, k, l). These improvements are smaller at tropical region ranging from 0.03 k to 0, indicating comparable skill between two methods. The RMSE at Southern Ocean is increased by 0 to 0.06 k, with the smallest differences with the comprehensive data base (Fig. 3j, k, l). Most of differences at high latitudes and tropical region are significant (p < 0.05), except for the reconstructions with PAGES2k (Fig. 3j).

Fig. 3: Spatial patterns of RMSE from reconstructed temperature relative to instrumental reanalyses.
figure 3

Spatial distributions of RMSE from OEnKF (a–c), HGAOEnKF (d–f), their differences (g–i), and corresponding zonal mean RMSE (j–l) relative to the BE data set. The reconstructions using PAGES2k, PAGES2k + M08, and the comprehensive database are shown in left column, middle column, and right column, respectively. In (j–l), the blue lines represent the zonal mean RMSE from OEnKF, and red lines represent zonal mean RMSE from HGAOEnKF. The differences between OEnKF and HGAOEnKF are shown in green lines. The circle indicates the difference is significant at 0.05 level.

Full size image

Validation against independent proxy records

For the independent validation of the DA methods, climatological mean values of the 25% independent proxy records not used in the DA (i.e., out-of-sample proxy records) and predicted values are compared. The predicted values are estimated based on the corresponding reconstructions through the proxy system models. It is found that the climatology of the out-of-sample proxy records and the reconstructions are close, with correlation coefficients nearly 1 in both DA methods (Fig. S6).

The correlation coefficients, coefficients of efficiency, and RMSE between reconstructions and the out-of-sample proxy records are then examined (Table 2). The correlation coefficients range 0.2-0.3 for tree-ring records, 0.01-0.51 for coral records, and 0.01-0.24 for ice-core records, with mean values ranging 0.17-0.28. The coefficient of efficiency differences range 0.64-1.13 for tree-ring records, 0.01-1.41 for coral records, and -0.29-0.44 for ice-core records, with mean values ranging 0.24-1.22. The absolute CE values from both assimilation methods are all negative (Table S1), indicating that both assimilation methods do not perform well on this metric, although these CE values are higher than the CE values from the prior.

Table 2 Verification of OEnKF with PAGES2k, OEnKF with PAGES2k + M08, OEnKF with the comprehensive database, HGAOEnKF with PAGES2k, HGAOEnKF with PAGES2k + M08, HGAOEnKF with the comprehensive database against the 25% independent proxy records, i.e., the mean of correlation coefficient (Cor), coefficient of efficiency difference (posterior CE minus prior CE, ∆CE), and RMSE between the standardized proxy value and the standardized predicted value, across all the proxy records
Full size table

The RMSE ranges 0.23-0.24 for tree-ring records, 0.22-0.31 for coral records, and 0.22-0.24 for ice-core records, with mean values of about 0.23. These comparisons show that both assimilation methods generally reproduce the variability of the out-of-sample proxy records. Meanwhile, there are still some low values in correlation coefficients and negative CE values potentially due to the uncertainties among the proxy records, indicating the reconstruction skill is low at these specific sites.

When using the same proxy database, the reconstruction skill scores for most out-of-sample proxy records are improved by the HGAOEnKF method and none are degraded (Table 2). When using different proxy databases, increase in the proxy record amounts and quality will improve reconstruction skill for each DA method, which are reflected in the increases of correlation coefficients, and the decrease of RMSEs across the reconstructions. The largest ∆CE of tree-ring records are from PAGES2k database, followed by the comprehensive database. The largest ∆CE of coral and ice-core records are from the comprehensive database, followed by the PAGES2k database.

Global temperature and precipitation evolution

Figure 4 compares reconstructed GMT and global mean precipitation (GMP) based on the two assimilation methods with three proxy databases. Reconstructed GMT based on the two assimilation methods have similar characteristics, with a cooling trend through most of the last two millennium until the warming after 1850s, consistent with the raw proxy records1. On centennial scales, there is a relatively warm Medieval Climate Anomaly (MCA) and a cold Little Ice Age (LIA). The variability of reconstructed GMT with HGAOEnKF method is larger than that with OEnKF (Fig. 4a, b). Reconstructed GMT mean values during the period 0-900 CE from three proxy data sets with HGOEnKF are -0.23 K, -0.20 K, -0.22 K, respectively, which are significantly larger than those with OEnKF, -0.11 K, -0.10 K, -0.09 K. Due to the advantages of analog ensembles with more informative prior information and “state-dependent” background error covariances (Sun et al., 2022&2024), the posterior with analog ensembles and hybrid gain assimilation strategy could be superior to the traditional OEnKF. Reconstructed northern hemisphere mean temperature from the six reconstructions are similar to those from proxy reconstructions and LMR, with similar centennial to decadal scale variations (Fig. 4c). Correlation coefficients between reconstructed northern hemisphere mean temperature from OEnKF and LMR are 0.72, 0.75, 0.72, and the correlation coefficients are 0.58, 0.63, 0.58 for HGAOEnKF.

Fig. 4: GMT and GMP evolutions during last two millennia.
figure 4

Comparison of the reconstructed global mean 2 m air temperature (GMT) over the Common Era from OEnKF with PAGES2k, OEnKF with PAGES2k + M08, OEnKF with the comprehensive database (a), and HGAOEnKF with PAGES2k, HGAOEnKF with PAGES2k + M08, HGAOEnKF with the comprehensive database (b). Comparison of the Northern Hemisphere 2 m air temperature from the six sets of reconstructions with reconstructions from other authors, i.e., LMR2 (Tardif et al., 2019), Neukom_AM, Neukom_CCA, Neukom_CPS, Neukom_DA, Neukom_GraphEM, Neukom_PCR (Neukom et al., 2019), MBH1999 (Mann et al., 1999), MJ2003 (Mann and Jones, 2003), RMO2005 (Rutherford et al., 2005), MSH2005 (Moberg et al., 2005), Ma08eivf (Mann et al., 2008), Ma09regm (Mann et al., 2009) (c). Comparison of the global mean precipitation (GMP) over the Common Era form OEnKF with PAGES2k, OEnKF with PAGES2k + M08, OEnKF with the comprehensive database (d), and HGAOEnKF with PAGES2k, HGAOEnKF with PAGES2k + M08, HGAOEnKF the comprehensive database (e). The solid lines represent grand ensemble mean, and the shading represent the 5th-95th percentile range. All series in panel (c) represent anomalies from the 1900–1990 mean and have been smoothed with a 31-year running mean.

Full size image

The main characteristics of reconstructed GMP are similar to GMT, with positive anomalies during MCA and present warming periods and negative anomalies during LIA (Fig. 4d, e). The correlation coefficients between GMP from OEnKF and LMR are 0.59, 0.65, 0.65, and the correlation coefficients are 0.23, 0.40, 0.36 for HGAOEnKF. In contrast with temperature reconstructions, reconstructed precipitation variability is larger for the OEnKF than that for the HGAOEnKF. This is because, in HGAOEnKF, the analog ensembles are selected based on the correlations with the proxy records that are sensitive to temperature. Thus, the analog ensembles could have smaller magnitudes of correlations with precipitation compared to those randomly sampled climatological ones, which leads to less variability of reconstructed precipitation of HGAOEnKF than OEnKF. This also explains the lower skill of precipitation reconstruction using HGAOEnKF than OEnKF, similar to the GMT. When comparing with LMR, all the reconstructed GMP show similar centennial to decadal scale variations (Fig. 4f).

Sensitivity experiments

To further examine the influences of the proxy record amount on the assimilation results, the proxy records available at 500 CE (33 proxy data), 1000 CE (67 proxy data) and 1500 CE (210 proxy data) from the PAGES2k dataset (Fig. 5a–c), are chosen to reconstruct temperature by OEnKF and HGAOEnKF methods. For the instrumental era, with an increase of proxy record amount, the correlation coefficients between the reconstructed and observed temperature increase (Fig. 5d), with higher CE (Fig. 5e) and lower RMSE (Fig. 5f). The overall reconstruction skill of HGAOEnKF is larger than that of OEnKF. The reconstructions based on only 33 suitably distributed proxy data with HGAOEnKF can achieve a correlation coefficient of about 0.3, which is similar to the correlation coefficients of 1000 CE and 1500 CE proxy data network assimilation (Fig. 5b), and equal to about 50% of the correlation coefficient of 2000CE proxy data network assimilation with 692 proxy records. The CE values of reconstructions based on HGAOEnKF are usually closer to 0 than those based on OEnKF (Fig. 5e), while the RMSE of both assimilation methods are smaller than 0.8, with smaller values based on HGAOEnKF with more proxy records (Fig. 5f). When averaged zonally, there is relative high skill at tropical to mid-latitude regions and low skill at northern high-latitude regions (Fig. S7). This also confirms that HGAOEnKF can effectively extract information from the sparse proxy data compared to OEnKF, because of more accurate mean prior and more realistic “state-dependent” background error covariances.

Fig. 5: Proxy data networks, temporal availability, and assessments of reconstructions at 500CE, 1000CE, and 1500CE.
figure 5

Locations and temporal distributions of PAGES2k proxy records available at 500CE (a), 1000CE (c), and 1500CE (e), and the global latitudinal weighted mean correlation coefficients (b, g), CE (d, h), and RMSE (f, i) the reconstructed temperature and precipitation fields with respective to observed data sets, i.e., 20CR (Compo et al., 2011), BE (Rohde et al., 2013), GISTEMP (Hansen et al.,43), HadCRUT4 (Morice et al., 2012), MLOST (Smith et al., 2008), NOAA (Smith et al., 2008), GPCC (Schneider et al.,44) in instrumental era.

Full size image

However, sensitivity experiments show that the skill of precipitation reconstructions to the number of proxy records are different from temperature reconstructions. The correlation coefficients between the reconstructed and observed precipitation decrease (Fig. 5g), with lower CE (Fig. 5h) and comparable RMSE (Fig. 5i) with the increase of proxy record amount. The latitudinal distribution of the skill shows that there is relatively high skill at mid-latitude regions and low skill at tropical and southern high-latitude regions (Fig. S8). This indicates that the skill of precipitation reconstructions is not sensitive to the number of proxy records, probably because the tree-ring and coral records are more sensitive to temperature than precipitation. The lower skill of precipitation reconstructions also implies necessity for systematic improvements to future data assimilation effects to reconstruct precipitation.

The influences of the proxy record locations on the reconstructions are also investigated. The PAGES2k database is divided into four major regions, i.e., Asia land (0-80 °N, 25-170 °E), Europe (35-72 °N, 10 °W - 66 °E), North America (9-85 °N, 168-10 °W), ocean (Fig. S9). The assimilation is then performed using proxy records from three regions with one region left out, to examine contributions of proxy record locations to the reconstruction skill. The correlation coefficients between the reconstructed GMT from OEnKF and instrumental GMT are 0.92, 0.91, 0.92, and 0.84, respectively, with the smallest correlation coefficient from the assimilation without ocean (Fig. S10). The correlation coefficients between the reconstructed GMT from HGAOEnKF and instrumental GMT are 0.89, 0.88, 0.89, and 0.83, respectively, with the smallest correlation coefficient also from the assimilation without ocean (Fig. S10). This indicates that the proxy records from ocean contribute most to the skill of GMT reconstruction with both assimilation methods, followed by comparable contributions from three land regions. Because the number of proxy records from the ocean are not the largest, this also indicates that the location (i.e., the ocean) of proxy records contributes more to the increased skill than the number of proxy records, in this sensitivity experiment.

Discussion

In this study, six reconstructed temperature and precipitation fields in the past 2000 years are generated by applying two DA methods (OEnKF and HGAOEnKF) on three proxy record databases (PAGES2k, PAGES2k + M08, and the comprehensive database). The assimilation results are then compared with instrumental observations and out-of-sample proxy records as well as previous assimilation products.

In the instrumental era, the six reconstructed temperature products have comparable skill with similar correlation coefficients, CEs, and RMSEs, respective to the observed temperature. The reconstructed temperature using OEnKF has positive RMSEs at high latitudes and negative RMSEs at middle and low latitudes. When the HGAOEnKF method is used, this contrast between high latitudes and tropics is reduced. Furthermore, the HGAOEnKF also increases the reconstruction skill in most places.

In validation with out-of-sample proxy records, the spatial distributions of correlation coefficients of OEnKF and HGAOEnKF reconstructions are similar when using the same proxy databases, and HGAOEnKF improved the reconstruction skill of coral proxy records. For both assimilation methods, increasing the number of proxy record significantly improves the reconstruction skill, with larger improvements for the HGAOEnKF method. Through the whole period, both reconstructed temperature and precipitation show similar centennial to decadal scale variations with previous proxy reconstructions and LMR, which are induced by the solar radiation and volcanic eruptions31.

In the pre-instrumental era, the performance of temperature reconstructions with both assimilation methods increases with the number of proxy records. The results of sensitivity experiments show that, with the HGAOEnKF method, the reconstructions with only 33 proxy records at 500 CE and an appropriate distribution around the world can achieve a correlation coefficient about 0.3, a CE close to 0, and a RMSE less than 0.8. Therefore, the HGAOEnKF method shows a higher fidelity for temperature reconstructions especially in the earlier period when the proxy network is sparser. However, the general skill for precipitation reconstructions is lower, indicating that this is an important aspect to improve upon in future paleoclimate data assimilation study. Further sensitivity tests show that the skill of GMT reconstructions with both assimilation methods are most sensitive to the proxy records from ocean.

Methods

PDA methods

Two PDA methods, the OEnKF5,6 and the HGAOEnKF26, are used in this study. The details of each method are briefly introduced as following.

The OEnKF method is a widely used paleoclimate data assimilation reconstruction of climate during the last millennium5,6. This method solves the following equation to calculate an ensemble of climate states (xa) from prior states taken from a climate model (xb) and proxy records (y):

$${{\boldsymbol{x}}}_{{\rm{a}}}={{\boldsymbol{x}}}_{{\rm{b}}}+{\rm{K}}\left({\boldsymbol{y}}{-}{{\boldsymbol{y}}}_{{\rm{e}}}\right)$$
(1)

xb is an ensemble of climate states taken from model simulations, with a dimension of N state vector elements (temperature and precipitation) by M ensemble members (100, in this study). y − ye is the innovation, i.e., the difference between the column vector y (dimension of P × 1) of observed proxy values for each location P and the matrix ye (dimension of P × M) of proxy values calculated from the model prior at the same location. The innovation is the new information from the proxy records not already contained in the prior. This new information is weighted against the prior through the Kalman gain matrix:

$$K=B{H}^{T}{[{HB}{H}^{T}+R]}^{{-}1}$$
(2)

where B is the prior covariance matrix, R is the error covariance matrix for the proxy data, and H is the linearization of the PSM about the mean value of the prior.

Solutions for the ensemble mean \(\bar{{x}_{a}}\), and perturbations \({x}_{a}^{{\prime} }\,\) for the single kth proxy \({{\boldsymbol{y}}}_{k}\), are obtained from

$${\bar{{\boldsymbol{x}}}}_{a}={\bar{{\boldsymbol{x}}}}_{b}+\frac{{{\mathfrak{w}}}_{{loc}}\diamond {\mathrm{cov}}\left({{\boldsymbol{x}}}_{b},\,{\bar{{\boldsymbol{y}}}}_{e,k}\right)}{\mathrm{var}({{\boldsymbol{y}}}_{e,k})+{R}_{k}}\left({{\boldsymbol{y}}}_{k}{-}{\bar{{\boldsymbol{y}}}}_{e,k}\right)$$
(3a)
$${{{\boldsymbol{x}}}^{{\prime} }}_{a}={{{\boldsymbol{x}}}^{{\prime} }}_{b}{-}{\left[1+\sqrt{\frac{{R}_{k}}{\mathrm{var}\left({{\boldsymbol{y}}}_{e,k}\right)+{R}_{k}}}\right]}^{{-}1}\frac{{{\mathfrak{w}}}_{{loc}}\,\diamond {\mathrm{cov}}\left(\,{{\boldsymbol{x}}}_{b},\,{\bar{{\boldsymbol{y}}}}_{e,k}\right)}{\mathrm{var}\left({{\boldsymbol{y}}}_{e,k}\right)+{R}_{k}}({{{\boldsymbol{y}}}^{{\prime} }}_{e,k})$$
(3b)

where \({{\boldsymbol{y}}}_{e,k}\) is the prior estimate of the proxy, \({R}_{k}\) is the diagonal element of R corresponding to proxy \({{\boldsymbol{y}}}_{k}\), and cov() and var() represent the covariance and variance functions, respectively. Covariance localization, given by a Schur product denoted by ◦ in Eq. (3b) (i.e., element-wise multiplication), is a distance-weighted filter wloc on the prior covariance matrix32.

The ensemble of updated states is then recovered by combining the posterior ensemble mean and perturbations:

$${{\boldsymbol{x}}}_{a}={\bar{{\boldsymbol{x}}}}_{a}+{{\boldsymbol{x}}}_{a}^{{\prime} }$$
(4)

The HGAOEnKF27 a linear superposition of Analog Offline EnKF (AOEnKF) and Analog Offline EnKF with Static Background Error Covariance (AOEnKF-B). The AOEnKF is an updated version of OEnKF, and it selects the ensemble priors from the prior set \({{\boldsymbol{x}}}_{{\rm{prior}}}\) based on a criterion that measures the “similarity” between the sample state and the proxy record26. Compared to OEnKF with the same randomly drawn ensemble priors for each assimilation time, AOEnKF has different ensemble priors for each assimilation, which provides somewhat “state-dependent” prior estimates of model state. Meanwhile, AOEnKF still maintains the computational efficiency of the traditional OEnKF26.

At each assimilation time, proxy vector y is used to select the ensemble priors \({{\boldsymbol{x}}}_{b}\). One criterion used in AOEnKF for selecting ensemble priors \({{\boldsymbol{x}}}_{b}\) is based on the root-mean-square error (RMSE) of the state variables from the proxy records. The results using correlation criterions are consistent, as found in Sun et al.26. For each sample of state vector \({x}_{j}\) (j = 1, …,T), the RMSE of \({x}_{j}\) relative to y is

$${\rm{RMSE}}\left({\rm{j}}\right)=\sqrt{\frac{{\sum }_{{\rm{m}}=1}^{{\rm{p}}}{({{\boldsymbol{y}}}_{{\rm{m}}}{-}H({{\boldsymbol{x}}}_{{\rm{j}},{\rm{m}}}))}^{2}}{{\rm{p}}}}$$
(5)

By sorting the RMSE of Eq. (5) in an ascending order, the first M samples of the state vector \({X}_{b}\,\) is the selected as ensemble priors for this assimilation time.

Although there are improvements of AOEnKF over OEnKF due to the “state-dependent” background error covariances computed from the analog ensembles, this “state-dependent” background error covariances might be contaminated by sampling errors since limited ensemble members are used26. Thus, an AOEnKF with static background error covariance (AOEnKF-B) is also implemented. It uses the analog prior ensemble \({{\boldsymbol{x}}}_{n\times M}\,\) as prior state estimate (\({{\boldsymbol{x}}}_{{\rm{b}}}\) and \({{\boldsymbol{y}}}_{{\rm{e}}}\) in Eq. (1)), but the whole sample set \({{\boldsymbol{x}}}_{b}\,\) as background error statistics (\({\rm{K}}\) in Eqs. (1) and (2)).

Similar to AOEnKF, HGAOEnKF uses the analog prior ensemble \({{\boldsymbol{x}}}_{b}\) that contains members having large correlations with the observations to construct the prior information. HGAOEnKF combines the “state-dependent” background error covariance estimated from the analog prior ensemble \({{\boldsymbol{x}}}_{n\times M}\) and the static background error covariance from the whole sample set \({{\boldsymbol{x}}}_{b}\) through a hybrid gain approach in an ensemble framework. The ensemble mean update is

$${{\boldsymbol{x}}}_{a}={{\rm{\alpha }}* {\boldsymbol{x}}}_{{AOEnKF},a}+(1{-}{{\rm{\alpha }})* {\boldsymbol{x}}}_{{AOEnKF}{-}B,a}$$
(6)

where α and 1 − α are the weights for the “state-dependent” and static background error covariances, respectively. The value of α is tuned by the minimum error obtained, as shown in Sun et al.27. Note that when analog ensemble size M equals climatological sample size, HGAOEnKF has the same posterior solution as OEnKF-B, AOEnKF-B, and AOEnKF, due to the same prior mean and prior perturbations.

The analog ensembles selected by RMSE in Eq. (5) could be more correlated and more likely be dependent than randomly drawn ones. The covariance localization can help increase the rank of the “state-dependent” background error covaraince matrix, and the hybridization of the background error covariances in Eq. (6) can also increase the rank of the “state-dependent” one. Results from Sun et al.26,27 demonstrate the advantages of analog ensembles compared to randomly drawn ensembles, and HGAOEnKF is naturally adopted in this study for the last two millennium reanalysis.

Proxy records

Three sets of proxy records are used, i.e., PAGES2k database (PAGES 2k consortium, 2017), PAGES2k + M08 database combining the PAGES2k and proxy records from Mann et al. 20 (·a.k.a., M08), and the comprehensive database combining PAGES2k, M08, LMRdb1.0 database33, and Iso2k database34. The PAGES2k dataset contains 441 tree rings, 37 ice cores, 91 corals, and 33 others, with median temporal resolutions of 1, 1, 1, and 3 years, respectively. The PAGES2k + M08 database contains 709 tree rings, 88 ice cores, 37 corals, with median temporal resolutions of 1, 1, and 1 year, respectively. The comprehensive database contains 968 tree rings, 85 ice cores, 142 corals, with median temporal resolutions of 1, 1, and 1 year, respectively. Figure 1 shows the spatial and temporal distributions of the three proxy networks. Before preparing the PAGES2k + M08 and comprehensive databases, the screening has been applied to proxy records from original databases. The overlaps between the different databases are identified, and only one record in each overlap is used in the following two new databases. Then, additional quality control is applied, and only proxy records with significant correlation coefficients (p < 0.05) with instrumental temperature data are kept. The full proxy databases and additional proxy metadata are available to public at the National Oceanic and Atmospheric Administration website (https://www.ncei.noaa.gov/pub/data/paleo/ and https://www.ncdc.noaa.gov/paleo-search/study/).

Then, we conduct a group of analyses to investigate the sensitivity of PDA to the number of proxy records. The reconstructions based on three databases, i.e., PAGES2k, PAGES2k + M08, and the comprehensive database, with each DA method are compared.

Prior data generation

For all data assimilation experiments in this study, to be consistent with previous studies5,26,27, the prior state vectors are formed using the Community Climate System Model version 4 (CCSM4) last millennium simulation with resolution of 1.875°×2.5° from Coupled Model Intercomparison Project phase 5 (CMIP5) archive35. This simulation covers the period 850 to 1850 CE, and includes external forcings such as incoming solar radiation, greenhouse gas concentrations, as well as stratospheric aerosols from volcanic eruptions etc.3. The sampled states are anomalies from the mean over the entire simulation. The prior ensemble mean does not contain time-specific information about climate events (e.g., a volcanic eruption) or trends characterizing specific periods (e.g., 20th century warming). Consequently, all trends and temporal variability in reconstructed fields result from the proxy records.

PSMs

One important procedure of PDA is to use simulated climate variables to estimate proxy observations (\({\mathcal{H}}({x}_{b})\) in Eq. 2). For example, simulated temperature and precipitation values can be used to estimate tree-ring width through a proxy system models (PSMs)36. Despite recent progresses in the development and applications of process-based PSMs, the traditional statistical PSMs are still the most popular within the PDA with tree-ring and coral records6,37,38,39,40,41,42. Consistent with LMR27, univariate and bivariate statistical PSMs are considered in this study:

$${y}_{k}={\beta }_{0k}+{\beta }_{1k}\overline{{X}_{1}^{\prime} }\,+\,{\epsilon }_{k}$$
(8)

and

$${y}_{k}={\beta }_{0k}+{\beta }_{1k}\overline{{X}_{1}^{\prime} }+{\beta }_{2k}\overline{{X}_{2}^{\prime} }\,+\,{\epsilon }_{k}$$
(9)

where \({y}_{k}\) are annualized observations from the kth proxy time series, \({X}_{1}^{{\prime} }\), \({X}_{2}^{{\prime} }\) are anomalies with respect to the mean over a reference period (1951-1980) of key climate variables (e.g., temperature and precipitation) from calibration data sets during instrumental era, \({\beta }_{0}\) is the intercept, \({\beta }_{1}\) and \({\beta }_{2}\) are the slopes with respect to the \({X}_{1}^{{\prime} }\) and \({X}_{2}^{{\prime} }\) independent variables, respectively, and \(\epsilon\) is a Gaussian random variable with zero mean and variance σ2. The overbars in Eqs. (8) and (9) denote annual averages, or over appropriate seasonal intervals for the seasonal PSMs. Calibration data concurrent with available proxy records are taken at the grid point nearest the proxy location, and the appropriate least-squares solution determines regression parameters (\({\beta }_{0}\), \({\beta }_{1}\), \({\beta }_{2}\), σ). As LMR2, PSM configuration is consistent for each proxy category, e.g., univariate PSMs for all coral δ18O records, bivariate PSMs for all tree-ring width records.

The PDAs with regression-based PSMs measure the diagonal elements in matrix R through the variance of regression residuals. This is a key parameter in PDA, because it determines the extent, to which the information provided by the proxy is weighted against prior information in the resulting reanalysis. For example, a record with a poor fit to calibration data will be characterized by larger residuals, hence larger error variance and less weight, relative to a record that has a stronger correlation with calibration data.

The calibration data sets used in this study include the NASA Goddard Institute for Space Studies (GISS) Surface Temperature Analysis (GISTEMP) version 4 temperature data set43, and the Global Precipitation Climatology Centre (GPCC) version 6 precipitation data set44. Table 1 summarizes these two calibration data sets, and the data sets used for verification of reconstructed results in this study.

Validation and bias analysis

In this study, to validate the reconstructions, we randomly select 75% of the proxy records from each database to the PDA procedure. The remaining 25% of the proxy records are used for independent verification, which is performed over the pre-instrumental era and instrumental era. For each database, this selection procedure is repeated for ten times, within which 100 Monte Carlo realizations are performed based on the same proxy records.

Three metrics, i.e., correlation coefficient, coefficient of efficiency (CE), and root-mean-square error (RMSE) are used in the validations. Given a time series of n values for reconstructions \(x\) and observation \(v\), the correlation coefficient is defined by

$${corr}=\,\frac{1}{n}\mathop{\sum }\limits_{i=1}^{n}\frac{({x}_{i}{-}\bar{x})({v}_{i}{-}\bar{v})}{{\sigma }_{x}{\sigma }_{v}}$$
(10)

and the CE45 is defined by

$${CE}=1{-}\frac{{\sum }_{i=1}^{n}{({v}_{i}{-}{x}_{i})}^{2}}{{\sum }_{i=1}^{n}{({v}_{i}{-}\bar{v})}^{2}}$$
(11)

Here an overbar represents a mean value and σ represents the standard deviation. For the reconstructions, the basis for evaluation should be the reference CE values calculated using the prior, and negative values may still reflect improvement upon the prior. Therefore, when discussing the results, we include both the CE and a ΔCE, which is the difference between CE in the posterior and its value for the prior.

Temporally averaged RMSE and spatial distributions of the RMSE are also examined. The RMSE is defined by

$${RMSE}=\sqrt{\frac{1}{\tau }\mathop{\sum }\limits_{t=1}^{\tau }\,{(\bar{{x}_{t}^{a}}{-}{v}_{t})}^{2}}$$
(12)

Here, τ is the number of years.

After the PDA is complete, the linear PSM Eqs. (8) and (9) are used on the reconstructed temperature field to predict the values of the out-of-sample proxy records. Then, the predicted values can be verified against the raw proxy records themselves using the correlation coefficients, CE, and RMSE values.