Main

As accumulated CO2 emissions increase and the climate continues to warm, we will probably face more ecosystem impacts worldwide. Higher global temperatures and changing rainfall patterns will probably lead to changes in fire regimes and their impacts on ecosystems1,2,3,4,5. At 1.26 °C (ref. 6) of warming above pre-industrial (PI) levels, we already see changes in the intensity and frequency of extreme weather events, many of which have become more likely due to climate change7 and some almost impossible without current levels of warming8.

Fire regime and biome shifts are also occurring across the world’s ecosystems due to climate change9,10,11,12,13,14, and models project increased transformation as warming increases4,15. Transition can result from climate, land use and fire interactions, such as shifting tropical forest ecosystems to seasonal forests or savannahs9,16,17. Fire substantially impacts ecosystems and carbon stores through vegetation mortality, hydrological cycle changes and emissions of greenhouse gases, aerosols and aerosol precursors. Fires may also determine alternate stable states of ecosystems within similar climates18,19,20,21,22,23, and spatial and temporal changes in fire regimes may alter the outcome when considering the future resilience of ecosystems24.

The further we limit emissions and global temperature rise, the more we can limit the worst impacts of climate change. Studies using ‘fire weather’ show an increased fire risk for temperatures at and beyond 1.5 °C (refs. 25,26). However, no studies have yet explored the impact on land carbon sinks of including changes in fire feedbacks to understand if 1.5 °C is consistent with avoiding significant ecosystem changes.

Large uncertainties remain around the terrestrial biosphere’s ability to continue to uptake carbon1,2,27, particularly given nutrient limitations and fire changes. Our understanding of carbon budgets necessary for staying below 1.5 °C is therefore incomplete. Though many carbon-rich ecosystems will very likely experience considerable shifts in fire regimes over the coming decades4,22, few studies specifically address differences in ecosystem response to fire. Most so far have drawn on Coupled Model Intercomparison Project (CMIP) models, where fire has been assessed as lacking or poorly modelled5,28 and needing further analysis24. Biases in underlying climate models and resultant vegetation distributions also hamper many projections29.

This study assesses the impacts of fire regime changes on land carbon sinks at different global warming levels. We show that additional fire–vegetation feedbacks may reduce the capacity of the global sink to store carbon, as fire regimes change in the future with climate. We account for missing processes, including nitrogen limitation, dynamic vegetation and fire, using the fire-enabled land-surface model JULES-INFERNO and validate the output using benchmarking metrics30 to assess performance in changing fire, tree cover and carbon uptake (Methods). We address regional and climate model uncertainty using four different Earth System Models (ESMs) that simulate different climate and land-use outcomes according to Representative Concentration Pathways (RCP), using a subset of the CMIP5 multi-model ensemble per the ISIMIP (Inter-sectoral Impact Model Intercomparison Project31) 2b framework. ISIMIP allows us to reduce uncertainty in climate-system response to emissions by using bias-corrected General Circulation Model (GCM) data, which correct trends while maintaining variability in the models29. Hereafter, we refer to the JULES simulations driven by the bias-corrected ISIMIP data by the ESM name.

Here we ask how considering fire changes affects vegetation transitions, tree cover and net biomass productivity (NBP). We carried out two sets of model runs—one ‘with fire’ and one ‘without fire’. In the simulations ‘with fire’, we reduce the background mortality and simulate spatially and temporally varying burnt area dependent on fuel load, flammability, land cover and ignitions27. After validating the model, we split the results into three sections: projected burnt area changes, changing trends in tree cover with fire and changes in NBP. For fire, tree cover and NBP, we ask if impacts could happen earlier or later when considering changes in burning and the implications for remaining carbon budgets.

Evaluation

We evaluate JULES against a range of observations to characterize model performance and validate it for use in this study (Supplementary Tables 2–12). The fire model driven with bias-corrected climate from four ESMs reproduces the observed spatial pattern in present day burnt area across regions (Supplementary Figs. 1 and. 2 and Supplementary Table 3) with a trend significantly within observational constraints (Supplementary Table 4). The model also reproduces spatial patterns and trends in vegetation carbon and tree cover (Supplementary Material ‘Model Evaluation’) compared to observations—important metrics in this analysis, verifying the use of the model for assessing future changes in carbon and tree cover.

Global changes in future fire

In most Global Fire Emissions Database (GFED) regions, burnt area is projected to increase at 1.5 °C compared to the present day and increase further at 2.0 °C (Fig. 1). Some of the largest increases are in Europe (Fig. 1b) (15% and 25% increase for 1.5 °C and 2.0 °C, respectively; Fig. 1) and in boreal North America (Fig. 1a) (12% and 20% increase, respectively). The main exception is Africa (Fig. 1i,j), where the model projects that burnt area will decrease in many fire-dominated areas, following the recent decline observed over the continent32. Generally, there is good agreement in the direction of change in burnt area across the driving ESMs. However, simulations driven with MIROC for the Middle East (Supplementary Fig. 3g) and HadGEM2-ES for equatorial Asia (Supplementary Fig. 3m) tend to show an increase in burnt area. In contrast, simulations driven by the other ESMs drive a flat or slightly decreasing trend. The increases in South America, North America and southern Europe are mainly dominated by hotter and drier conditions (Supplementary Fig. 4). In contrast, an increase in population causes increased fire suppression and drives decreases in Africa (Supplementary Fig. 5).

Fig. 1: Percentage burnt area change from present day at 1.5 °C and 2.0 °C by GFED region.
figure 1

Percentage burnt area change from present day (PD, 2010–2019 average) at 1.5 °C (blue) and 2.0 °C (yellow) by GFED region42. The range across four driving ESMs is shown in range bars, the distribution is shown in the coloured violins and the mean is shown in a central horizontal line. The central map shows the multi-driving model mean percentage change in burnt at 2.0 °C above PI compared to the present day across GFED regions. a–n, GFED regions are as follows: BONA (boreal North America; a), EURO (Europe; b), MIDE (Middle East; c), BOAS (boreal Asia; d), CEAS (Central Asia; e), SEAS (Southeast Asia; f), EQAS (equatorial Asia; g), AUST (Australia; h), SHAF (Southern Hemisphere Africa; i), NHAF (Northern Hemisphere Africa; j), SHSA (Southern Hemisphere South America; k), NHSA (Northern Hemisphere South America; l), CEAM (Central America; m), TENA (temperate North America; n).

Full size image

Global change in vegetation cover

At a GWL of 1.5 °C, climate and CO2 fertilization generally lead to higher tree cover in northern high-latitude regions and Congo, but a decrease in tropical South America and Asia (Fig. 2). Tropical regions that show decreasing tree cover at 1.5 °C also show a shift to more heat and drought-tolerant C4 grasses. Higher temperature and CO2 concentration in boreal regions probably increase tree cover as the tree line moves north (Supplementary Fig. 6), driving the changes of the distribution of land carbon sinks. In the Southern Hemisphere, particularly over Africa, projected increases in agriculture (Supplementary Fig. 4) probably contribute to the reduced burnt area and tree cover.

Fig. 2: Change in tree cover at 1.5 °C above PI and temperature of equivalent impact with fire.
figure 2

a–d, Left column: the direction of change in tree cover at 1.5 °C above PI (reducing/recovering/increasing/diminishing where there is agreement between with and without fire simulations or grey where there is disagreement). e–h, Right column: the difference in temperature of the equivalent impact with fire compared to 1.5 °C above PI without fire (or dark blue where equivalent impact is not reached within the simulation). Each row shows the driving climate ESM. Tree cover change is a ratio of 20-year average tree cover at 1.5 °C above PI compared to 1860–1900 tree cover. In the right column, blue colours show where the same level of change happens later (at higher temperatures) with fire compared to without fire, and red shows where the change happens earlier (at lower temperatures), with the intensity of the colour relating to the difference in temperature. Maps created withe maps R package43.

Full size image

At 1.5 °C, total tree cover is projected to increase across western Europe, eastern and western Canada, Southeast China and central and southern Africa (Fig. 2a–d). In contrast, tree cover decreases in southern Brazil, northeastern Africa and western China/Tibet (Fig. 2a–d). There is some spread between climate projections in other regions. GFDL and IPSL simulations (Fig. 2b,d) show an increase in Amazonian tree cover, with a projected rainfall increase mitigating some of the fire-induced tree cover losses (Supplementary Fig. 4). HadGEM2 and IPSL (Fig. 2a,d) generally project tree cover decreases in North America, in line with reduced precipitation (Supplementary Fig. 4) whereas GFDL and MIROC (Fig. 2b,c) show disagreement or recovery. In central Canada, the direction of change is mixed, and ESMs are split between reducing tree cover (HadGEM2, GFDL, IPSL) and a recovery in tree cover (MIROC). In northwest Africa, MIROC tends to show reducing and diminishing tree cover, whereas the other three ESMs show some areas as recovering or increasing. Russia is mixed, with some decreasing and some increasing tree cover. Tree cover is reducing in Southeast Asia and northern Australia but recovering or increasing in some central parts of Australia. In the regions where there is disagreement between the simulations with and without fire, we have less confidence about the future direction of change.

We define the ‘equivalent impact of tree cover change’ as the point at which tree cover is at the same level with fire as without fire at 1.5 °C above PI, showing the same direction of change where possible (Methods) and at the closest point to 1.5 °C above PI (Supplementary Fig. 7). In most regions this equivalent impact occurs at lower global temperatures when fire is simulated explicitly (red colouring, Fig. 2e–h), meaning impacts could happen earlier than previously thought when accounting for fire–vegetation feedbacks. For example, fire increases vegetation mortality and accelerates tree loss, leading to a tree cover reduction earlier in North America (HadGEM2 and IPSL; Fig. 2e,h), southern Brazil, West Africa and Western Australia (Fig. 2e–h). In other areas, tree cover gain happens earlier because there is more fire suppression (for example, in the Congo region). However, there are a few regions where impacts happen later (blue in Fig. 2e–h). In northwestern Russia, central Australia, western India (HadGEM2, MIROC, IPSL), central Amazonia (IPSL) and northern Africa (GFDL, MIROC, IPSL), fire mortality slows tree gain, and equivalent impacts are projected to occur at higher temperatures with fire, taking more time to reach the same state. In some regions, the direction of change is the same, but the equivalent level of impact is not reached within the length of the simulation (dark blue areas). Here fire causes a lower PI tree cover, so the same level of tree loss can no longer be reached or reached beyond 2100.

Weakening of land carbon sink

We use Net Biome Productivity (NBP) to quantify carbon sinks and sources of atmospheric CO2. A positive NBP (net primary production (NPP) minus soil respiration, wood harvest, wood products and fire emissions in the fire simulations) signifies a net carbon sink. A negative NBP, from reductions in carbon storage from reduced NPP, land-use change, fire and/or respiration, indicates a net carbon source. In periods of climate stress from heat or drought or when disturbance from fire or land use is greater than the capacity of the system to absorb carbon, ecosystems can switch from being a net sink of carbon to a net source28,33,34. This can also shift ecosystem composition, impacting carbon uptake in subsequent years.

Globally, we project total NBP to increase with temperature (Fig. 3a) resulting from high-latitude warming and CO2 fertilization, allowing the northward expansion of the tree line, increasing tree cover (Supplementary Fig. 4 and Fig. 2) and NPP in colder regions including BONA, EURO, BOAS and CEAS (Fig. 3). In most regions, the addition of fire decreases NBP, except for MIDE, NHAF and SHAF, where there is a slight increase at lower GWLs due to the declining trend in burnt area (Fig. 1 and Supplementary Fig. 2). The negative impact of fire on NBP appears to be smaller in most regions with MIROC compared to the other driving ESMs. However, all ESMs generally show a negative impact of fire on carbon uptake (Supplementary Fig. 8).

Fig. 3: NBP with and without fire by temperature and cumulative NBP over time.
figure 3

a–o, Climate ESM mean (solid line) and range (plume) NBP (Pg C yr−1) binned by 0.1 °C increments global mean temperature above PI (°C) for each GFED region, with fire (red) and without fire (blue). Dashed horizontal lines indicate the 0 Pg C sink/source threshold, and shaded vertical bands show the temperature range of emergence of significant impact. p–s, Cumulative NBP (Pg C) without fire (top row) and with fire (middle row), where green and brown lines indicate the land has, on average, been a net sink or source of carbon relative to the present day. The bottom row shows the difference between the top and middle row (difference due to fire), where green is an increase in NBP, and brown is a decrease. Black lines indicate the year fire first significantly impacts NBP vs present day (2010–2019) using a significant difference Wilcoxon test, alongside the global mean temperature above PI (°C) in the driving ESM. Here we show four GFED regions (all regions are shown in Supplementary Fig. 8). Each row is a different driving ESM (top to bottom: HadGEM2, GFDL, IPSL, MIROC); these are highlighted in p using the first letter of each model, repeated throughout this figure and Supplementary Fig. 8.

Full size image

We highlight three regions where including fire can shift the system from a net sink to a net carbon source and that evaluate well against observed trends (Fig. 1 and Supplementary Fig. 1). TENA, NHSA and SHSA have large forest areas creating potentially large future carbon sinks and have high fire activity. In some extreme years, these regions shift to negative NBP with fire (Fig. 3c,e,f). All four ESM projections indicate years where these regions switch between a sink and a source of carbon, which happens more frequently with fire (Fig. 3p–s). In most regions, fire negatively impacts NBP by the end of the century (all except MIDE, NHAF, SHAF). In many regions, fire reduces NBP throughout the historical and future period (Supplementary Fig. 8).

Using a significant difference unpaired two-tailed Wilcoxon signed-rank test with fire vs without fire (as per Burton et al.27), the range of global mean temperature above PI at which fire first has a significant impact on global NBP is 0.8–1.34 °C, with a range of 0.58–2.02 °C across the three highlighted regions (Fig. 3p–s). Therefore, regions previously projected to continue as a net sink of carbon into the future may be closer to a threshold than previously understood, and those impacts could be starting now.

Remaining carbon budget

The Intergovernmental Panel on Climate Change 2018 ‘Special Report on Global Warming of 1.5 °C’ (IPCC SR1.5) assessment of remaining carbon budgets presented an explicit uncertainty due to unrepresented Earth system processes, estimating up to 100 Gt CO2 correction due to permafrost thaw this century30. Carbon budgets presented in IPCC’s Sixth Assessment Report (AR6)35 attempted to account for these feedbacks, including corrections due to permafrost CO2 and CH4 feedbacks, but assess very low confidence in the magnitude of fire effects. Here we present explicitly quantified adjustments to estimates of the remaining carbon budget, taking into account our fire simulations across a range of warming levels (Table 1). Whereas there have been more recent carbon budget estimates6, IPCC estimates directly informed the global temperature ambitions we are re-evaluating here. Our simulations estimate that fire may reduce the remaining carbon budget by around 15 Gt CO2 to limit warming to 1.3 °C above PI, and studies since AR6 suggest this GWL may already have been reached6,36. As GWL increases, the reduction in the carbon budget is greater, and all four driving ESMs agree on a reduction in the remaining carbon budget due to fire feedbacks. To limit warming to 1.5 °C above PI, the mean projected reduction in the remaining carbon budget is 25 Gt CO2, and to limit warming to 2.0 °C, the mean reduction is 64 Gt CO2 (4–6% reduction in both cases). The remaining carbon budgets presented here35 are from the beginning of 2020, and there has been an additional ~120 Gt CO2 emitted since these figures were published (~40 Gt CO2 yr−1 (ref. 37), reducing the budget further.

Table 1 Remaining carbon budget at different GWLs from IPCC AR6, with new columns showing the reduction in the remaining carbon budget resulting from fire
Full size table

Implications for global temperature ambitions

In extreme years, fire feedbacks are causing South America and temperate North America to shift from carbon sinks to carbon sources. In South America, we project increased burnt area, and an ecosystem shift to more heat and drought-tolerant C4 grasses across southern Amazonia. In temperate North America, burnt area and C3 grasses increase. In these areas, a reduction in tree cover could happen earlier, with over 1.0 °C less warming when we account for fire changes. We found that fire significantly impacts global NBP at global temperatures of 0.8–1.34 °C above PI (central estimate 1.07 °C). For some regions, the global warming level identified for significant impact has already been reached, meaning climate change has weakened global land sinks through fire.

Regions projected to continue as a net sink of carbon into the future may be closer to becoming a source than previously understood. In TENA, NHSA and SHSA, NBP is lower with fire, and in some extreme years, these systems could shift between a net sink and a net source of carbon. We found that fire impacts these regions significantly at 0.58–2.02°C.

For the first time, we quantify fire’s impact on the remaining global carbon budget, finding a reduction of 25 Gt CO2 for 1.5 °C above PI and 64 Gt CO2 for 2.0 °C. Our results show that the risk of reaching global tipping points may change when we consider hitherto missing processes. We show that fire not only reduces the global land carbon sink but may also cause biome shifts, which due to inertia in the system38, could take decades or longer to reverse. Fire is both a direct climate change impact and a feedback process, meaning we must respond with both adaptation and mitigation planning. We have shown that 1.5 °C is not a line between safe and unsafe, and fire impacts on carbon stores continue to increase with every increment of warming. Our results suggest that fire could play a major role in decreasing the effectiveness of land carbon sinks as warming continues, making reducing warming following an overshoot of 1.5 °C more difficult. These results also have implications for the level of committed warming following the achievement of net zero emissions.

We frame the results using global mean warming levels to inform our understanding of global temperature targets. However, in many cases, regional temperatures and temperatures reached in extreme years are much greater than the global mean and can have larger regional impacts such as local heatwaves, impacts on biodiversity and air quality, permafrost thaw and so on. Some regions in the northern latitudes, for example, will reach 1.5 °C of warming above PI much earlier than the global mean because of Arctic amplification, and in Amazonia, the 1979–2018 mean warming trend was 1.02 ± 0.12 °C compared to 0.98 °C for the global mean39.

We have demonstrated that the model captures the essential metrics to perform fire impact analysis on tree cover and terrestrial carbon storage, and using four bias-corrected ESMs helps to reduce uncertainty and balance out climate sensitivity across the models. As the fire sector has only recently been added to ISIMIP, we have only one fire and land-surface model for this study; results may differ across other fire-enabled dynamic global vegetation models (DGVMs), and when the third phase of ISIMIP data becomes available, we recommend that this analysis is repeated with more fire models. Further work could also look at the impact of projected changes in lightning on fire. Assumptions in INFERNO around land use and fire are based on global trends14,40,41 and the underlying RCP scenarios for how agriculture may change over the twenty-first century, which could also vary across other models and based on the decisions society makes around land use in the coming decades. We have shown that the driving ESMs produce different outcomes of burnt area for different GWLs despite being bias corrected; this is because the bias adjustment only corrects for the mean offset over the observational period, not the trend, and this therefore enables us to sample uncertainty in future projections across different ESMs.

In this paper, we have furthered the understanding of how fire could impact ecosystems in the future considering nutrient limitation, dynamic vegetation and climate change, in the context of different global warming levels. We have shown that tree cover impacts could happen sooner when we account for missing fire processes and that these additional feedbacks weaken land carbon sinks before 1.5 °C. This has implications for our remaining carbon budget, which could also be reduced. Our results suggest that we can still limit the worst impacts by reducing our emissions, but some regions may be closer to a threshold than we previously thought.

Methods

Model set-up

We use the land-surface model JULES (Joint UK Land Environment Simulator)44,45 at version 5.5, which includes nitrogen limitation46, dynamic vegetation from TRIFFID (Top-down Representation of Interactive Foliage and Flora Including Dynamics47,48) coupled to land-use change and fire from INFERNO (Interactive Fire and Emission algoRithm for Natural enviOnments)27,49. JULES forms the land-surface component of the UK Earth System Model (UKESM), and we use the JULES-ES impacts configuration29, which aligns with the representation of the land surface in UKESM150. This includes nine natural Plant Functional Types (PFTs) and four crop and pasture PFTs. The agricultural area is determined by HYDE3.2 land-use data51 for the historical period, and LUH2 harmonized future projections from the MAgPIE land-use model according to RCP 2.6 and 6.052. Within each grid box the fractional coverage of each PFT is determined by a competition hierarchy defined by TRIFFID27,50, including C3 and C4 crop/pasture in the agricultural regions34. The fire model INFERNO is coupled here to dynamic vegetation27, including fire mortality which varies by PFT and reduces burning in crop-land areas41. Ignitions are represented by population density and lightning, where population is from HYDE3.251 for 1861–2005, and from national Shared Socioeconomic Pathway 2 (SSP2) population projections53 for 2006–2099, and lightning is a monthly climatology from LIS/OTD54 covering 1995–2014. The lightning data were scaled by 0.25 for cloud-to-ground strikes, and all ancillary data were regridded to 0.5° resolution.

The ISIMIP2b protocol uses modelled historical climate for the period 1860–2006, which is bias corrected to EWEMBI observations55. For the period 2006–2100, we use the four bias-corrected ESMs to drive JULES: HadGEM2-ES (HadGEM2), GFDL-ESM2M (GFDL), IPSL-CM5A-LR (IPSL) and MIROC5 (MIROC)31.

We used the two future RCP scenarios, combined with SSP2 land use and population projection, from ref. 31. RCP6.0 represents a no-mitigation, higher-emissions scenario, whereas RCP 2.6 represents strong mitigation action. However, for the analysis in this paper, we use RCP6.0 so we can include all of the ESMs, as some do not reach 2.0 °C in RCP 2.6 (Supplementary Table 14). JULES is run on a 0.5° grid and spun up for 10,000 years, then another 500 years with fire switched on.

Where relevant, we frame the analysis in terms of Global Warming Levels (GWLs) using the method outlined by ref. 56. We use the global mean PI (1860–1900) near-surface air temperature from the four driving ESMs. Using the central year in a rolling 21-year mean global temperature, we find the first year of exceedance in each GWT. We then use the exceedance year as the central year in 21 years for the analysis in JULES.

We describe the direction of 20-year running mean of tree cover change in Fig. 2 as follows (Supplementary Fig. 7 for a schematic representation of changes):

  • Increasing if there is a positive change in tree cover between that time step and the previous monthly time step and the ratio of tree cover to PI tree cover is greater than 1.

  • Decreasing if there is a negative change in tree cover and the ratio to PI is less than 1.

  • Recovering if there is a positive change and a ratio less than 1.

  • Diminishing if there is a negative change and a ratio greater than 1.

We assess the impact of fire at GWLs by comparing tree cover change in the runs with fire with equivalent tree cover change in runs without fire at 1.5 °C global temperature above PI. Identifying equivalent tree cover change used four criteria:

  1. 1.

    The ratio of tree cover to PI tree cover is the same at 1.5 °C in the simulation without fire. To identify the same points, we fit a linear line between monthly outputs in the without fire runs.

  2. 2.

    The direction of change is the same. 1 and 2 combined means we are also identifying equivalent states between runs.

  3. 3.

    If 1 and 2 identify more than one point in the simulations with fire, we select the closest in time to 1.5 °C.

  4. 4.

    If 1 to 3 did not identify any equivalent point, but the state at the end of the run without fire is the same as at 1.5 °C with fire, we assume that the equivalent change is greater than the largest warming level of that model run.

We tested the significant change in NBP from the present day (2010–2019) using an unpaired Wilcoxon test in the R programming stats package57 on a 20-year rolling window after the present day. The year of significant change for a given model was the central year of the 20-year window with a p value < 0.05.

Model evaluation

We performed JULES model benchmarking using a modified version of the FireMIP benchmarking package58,59 as outlined in detail by ref. 22. We compared the ESM-driven simulations against observations of burnt area, fire emissions, vegetation cover, vegetation carbon and recent changes in tree and woody cover (Supplementary Material ‘Model Evaluation’). Vegetation cover includes woody, grass and bare soil cover for VCF (Vegetation Continuous Fields from MODIS; Supplementary Table 1) and tree, shrub, grass and bare soil cover for ESA CCI (European Space Agency Climate Change Initiative; Supplementary Table 1). Woody cover means trees and shrubs compared to all other cover. Trees mean broadleaf and needleleaf compared to all other cover. Where possible, we used multiple observational datasets to sample observational uncertainty in our model evaluation.

The normalized mean error metric (NME) is appropriate for comparisons of non-normal variables such as burnt area and vegetation cover and is therefore the standard metric to assess global fire model performance59,60. We use NME here to assess the model’s ability to simulate spatial burnt area, vegetation carbon and change in carbon and fire emissions. NME represents the area-weighted absolute mean difference between burnt area maps of observations and the model, normalized by the mean variation in the observations. The NME calculation has three steps44,45 with step two removing mean bias, and step three58 removing mean bias and absolute variance before comparison:

$${\rm{NME}}\; {\rm{step}}\,1:{\mathrm{NME}}(\{{\mathrm{obs}}_{i}\},\{{\mathrm{mod}}_{i}\})=\frac{\mathop{\sum }\nolimits_{i}^{\mathrm{cells}}{A}_{i}\times {{|}}{\mathrm{obs}}_{i}-{\mathrm{mod}}_{i}{{|}}}{\mathop{\sum }\nolimits_{i}^{\mathrm{cells}}{A}_{i}\times {{|}}{\mathrm{obs}}_{i}-\overline{\mathrm{obs}}{{|}}}$$
(1)

Where Ai is the area of grid cell or region i, obsi and modi is the observed and modelled values for i and \(\overline{\mathrm{obs}}\) is area-weighted mean of the observations, that is, \(\overline{\mathrm{obs}}=\frac{\mathop{\sum }\nolimits_{i}^{\mathrm{cells}}{A}_{i}\times\mathrm{ob}\mathrm{s}_{i}}{\mathop{\sum }\nolimits_{i}^{\mathrm{cells}}{A}_{i}}\,\)

$${\rm{NME}}\; {\rm{Step}}\,2:{\mathrm{NME}}_{2}(\{{\mathrm{obs}}_{i}\},\{{\mathrm{mod}}_{i}\})=\mathrm{NME}(\{{\mathrm{obs}}_{i}-\overline{\mathrm{obs}}\},\{{\mathrm{mod}}_{i}-\overline{\mathrm{mod}}\})$$
$${\rm{NME}}\; {\rm{Step}}\,3:{\mathrm{NME}}_{3}(\{{\mathrm{obs}}_{i}\},\{{\mathrm{mod}}_{i}\})={\mathrm{NME}}_{2}\left(\left\{\frac{{\mathrm{obs}}_{i}}{V\left(\left\{{\mathrm{obs}}_{i}\right\}\right)}\right\},\left\{\frac{{\mathrm{mod}}_{i}}{V\left(\left\{{\mathrm{mod}}_{i}\right\}\right)}\right\}\right)$$

Where \(V\left\{{x}_{i}\right\}=\frac{\mathop{\sum }\nolimits_{i}^{\mathrm{cells}}{A}_{i}\times {\rm{|}}{\mathrm{obs}}_{i}-\overline{\mathrm{obs}}{\rm{|}}}{\mathop{\sum }\nolimits_{i}^{\mathrm{cells}}{A}_{i}}\) is the absolute variance58.

To assess the model’s ability to represent the change in carbon, we also use NME to compare the change in vegetation carbon between 2010 and 2018 against CCI aboveground observations. We assess vegetation distribution using the Manhattan Metric (MM)58,60, which calculates the mean bias across ‘items’ that in each grid cell sum to unity and which is therefore more appropriate for evaluation across more than one item:

$${\rm{MM}}=\mathop{\sum}\limits_{{ij}}{A}_{i}\times {\Big{|}}{q}_{{ij}}-{p}_{{ij}}{\Big{|}}{\Big/}\mathop{\sum }\limits_{i}^{\mathrm{cells}}{A}_{i}$$
(2)

Where q is the modelled and p is observed proportion of item j in cell or region i. When comparing a single land cover type (that is, tree cover), that type is considered as one item, and other vegetation cover types (shrubs, grasses, agriculture, bare soil) are combined into a second item. If there are non-vegetative/non-bare soil types in the grid cell (water bodies, urban areas and so on), these areas are removed from the comparison and the grid cell area weighting is adjusted accordingly.

The lower the score for NME and MM, the closer the match between observation and simulation. The benchmark system uses three null models as per Burton et al.27: the median and mean null models compare observations to the median or mean of the observations, whereas the randomly resampled null model randomly compares a dataset generated by sampling the observations without replacement with the observations. As the resultant randomly resampled score depends on sampling order, we repeat this procedure 1,000 times to generate a ‘randomly resampled’ distribution of null model scores. A model is better than the randomly resampled null model if its score is less than the mean plus one standard deviation of this distribution58. In addition to benchmarking, we include an estimate of aboveground carbon61 at two snapshots, 2010 and 2018. Here the same mean null model is equivalent to comparing the observed change in biomass vs no change in biomass, enabling a qualitative assessment of the model’s ability to determine the direction of change in vegetation carbon. As the NME mean null model always scores 1, if a simulation scores less than 1, this demonstrates that the simulation is reproducing the direction of change in aboveground carbon correctly.

We also assess the models’ ability to reproduce trends in the context of uncertainty in observational constraints—particularly important for burnt area, fire emissions and NBP, which, regionally, have high uncertainty in trends due to inter-annual noise and disagreement between observational products60,62. To do this, we calculated a probability density of the trend from observations and the model using a simple log-transformed linear regression and test for the overlap.

When observations have a large uncertainty range on the trend, as is the case for burnt area, how much each simulation’s probability density falls inside the combined observations tells us how well it captures trends. We calculate the probability of the model trend being within observational uncertainty (Pmod) using a modified version of the distribution overlap metric from ref. 4:

$${P}_{\mathrm{mod}}=\frac{{\int }_{-\infty }^{\infty }Q\left({\mathrm{BA}}_{\mathrm{mod}}\right)\times \min \left(Q\left({\mathrm{BA}}_{\mathrm{mod}}\right),Q\,({\mathrm{BA}}_{\mathrm{obs}})d\delta\right.}{{\int }_{-\infty }^{\infty }Q\,{\left({\mathrm{BA}}_{\mathrm{mod}}\right)}^{2}d\delta }$$

Where \(Q\,(\;y)=P\left(\delta |\log (\;y)\right)\) is the probability of trend \(\delta\) given the log of y where y takes either annual burnt area from observation (\({\mathrm{BA}}_{\mathrm{obs}}\)) or simulated by the model (\({\mathrm{BA}}_{\mathrm{mod}}\)).

If the model’s probability density falls fully inside the observations, it scores 100%. If it’s completely outside, it scores 0%. Fifty percent indicates that considering observational uncertainty, the model has a 50% likelihood capturing the correct trends. Note that this metric is most relevant where trend is uncertain. Where trends are more certain, such as burnt area in NHAF, the model may get a low score but qualitatively capture the correct direction of trend.