Adaptive food price forecasting improves public information in times of rapid economic change

Abstract

The advent of COVID-19 ended an era of stable US retail food prices that followed the world food price crisis of 2010–2012. Pandemic-related disruptions, avian influenza outbreaks, and the Russia-Ukraine war drove 2022 food-at-home inflation to its highest rate since 1974 (11.4%). In 2023, U.S. Department of Agriculture (USDA) economists responded to these changes by updating food price forecasts using statistical learning protocols to select time series models and prediction intervals to convey their uncertainty. We characterise the public good provided by these “adaptive” inflation forecasts and enhance them by incorporating exogenous variables to improve their precision and explanatory power. COVID-19’s arrival highlighted the value of adapting to the growing relevance of the all-items-less-food-and-energy ("core”) index, the money supply, and wages in predicting food prices. The strong relationships between food prices and core prices and the money supply indicate the sensitivity of food markets to macroeconomic forces and government policy decisions.

Introduction

Following relative stability between 2013 and 2019, US retail food markets experienced rapid changes in 2020 through 2022^1,2. COVID-19 immediately impacted food prices as processing halted and grocery stores sold products typically consumed in restaurants³. Beyond the direct impacts of COVID-19 outbreaks, processing, and transportation disruptions, domestic (e.g. stimulus packages) and foreign government policy choices (e.g. military invasions and commodity export bans), and natural disasters increased costs for firms along the supply chain and altered the demand for food. For example, highly pathogenic avian influenza and the Russian-Ukraine war rapidly drove up costs for eggs and wheat, respectively^4,5. Natural and biological disasters, such as droughts in California and the emergence of African swine fever among China’s pork producers, altered the supply of food commodities and the international demand for agricultural inputs^6,7. These factors contributed to increases in wholesale prices for finished consumer foods (14.2%), retail food-at-home (grocery and supermarket purchases) prices (11.4%), and food-away-from-home (restaurant purchases) prices (7.7%) in 2022⁸. While this paper focuses on US retail markets, food prices worldwide experienced significant inflation and heightened volatility⁹.

When retail food market conditions dramatically shift, the simple necessity of food raises concerns among households, retailers, policymakers, and philanthropic groups. Stakeholders express the greatest concern for low-income households, who spend much of their budget on food. The lowest quintile of US households by income spends about 33% of their income on food, compared to just 8% for the highest quintile¹⁰. The contribution of food price inflation and volatility to household budget pressure, food insecurity, a growing disparity in income inequality, and even civil unrest—particularly among lower-income populations—dominates headlines in the popular press and ranks among the most pressing public issues^11,12. Policies such as food assistance programs can alleviate hardship from food price inflation. However, the efficient implementation of these programs hinges on expectations of food price inflation, as accurate predictions support timelier policy modifications and fewer budgetary adjustments during program development. Understanding the dynamic relationship between food prices and their drivers aids public management beyond food policy and business decisions¹³. Yet, existing research on food price forecasts remains largely limited to government research reports, despite intense public interest^2,14,15,16.

Since its establishment in 1961, the United States Department of Agriculture’s (USDA) Economic Research Service (ERS) has published forecasts of US retail food prices^2,15,16,17; as of 2024, the monthly Food Price Outlook (FPO) report published these forecasts¹⁸. In addition to describing recent food market and price trends, these reports provide point forecasts of food prices and probabilistic prediction intervals around them that convey uncertainty. FPO forecasts represent a public good, informing a range of policymakers and market participants. At various levels of government, policymakers use these forecasts to anticipate household food costs and budgets for food assistance programs. For example, USDA Food and Nutrition Service analysts use FPO forecasts in budgetary planning for the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC). WIC provided over $7.5 billion in federally funded food assistance in 2023, where food costs represent the bulk of the program’s costs¹⁹. Forecasts help officials anticipate budgetary issues, including a potential 2024 budget shortfall in the WIC program from increased program participation and higher food costs. The Center on Budget and Policy Priorities and USDA directly appealed to Congress for additional funding once they anticipated the shortfall, based on improved food price forecasting^20,21.

FPO forecasts similarly support cost estimates of commodity purchases for USDA food distribution programs, including the Commodity Supplemental Food Program and Food Distribution Program on Indian Reservations. USDA Child Nutrition Program budget estimates also use FPO forecasts for US National School Lunch Program reimbursements to US states²². State school lunch programs are growing in size and scope: in 2021, California and Maine became the first states to provide universal school meals, regardless of the student’s eligibility under previously established requirements²³. Other states, including Colorado, Minnesota, and Massachusetts, initiated similar programs in 2023²⁴.

Likewise, FPO forecasts inform participants in the food industry, consumers, and the media about the likely path of national prices. More accurate expectations of upcoming food prices improve the effectiveness of related decisions. For example, FMI, the Food Industry Association, a trade group of food-industry firms across the supply chain, regularly cites USDA forecasts in their publications and member outreach²⁵. Retail firms, in particular, use FPO forecasts to establish price expectations, informing decisions regarding inventory management, pricing strategies, and promotional efforts. Altogether, FPO ranked among ERS’s most widely consumed products in 2021–2024, signalling the intensity of public interest.

As with weather and electricity price forecasts^26,27, the value of FPO forecasts to their stakeholders rests on their accuracy. They inform expectations of food prices, which may follow patterns distinct from the more widely studied aggregate national prices^13,28. Over time, USDA economists refined their forecasting procedures^2,14,15,16. Through 2022, forecasting models informed analysts’ expectations, but expert opinion ultimately guided ‘legacy’ projections. In 2023, ERS economists began publishing forecasts based on an algorithm that selects a forecasting model each month based on past values of the series, allowing for continuous updating^2,29. This enhancement reduced forecast error, removed bias associated with subjectivity, and facilitated future refinements. Simultaneously, ERS replaced traditional ad-hoc 1-percentage-point ‘forecast ranges’ with more commonly used prediction intervals that convey forecast uncertainty, based on historical residual distributions and Monte Carlo simulations^2,30,31.

Model updating, a form of artificial intelligence and machine learning referred to as ‘statistical learning,’ fits a time series and then adjusts as the series changes over time. This adaptability serves food price forecasters particularly well, as food systems and markets frequently face structural changes through persistent supply and demand shocks^2,32,33,34. Moreover, adaptation is grounded in an extensive literature that enables rapid forecast adjustments to system shifts based on expert opinion, data transformations, or model design—including the optimal weighting of multiple forecasts^35,36,37. Our use of ‘adaptive forecasting’ closely aligns with a recent paper on predicting copper price volatility from the computer science literature; it proposes an algorithm to choose the best among a set of candidate time series models over a sliding window of time³⁸. However, our results indicate that adaptation alone does not statistically significantly reduce forecast errors for food-at-home price series. Instead, combining adaptation with leading indicators of food-at-home prices yields statistically and economically significant reductions in forecast error.

The 2023 updates represented a consequential methodological advancement for the typical approach of infrequently updating complex models. However, the FPO forecasts remain univariate, using only past values of the price series. In this paper, we extend the recent FPO procedure by introducing exogenous predictors of food prices^2,39,40, focusing on food-at-home prices in the United States. Economists term ‘exogenous’ those variables outside the system—able to influence, but not caused by, the series itself. Three research questions guide our work: (1) Does including exogenous information reduce forecast error? (2) Does adaptation reduce forecast error? (3) How can exogenous information and adaptive techniques enhance our understanding of forecast drivers and improve the interpretability of the forecasts? We jointly address questions (1) and (2) by testing whether the forecast error is statistically significantly reduced. The myriad ways results could be interpreted lead us to address question (3) with case studies.

Two adaptive approaches to including exogenous variables provide statistically significant reductions in forecast error compared to either using adaptive forecasts or including exogenous variables: an iterative, optimal-selection procedure and a simpler and computationally cheaper approach of including all considered exogenous variables while allowing the model of own-prices to update. The first algorithm selects an optimal model from many models (up to  ~150, 000 for each series each month) from seasonal-autoregressive-integrated-moving-average-with-exogenous-variables (SARIMAX) models that permit the inclusion of past observations of food prices, lagged exogenous variables, and seasonality. The latter approach includes all exogenous variables and selects the best accompanying SARIMA model each month. Practitioners can use the sequential relationship between included exogenous variables and food prices (i.e. their ‘Granger causality’) to provide empirically grounded explanations to policymakers and the public about observed food price changes rather than being limited to simple trend associations or drawing explanations from external sources^41,42.

The evolution of USDA’s food-price forecasting from relying on expert opinion to selecting models based on statistical fit provides an example of how public entities worldwide involved in forecasting for public benefit can research, develop, and implement statistical advancements while achieving forecast continuity. The availability of consistent rules for model selection, the ability to identify predictors, and improved out-of-sample performance support adapting these methods to various applications, including commodity-specific (e.g. crops, energy, and raw materials) and food-specific markets. Related approaches have been applied to labour markets⁴³. Additionally, incorporating statistical learning enables swift adjustment to changing time-series properties. Notably, these recent advances demonstrated by FPO meet the calls of researchers to address technical shortcomings of publicly-provided forecasts made by the USDA and twelve other US federal statistical agencies^31,44,45,46, as well as forecasting agencies around the world⁴⁷.

Each month, FPO predicts the annual percent change in food prices (inflation rate) by comparing average monthly prices (across forecasted or observed values, as available) in the upcoming calendar year to the previous year’s average prices. As the year progresses, FPO’s forecasts incorporate more data and fewer forecasts, adjusting expectations and reducing uncertainty. We propose incorporating exogenous information into this process. Figure 1 illustrates this procedure. Based on historical prices up to the month before forecast publication (represented by ${{{{\bf{Y}}}}}_{t-1}=\left[{y}_{1},...,{y}_{t-1}\right]$) and exogenous information through the previous month (${{{{\bf{X}}}}}_{t-2}=\left[{{{{\bf{x}}}}}_{0},...,{{{{\bf{x}}}}}_{t-2}\right]$), we develop an initial expectation for the next year’s annual food-at-home inflation rate eleven months in advance $\left(\left.{m}_{t}={{{\rm{E}}}}\left[100\times \left(\frac{\mathop{\sum }_{u=t}^{t+11}{y}_{u}-\mathop{\sum }_{u=t-12}^{t-1}{y}_{u}}{\mathop{\sum }_{u=t-12}^{t-1}{y}_{u}}\right)\right\vert {{{{\bf{Y}}}}}_{t-1},{{{{\bf{X}}}}}_{t-2}\right]\right)$ before any monthly price is observed. The distribution’s spread represents the uncertainty of the forecast. New data adjust the midpoint of our predicted price change $\left(\left.{m}_{t}\to {m}_{t+1}={{{\rm{E}}}}\left[100\times \left(\frac{{y}_{t}+\mathop{\sum }_{u=t+1}^{t+11}{y}_{u}-\mathop{\sum }_{u=t-12}^{t-1}{y}_{u}}{\mathop{\sum }_{u=t-12}^{t-1}{y}_{u}}\right)\right\vert {{{{\bf{Y}}}}}_{t},{{{{\bf{X}}}}}_{t-1}\right]\right)$ and narrow the distribution around the forecasts each month. Figure 1 depicts increased forecasted inflation, but a decrease is equally likely. A 95% prediction interval typically represents this uncertainty. The ‘Methods’ section and Supplementary Information comprehensively describe the estimation procedures.

Fig. 1: Adaptive food price forecasting.

(left) Historical prices and other data inform initial food price forecasts; (right) updates use new information.

The remainder of this paper addresses the optimal use of available information in adaptive food-price forecasts and its ability to support explanations from real-time forecasts. Data developers release many of our exogenous (external) series after forecasts are published, and using lagged rather than contemporaneous exogenous variables can meet stricter standards for causal interpretation⁴⁸. In the context of Fig. 1, we use x_t−1 to improve estimates m_t+1 and reduce uncertainty around it. For robustness, we compare in- and out-of-sample performance statistics, evaluate approaches for particularly difficult-to-predict years (2002, 2009, 2016, and 2023), simulate the impact of the proposed policy based on estimated results, and apply the same methods to food-away-from-home prices (see the ‘Adjustments for alternative markets: example of food-away-from-home prices’ section of the Supplementary Information). We discuss the implications of improved forecasts for US decision-makers and how similar methods could be used elsewhere.

Results and discussion

Including exogenous variables

This paper evaluates the FPO’s current univariate, seasonal-autoregressive-integrated-moving-average (SARIMA) modelling and forecasting approach and extends it to include exogenous variables (SARIMAX) between 2003 and 2024. The Bureau of Labor Statistics (BLS) provides food-at-home and food-away-from-home Consumer Price Indices (CPIs) to track food price inflation. Exogenous variables include ‘core’ CPI (excludes food and energy prices), transportation congestion, wholesale prices for finished consumer foods, and the principal components of US energy prices (energy principal component analysis index, PCAI), wages for relevant workers (wages PCAI), available funds for purchases (income PCAI), and the US money supply (money supply PCAI)^{8,10,49,50,51,52,53,54}. We select these variables for their historical importance in determining food prices, based on pressure from the supply- or demand-side of the market^55,56. Their ability to improve predictions remains an empirical question addressed in this paper. More details about each PCAI can be found in the ‘Methods’ section and Supplementary Information. We find that including these exogenous variables increases in-sample model performance and can reduce Kullback-Leibler information loss as measured by the Bayesian Information Criterion (BIC), compared to a univariate approach⁵⁷. Similarly, out-of-sample errors, as measured by the squared error and absolute error, are statistically significantly reduced, which could yield even greater improvements for specific food-price series with higher variance, such as eggs or dairy products.

We focus on the forecasting results of the food-at-home inflation rate. Similar results for the food-away-from-home CPI can be found in the Adjustments for alternative markets: example of food-away-from-home prices section of the Supplementary Information. Among the many SARIMAX models that outperform the benchmark SARIMA model, we highlight two sets of results. First, we report the ‘optimal’ specifications that yield the minimum BIC value, which differ depending on how well they explain the recent past. It can include no, some, or all of the exogenous variables. Second, we report the results from the SARIMAX model with the minimum BIC that includes all exogenous variables as the ‘kitchen-sink’ SARIMAX.

Table 1 compares the performance of the root mean squared error (RMSE), mean absolute error (MAE), and other statistics for the benchmark SARIMA, the optimal SARIMAX, and the kitchen-sink SARIMAX, aggregated over all months and years. We conduct the analysis with adaptation and compare the results with corresponding non-adaptive (i.e. static) models based on the preferred SARIMA specification identified at the time of the forecast, as well as different combinations of exogenous variables¹⁵. Preferred covariates were not identified at the time of analysis. Statistical models still generate these estimates, unlike the legacy ‘forecast ranges’ based on models and analysts’ institutional knowledge¹⁶. Given the selected time-series model, we find that including wholesale prices minimises BIC. We consider an alternative, non-adaptive approach in the Supplementary Information, where the model minimising BIC is selected in the first period, but never updated (see Table and Fig. S1).

Table 1 In- and out-of-sample performance of benchmark SARIMA, optimal SARIMAX and kitchen-sink SARIMAX^a point forecasts with and without adaptation^b, 2003–2024

The in-sample performance estimate (BIC) favours the inclusion of the optimal, non-empty set of exogenous variables in 58% of cases. In the remaining 42%, the optimal model excludes exogenous variables, and the benchmark SARIMA and optimal SARIMAX approaches are equivalent. By construction, neither the benchmark SARIMA nor the kitchen-sink SARIMAX models can reduce BIC relative to the optimal SARIMAX since both are considered in the optimal-model evaluation.

Including exogenous variables also offers improved out-of-sample performance over the baseline SARIMA, as suggested by the lower RMSE and MAE estimates. The RMSE-to-MAE ratio exceeds 1.25 in all cases, indicating that the MAE provides a more reliable measure of expected forecast error⁵⁸. Despite producing higher BIC values, the comparable out-of-sample performance of the kitchen-sink SARIMAX suggests that over-fitting does not significantly undermine forecasting performance in this context. Still, with so many included exogenous variables, inferring sequential relationships should be done cautiously and with nuance as described in the ‘Methods’, Adjustments for alternative markets: example of food-away-from-home prices (Supplementary Information), and Testing for reverse causality (Supplementary Information) sections⁴⁸. Additionally, the optimal SARIMAX generates the narrowest average prediction interval, while the benchmark SARIMA produces the widest.

We primarily focus our discussion on the kitchen-sink approach for clarity, as it is simpler and easier to explain than the optimal SARIMAX approach. The gains in out-of-sample performance from optimisation are not statistically significant and may not hold in other contexts or even for specific periods for FAH prices. Public forecasters may wish to consider either or both approaches when developing public forecasts, depending on their objectives. The comparable performance of the optimal and kitchen-sink SARIMAX approaches does not alter or diminish the core lessons presented in this paper regarding the value of adaptation and the inclusion of exogenous variables. Indeed, most results discussed here would be qualitatively similar if the optimal results were considered, other than evaluations of how changes in exogenous variables affect food prices. Additional discussions about how these assumptions affect the results are presented in the Supplementary Information.

Figure 2 presents the results of formal hypothesis tests of forecast error reduction from adaptive forecasting or including all exogenous variables. The percent change in MAE along each path indicates the magnitude of the expected differences in forecast performance. At the same time, the reported statistical significance indicates the confidence level at which we can reject the null hypothesis of zero change.

Fig. 2: Percent changes and statistical comparison of forecast errors (MAE) between model selection approaches using static or dynamic model selection or including all exogenous variables.

Note: ^† p < 0.1, ^* p < 0.05, ^** p < 0.01, and ^*** p < 0.001.

We can conclusively answer research questions (1) and (2). Neither adaptation nor the addition of exogenous variables alone significantly reduces forecast error at the 0.05 confidence level; in fact, adding exogenous variables by themselves increases MAE. This outcome strongly contradicts the common practice of infrequent model updates based on the most recent published research while including a complex set of exogenous variables^14,15,16. In contrast, combining adaptive forecasting with exogenous variables yields statistically significant, double-digit percentage reductions in forecast errors compared to the other three approaches.

New data typically improve forecast accuracy and reduce uncertainty, with the same out-of-sample metric shown in Table 1 declining from January through December. Figure 3 demonstrates how MAE and uncertainty decrease for the benchmark SARIMA and kitchen-sink SARIMAX as more information becomes available throughout the year. The shaded areas in Fig. 3a and b depict the average reductions in MAE or the width of the prediction interval from including exogenous variables in different months. As Fig. 3a shows, the kitchen-sink SARIMAX MAE falls from 1.60 to 0.02 between those months. The grey shading further indicates that its monthly average MAE remains below the benchmark SARIMA in all months except August. Similarly, the kitchen-sink SARIMAX model often produces narrower prediction intervals, between 0.53 and  <0.01 percentage points smaller, reflecting a decreased uncertainty around forecasts. Yet, Fig. 3 also shows inconsistent improvements in precision and uncertainty throughout the year. Seasonality, noise, a relatively small sample size, and the frequent and large deviations from typical trends observed during the sample period likely contribute to this uneven decline.

Fig. 3: Forecasting error and uncertainty decline when including exogenous variables.

Mean absolute error (MAE) (a) and average prediction interval width (b) by selection algorithm and the month of the forecast, 2003–2024. The difference is reported between selection algorithms based on including all exogenous variables (kitchen-sink) or none (benchmark).

Forecast improvements gained from including exogenous information early in the calendar year provide the greatest impact on federal budgetary planning. For example, the FPO publishes its February forecast before the White House’s annual President’s Budget, which must address federal food assistance programs. The combined food costs for WIC and the Supplemental Nutrition Assistance Program represent the bulk of associated costs at $111.5 billion in 2023⁵⁹. Absent major shifts in program structure or participation, the 0.15 percentage-point decrease in MAE in February implies that forecasts based on aggregate food prices could be $167 million closer to actual costs. Forecasts focused on the foods most commonly obtained via these programs would likely yield even greater gains.

Figure 4 displays coefficient estimates for exogenous variables included in each month’s forecast, following the optimal and kitchen-sink selection procedures. A dot or line in a given month represents inclusion, while blank spaces represent exclusion from the optimal procedure. Fainter lines of the same colour present the kitchen-sink approach’s parameter estimates.

Fig. 4: Evolving coefficient estimates shift rapidly during the COVID-19 pandemic.

Coefficient estimates for exogenous variables in the optimal (normal) and kitchen-sink SARIMAX (faint) forecasting models for food-at-home Consumer Price Index (CPI), 2003–2024. Forecasts include food-at-home CPI data through the previous month and exogenous data from 2 months prior.

Our transformation of the food-at-home CPI series and virtually all the exogenous variables permits most coefficients in Fig. 4 to be interpreted as short-term predictive elasticities, or the expected percentage change in food prices in response to a one-percent increase in the lagged exogenous variable. For example, we expect a 0.1-percent increase in food-at-home prices to follow a one-percent increase in the money-supply PCAI in June 2020, when the coefficient is largest. For context, the M1 money supply—the most volatile component of the money supply index—increased by over 230% in 2020 and over 50% in 2021. Coefficients from the transportation congestion index—typically very close to zero—cannot be interpreted as elasticities since they undergo a different detrending method before reporting.

The optimal and kitchen-sink approaches largely agree regarding the direction and magnitude of coefficients. Throughout the 22 years of monthly forecasts in Fig. 4, coefficients for different lagged exogenous variables remain consistent in the direction and magnitude of their relationship with food-at-home prices in the optimal approach. When they appear in Fig. 4, lagged, historical wholesale prices, the money supply, and transportation have positive predictive relationships with food-at-home prices. In contrast, increases in the lagged core CPI, energy PCAI, and wages typically predict decreased food-at-home prices. Figure 4 also shows that the optimal SARIMAX models did not include a consistent set of exogenous variables over 2003–2024. At times, only one variable is included, such as the wholesale prices in January 2003; at others, multiple variables are selected, as evidenced by the money supply, transportation, core CPI, and wages in December 2024. Notably, in some months, such as January 2010, the optimal model is univariate, containing no exogenous variables and relying solely on the past values of the food-at-home CPI. In such cases, the optimal SARIMA and SARIMAX models coincide.

The kitchen-sink approach largely aligns with the optimal SARIMAX but differs because certain relationships change signs during the study period. While the coefficients for core CPI and wholesale prices consistently maintained their signs, all others exhibit brief sign changes, which may reflect a combination of weaker correlations and changing food system dynamics. The optimal algorithm never selects the income PCAI as the optimal solution.

Figure 4 offers strong evidence that changes in food-at-home CPI significantly correlate to prior changes in core CPI and the money supply. The optimal SARIMAX model frequently includes core CPI, particularly late in the study period, and its decreases typically preceded increases in food-at-home CPI. This dynamic suggests a potential decline in food-at-home CPIs following an increase in core CPI⁵⁵. While core and food prices share a long-term growth pattern, they exhibit short-term divergences. For example, food prices soared around the Great Recession while core prices grew below their long-term average. During the years when core CPI was most often included in the estimation, from 2004–2007 to 2020–2025, the correlation coefficient between food and lagged core CPIs ranged between  −0.2 and  −0.6, indicating a strong negative relationship.

The mostly positive coefficients on the money supply indicate that its increases precede rises in food-at-home CPI; this aligns with the Quantity Theory of Money. All else equal, increases in the money supply require price increases to ration supply and demand⁶⁰. We further illustrate the impact of money supply in the Forecasting the impact of monetary policies on food prices subsection.

The strength of the relationship between core and food-at-home CPIs, the emerging connection between monetary policy and food prices, and the low volatility compared with, e.g., energy prices indicate that historical reasons for separating food prices from core CPI may not reflect current market conditions^18,61. However, the distinct patterns of food-at-home price movements from the core prices continue to support individual—albeit interrelated—examination.

Food-service, rather than grocery-store, wages largely drive the negative relationship between the wage PCAI and food-at-home prices. Overall, the mixed evidence that changes in food-at-home prices precede changes in grocery-store wages and other considered variables (see Supplementary Information) supports recent and ongoing efforts to provide theoretical and empirical decompositions of the drivers of food prices^16,55,62. We present a version of our forecasts in the Supplementary Information that omits transportation, along with a measure of the money supply, to evaluate the assumptions underlying the causal descriptions.

Improving food-at-home CPI forecast performance in 2022

While Table 1 documents the statistical benefits of incorporating exogenous variables in an adaptive forecasting procedure, we illustrate their practical advantages when predicting 2022 food prices. In 2022, the food-at-home CPI inflation rate reached its highest level since 1974, making predictions particularly difficult. Economists attribute this elevated inflation to a combination of general inflationary pressures and rare events affecting specific food products, including the largest bird flu outbreak in US history and the onset of the Russia-Ukraine war^55,63. The rate of price increases also varied greatly in 2022. The rapid price increases observed in early 2022 led to a pronounced tightening of fiscal policy. Figure 5 compares the forecasts generated by the USDA’s ‘legacy’ approach (orange), a non-adaptive SARIMA model selected in 2003 (pink), the USDA’s current adaptive SARIMA approach (grey), and our kitchen-sink adaptive SARIMAX approach (blue). Rather than using a statistical approach as described in the Methods section, the legacy approach used 1-percentage-point ranges to reflect analysts’ institutional-knowledge-based expectations—guided by model results and experience—and account ‘for some degree of inherent uncertainty in forecasting’¹⁶. Each forecast uses information from the previous month to predict the actual inflation rate (black).

Fig. 5: Visualizing forecast improvements from adaptation and including exogenous variables during peak U.S. food price inflation.

Comparisons of 2022 food-at-home inflation forecasts from alternative selection algorithms, generated monthly between January and December 2022. Forecasts include data through the previous month.

Figure 5 illustrates an instance when the SARIMA and kitchen-sink SARIMAX approaches dominate the non-adaptive SARIMA and the legacy forecasting techniques that depended on a combination of vertical price transmission or autoregressive-moving-average (ARMA) models and analysts’ judgement. While all forecasts eventually converge to the actual annual inflation rate as more information becomes available, the speed and accuracy of convergence differ. The kitchen-sink SARIMAX and benchmark SARIMA yield accurate forecasts and ranges by March and April, respectively, while the non-adaptive SARIMA and legacy forecast ranges do not include the actual rate until August and September, respectively. Compared to the benchmark SARIMA, including exogenous variables provides a subtle—but important—improvement. In 2022, SARIMAX forecasts could more rapidly predict record-high inflation and the subsequent moderation as the US Federal Reserve used quantitative tightening and increasing the federal funds rate to decrease the money supply. Periods of rapid price changes accentuate the value of timely and accurate forecasts. The kitchen-sink SARIMAX forecast was 0.72, 1.91, and 4.74 percentage points closer than the benchmark SARIMA, non-adaptive SARIMA, and legacy approaches, respectively. These results underscore the improved responsiveness and adaptability to changing market conditions gained from including exogenous variables in adaptive forecasts.

Forecasting the impact of monetary policies on food prices

The previous subsection describes policy-neutral forecasts. Policymakers may also consider the potential impacts of various interventions. While most of our estimated coefficients can be interpreted approximately as short-term predictive elasticities, policy projections benefit from accounting for the tendency of variables to follow recent trends (‘dynamic dependencies’), particularly when projecting more than a month into the future⁴². Furthermore, a more comprehensive approach enables us to consider changes beyond transitory (or single-month) shocks. Long-standing assumptions about the independence of retail food markets from monetary policy have led to few studies examining the interplay between food markets and monetary policy or the macroeconomy^61,64.

Forecasting models incorporating exogenous variables can provide median (point) forecasts and prediction intervals reflecting policy interventions in these variables (projections). We examine four noteworthy case studies where fiscal policy adjustments could have been considered: the 2002 recession following the dot-com bubble, the 2009 Great Recession, a period of food-at-home price deflation around 2016, and the year after peak food-price inflation (2023). In each case, policymakers faced a decision to alter the money supply or the federal funds rate. A projection produced by a model, such as the one in Fig. 5, could have offered input to these decisions by estimating likely food price increases following changes in the money supply. Figure 6 shows the projection for a range of gradual changes in the money supply—from a negative 50% annualised decrease to a 300% increase—applied evenly each month from January to December in the respective years.

Fig. 6: The relationship between U.S. monetary policy and food prices changes over time.

The expected percent change in annual food-at-home prices following monthly increases of the money supply principal component analysis index with 68, 90, and 95% prediction intervals in 2002 (a), 2009 (b), 2016 (c), and 2023 (d).

An expanding money supply may increase consumer willingness to pay for food or alter competition for inputs along the supply chain. It also increases the derived demand for labour, materials, and transportation throughout the economy, not just within the food system. Regardless of the underlying mechanism, Fig. 6 shows how price expectations increase with a growing money supply. Before the onset of COVID-19, we observe a general decrease in the strength of the relationship between the money supply and food prices¹. In 2023, moving from a 50% annual decrease in the money-supply PCAI to a 300% annual increase leads to an expected increase in the food-at-home inflation rate from 3.0 to 10.8%. The direction of the effect provides forecasters with an unambiguous signal on how to inform policymakers about the likely effects of their policy choices. The caveat that all forecasts tend to perform better when series remain within historical ranges, which the money supply did not between 2020 and 2023, should accompany this and similar projections.

The forecasts and projections discussed here use data as they are publicly released. Beyond applying the appropriate forecasting methods, the timeliness and quality of information are crucial to improving public decision-making. Alternative public or proprietary data, as well as nowcasting, provide promising avenues for enhancing forecast precision, more rapidly anticipating changes (as seen in 2022), and reducing uncertainty arising from reporting delays and measurement errors^65,66,67.

Implications for other forecasters

Government and industry forecasters—particularly those evaluating market dynamics—may wish to adopt the approaches used here while carefully accounting for the statistical and policy realities of their setting. Importantly, improved performance resulting from adaptation and exogenous information would likely depend on the system’s stability over time and the quality of the information, respectively. To provide an example from US retail food markets, the Adjustments for alternative markets: example of food-away-from-home prices section in the Supplemental Information evaluates the food-away-from-home CPI, which follows distinct patterns from food-at-home prices. Social distancing, remote work, and COVID-19-related restrictions contributed to markedly diminished and altered consumption in the latter years of our study³.

Several recent publications by researchers at USDA and in academia evaluate federal forecasts of US agricultural markets—such as the USDA’s short (World Agricultural Supply and Demand Estimates, WASDE) and long-term (Agricultural Baseline)^44,45,68. In developing these projections, USDA analysts must adhere to policy neutrality, which they typically meet by assuming USDA and other federal policies will not change. With minor modifications, the framework developed in this paper would contribute to enhancing the accuracy, precision, and interpretability of other government or industry forecasts and projections. This framework also allows for complementing point estimates with prediction intervals, capturing forecast uncertainty. Considering and identifying the predictors of commodity market indicators helps analysts characterise market conditions under rapidly changing economic conditions. Our SARIMAX results provide up-to-date estimates, reducing reliance on historical models of agricultural systems and the potential for decreased forecast efficiency discussed in the Including exogenous variables subsection⁶⁹. A SARIMAX approach may be particularly apt when evaluating a single series rather than a national, regional, or global system.

Globally, this econometric framework could complement those used by other national agencies. For example, the work behind Canada’s annual Food Price Report could achieve improved measures of uncertainty and allow for more frequent reporting⁴⁷. For national agencies without current forecasts, this approach may also be desirable. The United Kingdom’s (UK) Office for National Statistics, for instance, could provide forecasts with their historical price series, such as the series for ‘food and nonalcoholic beverage’ (CPIH), to complement internal forecasting and expert opinions⁷⁰. Such forecasts could facilitate outreach from related entities such as the Bank of England⁷¹.

The relevance of the forecasting advancements in this paper extends beyond food markets. Other forecasters may consider adopting the approaches we present. Existing academic work explores SARIMAX models for forecasting next-day energy markets⁷². The EIA’s Short-term Energy Outlook can narrow its relatively wide prediction intervals by including exogenous variables. Potential applications abound throughout supply chains across various markets and economic sectors, supporting more efficient decision-making.

This paper presents an approach for reducing forecast errors below those previously reported by the USDA and others developed using cutting-edge tools from forecasting industry leaders^2,73. However, advancements beyond those presented here could improve the accuracy and explanatory power of forecasts. Alternative selection criteria could outperform minimising BIC values. Other information criteria, such as the Akaike Information Criterion with a small sample correction (AICc) and the Kullback-Leibler Information Criterion (KLIC), have been endorsed elsewhere in the literature. All information criteria provide an uncertain measure of Kullback-Leibler divergence, and forecasting with a ‘model confidence set’ may reduce forecast error⁷⁴. ‘Ensemble’ forecasts combine distinct individual forecasts and have been used under high specification uncertainty, such as climatology, the impact of climate change, and modelling headline inflation^28,75,76,77. While machine-learning methods based on out-of-sample loss functions have yielded mixed results in the literature, these techniques could be adapted for food price forecasting, particularly to identify specifications not comparable with information criteria⁷⁸. For example, the appropriate window of time to generate predictions requires the forecaster to strike a delicate balance between using the most relevant data in a possibly time-varying system against the statistical power offered by including a longer history of observations³⁸.

Accurate nowcasts can provide information before public releases of food-price data or related series⁶⁵. Well-performing estimates have been constructed for aggregate prices for numerous countries (including the United States) and food prices outside the United States using large databases of online prices^66,67. Similar approaches could aid FPO forecasts in providing more accurate forecasts and earlier notice of rapid price changes on a regular basis.

As the USDA publishes standardised forecasts, the impact of forecast enhancements on budgetary and food security outcomes can be more formally assessed. For example, the Current Population Survey Food Security Supplement could be used to measure the downstream impacts of improved information resources⁷⁹.

Methods

We build on the approach for forecasting food prices developed by USDA in 2022 and implemented in 2023, which replaced the legacy approach of allowing model results to inform expert opinion with systematic model selection at the time of the forecast^2,16. The legacy models had been based on limited and infrequent model comparisons. Model comparisons and case studies in USDA reports found that while intuitively appealing, the assumptions behind the legacy forecasts did not always lead to selecting models best suited to the data, suggesting the need to revisit long-standing assumptions². For instance, assuming particular forms of price transmission (or ‘pass-through’) from agricultural to retail food markets did not hold for wholesale and retail pork prices². Instead, researchers showed that performance, as measured by an information criterion such as BIC⁸⁰, could be improved by including (lagged) input prices. Selecting over a large set of possible specifications for exogenous variables may improve forecasts of retail food prices beyond the current univariate approach. Considering the benchmark SARIMA (with all exogenous variables excluded) as a possibility ensures that the forecast model performance, as measured by the BIC, will not decline.

Adaptive forecasts allow price models to change with the underlying food system rather than assuming they remain consistent after the study. Adaptation reveals the best approach to prediction rather than an invariant truth about the food system. Beyond allowing the data to select the optimal model at a given time, these approaches allow for changes in the relationships between current and lagged food prices and exogenous variables. Models can only be selected from a pre-defined set that must satisfy the assumptions behind the model, particularly when drawing sequential causal inferences.

We evaluate the in- and out-of-sample performance of alternative selection criteria among the SARIMAX class of models. BIC values indicate the relative model fit, with a penalty for complexity, similar to the adjusted coefficient of determination (R²) value used in linear regressions. Metrics such as RMSE and MAE measure the absolute forecasting accuracy of various models⁵⁸. The ratio of RMSE and MAE provides a straightforward test for whether the RMSE or MAE should be used to measure an expected error⁵⁸. Testing for reverse causality can help identify complications arising from endogeneity. The Supplementary Information contains additional details (see the Testing for reverse causality section in the Supplementary Information and Table S2).

Data: food-at-home prices, input prices, and macroeconomic variables

Our study uses data series consisting of food-at-home or food-away-from-home CPI, core CPI, disposable income, transportation congestion, wholesale prices for finished consumer foods, the principal components of US energy prices, the principal components of grocery-store employee and food service wages, and the principal components of the US monetary base, the M1 money supply, and the M2 money supply^{8,49,51,52,53,54,81,82}. These series include those considered in past iterations of FPO or ongoing research of food prices^16,55. Figure 7 displays the food-at-home CPI (a), all exogenous variables except transportation congestion (b), and congestion alone (c). We collect several similar series into indexes of the money supply, US energy prices, and wages, based on principal components analysis (PCA), as described in greater detail in the next subsection and the Supplementary Information. Here, we weight each index based on weights developed for the whole series, while this weighting adapts like the other series used in estimation. The Federal Reserve Bank of New York detrends its transportation congestion index before publication. It also begins later than the other series, providing an example of how new series may be incorporated into forecasts to enhance performance and explanatory power.

Fig. 7: U.S. food price and macroeconomic data used in forecasting.

Time series of a food-at-home Consumer Price Index (CPI), b core CPI, energy principal component analysis index (PCAI), money supply PCAI, wage PCAI, income PCAI, and wholesale food prices, and c the detrended transportation congestion index. The detrended transportation congestion and energy principal component analysis indexes January 1998–December 2023; all other series January 1990–December 2024. Note: The principal component analysis indexes here use weights estimated for the entire series, while the weights are re-estimated each month in the forecasting exercises.

SARIMAX specifications provide parsimonious models to treat various linkages to past observations (through the auto-regressive, differencing, or moving average terms) and accommodate data with and without seasonality (through seasonal differencing lags, differencing or moving average terms) and stationarity through differencing. Differencing allows non-stationary series to satisfy the assumptions of the SARIMAX class of models, but heteroskedasticity (variance that changes over time) presents an undesirable feature of the food-at-home CPI. We apply a log transformation to the food-at-home CPI data to mitigate heteroskedasticity and prevent the possibility of negative price forecasts.

Besides the already-detrended congestion index, each exogenous series exhibits non-stationarity and heteroskedasticity before transformation. After a natural log transformation, we take the first difference to adjust these series to modified versions suitable for the SARIMAX framework.

Overview of data treatment, econometrics, and testing causality

This paper focuses on the estimation and selection of time-series econometric models. Due to the complexity and technical nature of this process, this subsection presents only an overview. The Supplementary Information includes a comprehensive description for interested readers or those replicating this study.

Including additional series introduces a significant computational burden. To reduce the number of considered series while preserving explainability, we construct PCAI before estimation⁸³. Grouping series that describe similar economic phenomena avoids problems of excessive correlation and reduces the size of the estimation and selection problem, while allowing for the evaluation of the broad drivers of food prices. We group variables describing energy prices, wages, income, and the money supply in separate PCAIs.

Out of the nearly infinite possible ways to estimate a forecasting model, we estimate many SARIMAX (up to  ~150, 000 for each series in each month) models using the data described in the previous subsection. These widely applied models often outperform their counterparts developed from neural networks, which require fewer computational resources^40,72,78,84. The selected model minimises information loss as measured by the BIC.^85,86 We develop prediction intervals using Monte Carlo simulations using the model estimates and residual distribution from the selected model.

The three candidates include the current benchmark using SARIMA models without exogenous information (‘benchmark’), the proposed optimally selected SARIMAX model (‘optimal SARIMAX’), and the kitchen-sink approach². We compare the forecast performance of each model using two loss functions: MAE and RMSE. The ratio of the loss functions indicates which loss function better characterises the expected error, which is the MAE in this application⁵⁸.

Coefficients of lagged variables have often been interpreted as causal. However, several conditions must be met to support causal inference⁴⁸. In discussing food-away-from-home prices (see The Adjustments for alternative markets: example of food-away-from-home prices section in the Supplementary Information and Fig. S2), we describe how a confounding process with an autoregressive process hampers causal inference of the money supply. Additionally, reverse causality prevents any causal inference. Therefore, we test for reverse causality using a test for Granger causality^41,87. We find no evidence of reverse causality for most of the series. A standard Granger causality test provides evidence of reverse causality for electricity, the M1 money supply, and transportation. For electricity and transportation, a likelihood-ratio test rejects the null hypothesis that a forecasting model excluding food-at-home prices fits the data as well as one that includes them. This mixed evidence leads us to test the sensitivity of our results to the assumption that these series are exogenous, as presented in the Supplementary Information, where we find limited sensitivity in forecast performance (see Table and Fig. S3) and the development of predictions of policy impacts (see S4).

Implementation

We completed calculations, estimation, simulation, and figure development using R. Application programming interfaces (APIs) developed by the BLS and the St. Louis Federal Reserve Bank’s Federal Reserve Economic Data (FRED) enabled the retrieval of data from public repositories, allowing for continuous updates with data releases or revisions. Users must request keys from these federal agencies and use them in the appropriate location within the code. We have made all code, except for API keys used in this manuscript, publicly available; links can be found below.

For each month, we estimated the parameters of Eq. S1 for candidate model specifications. Each iteration of the estimation procedure considered all observations in the past 12 years of data and all observations that would have been available at the time of the forecast in the current year. ‘Piping’ and parallel computing significantly reduced run time. As described in Model selection based on model fit and parsimony section of the Supplementary Information, the model with the minimum BIC is selected. We store the BIC and the other metrics used in the manuscript for each model. This approach compares the out-of-sample performance of any sub-optimal (according to BIC values) model. It also facilitates the ready implementation of a model-confidence set⁷⁴.

Figure 8 provides a simplistic representation of how parallel processing divides similar tasks across many cores available on a machine (computer) to reduce the overall runtime.

Fig. 8: Parallel processing expedites model selection for adaptive forecasts.

Schematics of serial and parallel processing used in statistical learning and adaptive forecasting.

The read-me document accompanying the code provides instructions on how to use, modify, and interpret the results. The program automatically installs most libraries, but users must install R, RStudio, the RStudioAPI library, and the BLS API library beforehand.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.