Bayesian Model Selection and Uncertainty Assessment

Bayesian model selection provides a coherent framework for comparing competing statistical models by balancing goodness of fit against model complexity through the use of prior distributions and likelihood functions. Rather than selecting a single “best” model, the Bayesian paradigm emphasises the quantification of uncertainty by computing posterior model probabilities or by averaging over candidate models. This approach addresses the risk of overfitting and acknowledges that no single model may fully capture the complexities of real-world phenomena. Uncertainty assessment in this context involves estimating the range of plausible parameter values and predictive outcomes, typically via credible intervals, posterior predictive checks and measures of predictive accuracy. By incorporating uncertainty at both the model and parameter levels, researchers achieve more robust inferences, vital for applications ranging from epidemiological forecasting and environmental modelling to economic risk analysis.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Research from all publishers

Recent studies have advanced computational efficiency and practical reliability in Bayesian model selection. One line of work has generalised model averaging by stacking predictive distributions, in which individual model predictions are combined with weights that optimise a cross-validation criterion. This approach offers improved predictive performance over traditional Bayesian model averaging, especially in “M-open” settings where the true data-generating process may lie outside the candidate set. Pareto smoothed importance sampling is employed to approximate leave-one-out posteriors efficiently, reducing computational demands while preserving accuracy.

Another development focuses on time series models, introducing approximate leave-future-out cross-validation (LFO-CV) to account for temporal dependence. By adapting methods from leave-one-out cross-validation and using importance sampling diagnostics, researchers can evaluate predictive accuracy for future observations without repeatedly refitting models from scratch. This innovation is particularly impactful in financial forecasting and climate projections where timely model evaluation is essential.

A third strand of research explores projection predictive variable selection using reference models. Here, a complex “reference” model captures the primary signal in the data, and its predictive distribution is projected onto simpler candidate models. This two-stage strategy filters noise and yields stable, interpretable submodels without sacrificing predictive quality. Applications in high-dimensional regression, spatial statistics and ecological modelling demonstrate how projection predictive methods can uncover sparse, meaningful structures while quantifying the uncertainty associated with each simplified model.

Bayesian Model Selection and Uncertainty Assessment publication trend

The graph below shows the total number of articles in bayesian model selection and uncertainty assessment across all publications each year (not limited to Nature Index journals).

Technical terms

Bayesian model selection: A framework comparing models via posterior probabilities derived from prior beliefs and data likelihoods.

Bayesian model averaging: A technique that weights predictions from multiple models by their posterior probabilities to account for model uncertainty.

Cross-validation: A data-driven method for estimating predictive performance by partitioning data into training and testing sets.

Pareto smoothed importance sampling (PSIS): A computational tool that stabilises importance sampling weights to approximate leave-one-out posteriors efficiently.

Leave-future-out cross-validation (LFO-CV): A validation method for time series that sequentially withholds future observations to assess predictive accuracy.

Projection predictive selection: An approach that projects the predictive distribution of a comprehensive reference model onto simpler submodels to achieve parsimony and stability.

References

1. Using Stacking to Average Bayesian Predictive Distributions. Bayesian Analysis (2018).
2. A Bayes Interpretation of Stacking for $\mathcal{M}$-Complete and $\mathcal{M}$-Open Settings. Bayesian Analysis (2017).
3. Approximate leave-future-out cross-validation for Bayesian time series models. Journal of Statistical Computation and Simulation (2020).
4. Using reference models in variable selection. Computational Statistics (2022).