Bayesian Variable Selection in High-Dimensional Models

Bayesian variable selection addresses the challenge of identifying influential predictors from a vast pool of covariates by integrating prior beliefs with observed data. In high-dimensional contexts, where the number of candidate variables far exceeds the sample size, traditional selection methods often suffer from overfitting, poor generalisation and computational bottlenecks. The Bayesian paradigm overcomes these hurdles by imposing sparsity-inducing priors—such as spike-and-slab, Lasso and global–local shrinkage priors—that regularise coefficient estimates, effectively shrinking irrelevant parameters towards zero while preserving true signals. Contemporary advances leverage hierarchical modelling and continuous shrinkage priors like the horseshoe family to achieve near-minimax estimation and adaptivity to unknown sparsity levels. Computational strategies range from Markov chain Monte Carlo algorithms to variational Bayes and expectation propagation, each balancing inferential accuracy against scalability. Applications span genomics, where selection of gene-expression markers refines disease prognosis; neuroimaging, for isolating brain regions associated with cognitive processes; and macroeconomics, to discern key indicators among extensive financial datasets. Interdisciplinary efforts explore model averaging and multitask frameworks, enabling information sharing across related problems and enhancing predictive robustness. As data dimensions continue to expand, Bayesian techniques remain at the forefront of reliable, interpretable variable selection, combining theoretical guarantees with practical versatility.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Research from all publishers

Recent comparative studies have assessed the empirical performance of Bayesian variable selection methods in practical settings. One investigation contrasted Bayesian Kernel Machine Regression, Bayesian Semiparametric Regression and Bayesian Lasso within simulated and clinical cohorts, demonstrating that kernel-machine approaches excel with modest sample sizes while semiparametric techniques gain traction at scale, and Lasso-based shrinkage suits monotonic relationships. Another foundational work on the horseshoe and its generalisations introduced systematic priors for global shrinkage parameters, resolving specification issues and proposing the regularised horseshoe to bound maximal parameter estimates, thus bridging spike-and-slab properties with scalable continuous priors. A further line of work compared various spike-and-slab formulations, evaluating sampling efficiency and posterior inclusion probabilities across different slab distributions, and highlighted practical trade-offs between sparsity control and computational tractability. Collectively, these studies underscore the evolving toolkit for sparse Bayesian inference, linking theoretical insights on posterior concentration with applied guidelines for method selection across diverse domains.

Bayesian Variable Selection in High-Dimensional Models publication trend

The graph below shows the total number of articles in bayesian variable selection in high-dimensional models across all publications each year (not limited to Nature Index journals).

Technical terms

High-dimensional data: Data in which the number of variables greatly exceeds the number of observations.

Bayesian variable selection: A probabilistic framework for choosing relevant predictors by combining likelihoods with sparsity-inducing priors.

Shrinkage prior: A prior distribution that pulls coefficients towards zero to promote sparsity and reduce overfitting.

Spike-and-slab prior: A mixture prior comprising a point mass at zero and a diffuse component, enabling discrete inclusion decisions.

Horseshoe prior: A continuous global–local shrinkage prior notable for strong sparsity enforcement and heavy tails to retain large signals.

References

1. Sparsity information and regularization in the horseshoe and other shrinkage priors. Electronic Journal of Statistics (2017).
2. The horseshoe estimator: Posterior concentration around nearly black vectors. Electronic Journal of Statistics (2014).
3. Adaptive posterior contraction rates for the horseshoe. Electronic Journal of Statistics (2017).
4. Comparing Spike and Slab Priors for Bayesian Variable Selection. Austrian Journal of Statistics (2016).
5. Mean field variational Bayes for continuous sparse signal shrinkage: Pitfalls and remedies. Electronic Journal of Statistics (2014).