High-Dimensional Model Selection in Multivariate Statistics

High-dimensional model selection in multivariate statistics addresses the challenge of choosing an appropriate statistical model when both the number of variables and the sample size can grow large relative to each other. Traditional procedures often struggle as the ratio of dimensions to observations increases, leading to overfitting, spurious correlations and unreliable inference. Modern approaches adopt high-dimensional asymptotic frameworks—where variable and sample dimensions scale together—and focus on consistency guarantees, computational tractability and interpretability. Key advances include adaptive penalised likelihood methods, refined information criteria with dimension-dependent penalties and sequential screening techniques that respect complex covariance structures among multiple responses. These developments have enabled reliable selection of predictive and parsimonious models in fields such as genomics, neuroimaging, finance and environmental science, where the number of features frequently rivals or exceeds available samples. By balancing model complexity against predictive accuracy, current research is paving the way for robust inference even in ultra-high-dimensional settings.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Research from all publishers

Researchers have established the high-dimensional consistency of KOO methods for multivariate linear regression under various covariance structures, showing that true response–explanatory variable subsets are recovered with probability tending to one when both response and predictor dimensions grow proportionally with sample size. A complementary approach introduces a fast and consistent variable selection algorithm based on a generalised Cp criterion, which greatly reduces computational burden in large predictor spaces while retaining high probability of identifying the correct model even under moderate sample sizes. More recently, a new class of information criteria—PanIC—has been proposed to deliver broadly applicable consistency guarantees across both likelihood- and loss-based contexts. With easily verifiable regularity conditions, PanIC extends beyond classic Akaike and Bayesian penalties and demonstrates effective performance in mixture modelling, support vector regression and principal component analysis, unifying theory and practice in high-dimensional model choice.

High-Dimensional Model Selection in Multivariate Statistics publication trend

The graph below shows the total number of articles in high-dimensional model selection in multivariate statistics across all publications each year (not limited to Nature Index journals).

Technical terms

High-dimensional asymptotic framework: A theoretical setting in which the ratios of variable dimensions to sample size converge to positive constants, allowing both to grow jointly. Model selection consistency: The property that a selection procedure identifies the true underlying model with probability approaching one as sample size increases. Information criterion: A summary statistic combining a goodness-of-fit measure (often the log-likelihood) with a penalty proportional to model complexity to discourage overfitting. Penalised likelihood: An estimation strategy that augments the likelihood function with penalty terms (for example, L1 or L2 norms) to enforce sparsity or control model complexity. Covariance structure: The configuration of variances and covariances among multiple response variables, which influences both inference and model selection. Generalised Cp criterion: An extension of the classic Cp statistic designed to evaluate the trade-off between fit and complexity in regression settings with many predictors. Principal Component Analysis (PCA): A dimension-reduction method that identifies orthogonal directions capturing the greatest variance in multivariate data.

References

1. High-Dimensional Consistencies of KOO Methods for the Selection of Variables in Multivariate Linear Regression Models with Covariance Structures. Mathematics (2023).
2. A fast and consistent variable selection method for high-dimensional multivariate linear regression with a large number of explanatory variables. Electronic Journal of Statistics (2020).
3. PanIC: Consistent information criteria for general model selection problems. Australian & New Zealand Journal of Statistics (2024).