High-Dimensional Mean Testing in Statistical Inference

High-dimensional mean testing addresses the problem of comparing population average vectors when the number of variables approaches or exceeds the sample size. Classical multivariate procedures, such as Hotelling’s T², lose accuracy or become undefined as dimensionality grows, motivating a wave of methodological innovations. Modern approaches introduce regularisation techniques to stabilise covariance estimates, employ robust transformation-invariant statistics to mitigate the influence of outliers, and leverage Bayesian frameworks to integrate prior information. Recent work has established nonasymptotic bounds on test statistics, ensuring finite-sample guarantees even under weak distributional assumptions. Concurrently, advances in spatial-sign methodology and permutation-based inference have extended the toolbox to non-Gaussian and heavy-tailed settings. Across genomics, neuroimaging and finance, high-dimensional mean tests are now central to discovering subtle signal differences, guiding feature selection and controlling false discovery rates in large-scale experiments. By balancing statistical power, robustness and computational feasibility, current research is forging a unified theory that encompasses penalised likelihood, nonparametric rank-based methods and Bayesian decision rules, thereby expanding the applicability of mean comparison tests to ever more complex data structures.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Research from all publishers

Recent studies have established exponential tail bounds for regularised versions of the Hotelling’s T² statistic, demonstrating that a penalised covariance estimator with optimally chosen shrinkage parameters controls Type I error rates in finite samples without assuming exponential moment conditions. Parallel efforts have introduced robust permutation tests based on the minimum regularised covariance determinant estimator, showing that these procedures maintain nominal size and high power in the presence of outliers and contamination, with successful applications to Alzheimer’s disease biomarker data. In the Bayesian realm, the posterior Bayes factor for two‐sample mean comparison has been shown to converge to a normal limit under high-dimensional scaling, enabling the construction of tests with asymptotically valid significance levels and competitive power against classical competitors. These diverse threads converge in emphasising balance between regularisation strength, robustness under model misspecification and computational tractability, thereby shaping a more resilient paradigm for mean testing in ultra-large variable spaces.

High-Dimensional Mean Testing in Statistical Inference publication trend

The graph below shows the total number of articles in high-dimensional mean testing in statistical inference across all publications each year (not limited to Nature Index journals).

Technical terms

High-dimensional data setting: A regime in which the number of variables is comparable to or exceeds the number of observations, challenging classical inference methods.

Hotelling’s T² statistic: A multivariate extension of the Student’s t-test that compares sample mean vectors by accounting for covariance structure.

Regularisation: The introduction of a penalty or shrinkage term in estimation to prevent overfitting and to stabilise covariance or precision matrices in high dimensions.

Bayes factor: A ratio of marginal likelihoods under competing hypotheses, used in Bayesian testing to quantify evidence for one hypothesis over another.

Permutation test: A nonparametric inference procedure that assesses the significance of a test statistic by evaluating its distribution under random reassignments of labels.

Spatial sign: A transformation that projects multivariate observations onto the unit sphere, yielding test statistics that are robust to heavy tails and affine transformations.

References

1. Spatial-sign based high-dimensional location test. Electronic Journal of Statistics (2016).
2. Exponential bounds for regularized Hotelling’s T2 statistic in high dimension. Journal of Multivariate Analysis (2024).
3. A Two-Sample Test of High Dimensional Means Based on Posterior Bayes Factor. Mathematics (2022).