Influence Analysis in Regression Models

Influence analysis examines the impact of individual data points or small subsets of observations on the fitted parameters and predictions of a regression model. Central to this endeavour are techniques that quantify the sensitivity of estimates to case deletion, data perturbation or model assumption changes. Classic measures such as Cook’s distance and leverage identify observations whose removal or down‐weighting produces substantial shifts in coefficients or fitted values. More recent developments have extended these ideas to local influence frameworks, which explore the curvature of a likelihood or loss function under infinitesimal perturbations in case weights, covariates or distributional assumptions. Such methods yield normal curvature measures that reveal the direction and magnitude of maximal sensitivity. In complex settings—mixed‐effect models, spatial regression, time series and models with heavy‐tailed or skew‐ed distributions—bespoke diagnostics have been devised, combining EM or MCMC algorithms with perturbation theory. Practical applications span financial risk modelling, environmental health studies, epidemiology and social sciences, where undetected influential points can distort inference, degrade predictive accuracy and misguide policy recommendations. A robust influence analysis thus underpins trustworthy regression practice by safeguarding against undue leverage, aberrant residuals and unrecognised model misspecification.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Research from all publishers

Recent work in autoregressive modelling under skew‐normal assumptions has incorporated local influence diagnostics to assess parameter sensitivity in time series frameworks. By formulating normal curvature measures for case‐weight, response and explanatory‐variable perturbations, researchers have shown improved detection of outliers in financial return data, yielding better model fit and enhanced forecasting accuracy.

In spatial regression, Bayesian influence analysis has been advanced for skew‐normal spatial autoregression models. By defining Bayes factors, φ‐divergence measures and posterior mean distances under various perturbation schemes, this approach quantifies the effect of small data or prior changes on posterior summaries. Case‐influence measures based on Cook’s posterior distances further facilitate identification of spatial outliers in environmental and epidemiological studies.

Within survival and censored‐data contexts, local influence diagnostics for log‐logistic regression models have been developed under case‐weight, covariate and response perturbations. These methods employ generalized Cook’s distance and one‐step Newton–Raphson updates to detect influential observations that could bias hazard estimates, offering practitioners practicable computational tools and clear guidance on model robustness.

Influence Analysis in Regression Models publication trend

The graph below shows the total number of articles in influence analysis in regression models across all publications each year (not limited to Nature Index journals).

Technical terms

Cook’s distance: A scalar measure of the change in all fitted values when an observation is deleted.

Leverage: A measure of an observation’s potential to influence estimated coefficients, derived from the design matrix.

Local influence: Assessment of model sensitivity to infinitesimal perturbations, using normal curvature of the likelihood or loss surface.

Normal curvature: The second‐order derivative of the objective function in the direction of maximal change, indicating the strength of influence.

Perturbation scheme: A specific form of data or model alteration (e.g., case‐weight, covariate or distributional change) used to probe sensitivity.

References

1. Diagnostic Analytics for an Autoregressive Model under the Skew-Normal Distribution. Mathematics (2020).
2. Bayesian Influence Analysis of the Skew-Normal Spatial Autoregression Models. Mathematics (2022).
3. Influence diagnostics in Log-Logistic regression model with censored data. Alexandria Engineering Journal (2022).