Functional Data Analysis and Outlier Detection

Functional data analysis (FDA) encompasses a suite of statistical methods designed to handle data observed over a continuum such as time, space or wavelength, where each observation is naturally viewed as a smooth function or curve. Key steps include representation of curves via basis expansions, smoothing to reduce measurement noise and alignment to adjust for phase variation. Outlier detection within FDA seeks to identify curves that diverge from the principal behaviour of a sample, accounting for both amplitude and shape differences. Techniques range from depth‐based rankings, which assign centrality scores to curves, to projection methods that compare local densities or model‐free envelopes. Such approaches have found broad application in areas as diverse as biomedical monitoring, environmental studies, industrial quality control and finance, where functional observations arise from continuous monitoring systems or repeat measurements.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Research from all publishers

scikit-fda has emerged as a comprehensive open-source Python toolkit for FDA, integrating representation, preprocessing, exploratory analysis and machine learning pipelines within the scikit-learn framework. It offers extensive documentation and examples for tasks such as basis fitting, registration and functional principal component analysis, facilitating reproducible analysis across disciplines.

An implementation of the Local Outlier Factor (LOF) algorithm on a high-performance distributed computing platform demonstrates the scalability of density-based anomaly detection for functional and high-dimensional data streams. The study evaluates hyperparameter effects, proposes enhancements to handle duplicate observations and benchmarks LOF against alternative approaches across big-data frameworks such as Spark, Hadoop and HPCC Systems.

Functional extensions of classical Mandel h and k statistics adapt univariate outlier tests to the context of interlaboratory studies where data appear as curves. By estimating functional critical limits through bootstrap resampling, this method identifies laboratories with inconsistent functional responses, preserving the rich information contained in time-resolved analytical measurements and improving sensitivity compared to pointwise approaches.

Functional Data Analysis and Outlier Detection publication trend

The graph below shows the total number of articles in functional data analysis and outlier detection across all publications each year (not limited to Nature Index journals).

Technical terms

Basis Expansion: Representation of functional observations by a weighted sum of basis functions (e.g. splines or Fourier), enabling dimensionality reduction and smoothing.

Data Depth: A measure of centrality of an observation within a sample of curves, used to order functional data from most typical to most extreme.

Local Outlier Factor (LOF): An unsupervised anomaly detection algorithm that assigns each observation a score based on the local density of its neighbourhood relative to that of its neighbours.

Curve Envelope: A model‐free projection set comprising a subset of past curves deemed most representative of a target curve, used for forecasting or anomaly assessment.

References

1. scikit-fda: A Python Package for Functional Data Analysis. Journal of Statistical Software (2024).
2. Local outlier factor for anomaly detection in HPCC systems. Journal of Parallel and Distributed Computing (2024).
3. Functional extensions of Mandel's h and k statistics for outlier detection in interlaboratory studies. Chemometrics and Intelligent Laboratory Systems (2018).