Multi-Objective Feature Selection Techniques in Classification Systems

Feature selection is a critical pre-processing step in classification that aims to identify the most informative variables while discarding redundant or irrelevant ones. Multi-objective feature selection (MOFS) elevates this task by simultaneously optimising conflicting criteria, typically classification accuracy and model complexity or feature count. By casting feature selection as a Pareto-optimal problem, MOFS techniques provide a spectrum of trade-off solutions rather than a single optimum. Central approaches include filter methods that rely on statistical metrics, wrapper methods that integrate model performance, and hybrid or embedded schemes that fuse both philosophies. Evolutionary and swarm-intelligence algorithms—such as genetic algorithms, particle swarm optimisation and more recent chimp-inspired or gorilla-troop strategies—have shown particular strength in navigating large search spaces and maintaining diverse candidate solutions. These advances have found application across healthcare diagnostics, genomics, text classification and financial risk modelling, offering interpretable and computationally efficient classification pipelines on high-dimensional data.

Research from Nature Portfolio

Foundational work has demonstrated the utility of multi-objective particle swarm optimisation (PSO) for multi-label classification by constructing a Pareto archive of feature subsets that balance label coverage and subset cardinality. Novel adaptive mutation operators and local learning strategies have improved exploration of sparse regions in the search space, while crowding distance techniques have been incorporated into PSO to maintain solution diversity. Empirical studies on benchmark multi-label datasets established this approach as a versatile framework for discovering non-dominated feature subsets, paving the way for more sophisticated swarm-based MOFS algorithms.

Research from all publishers

A 2023 study introduced an evolutionary filter framework that integrates a neighbourhood component analysis objective into a differential evolution algorithm. This hybrid approach maximises class separation while minimising dimensionality and was shown to outperform state-of-the-art information-theoretic and rough-set methods across diverse datasets. In 2022, a discrete artificial gorilla troop optimisation technique was proposed for biomedical feature selection, deploying a label-mutual-information-based initialisation and multi-objective variants that combine filter and wrapper evaluations. Across ten medical datasets, this method achieved superior trade-offs between selected feature size and classification accuracy, including a case study on COVID-19. More recently, a binary multi-objective chimp optimisation algorithm with dual archives demonstrated robust performance on medical data, utilising chaotic maps and k-nearest neighbours to extract relevant clinical features. Comparative analyses against benchmark MOFS methods confirmed its strength in balancing feature reduction with predictive performance.

Multi-Objective Feature Selection Techniques in Classification Systems publication trend

The graph below shows the total number of articles in multi-objective feature selection techniques in classification systems across all publications each year (not limited to Nature Index journals).

Technical terms

Multi-objective optimisation: Simultaneous optimisation of two or more conflicting objectives, yielding a set of trade-off solutions rather than a single best solution.

Pareto set (front): The collection of non-dominated solutions in objective space, each representing a different balance between competing criteria.

Filter method: A feature selection approach that ranks features using statistical or information-theoretic metrics independent of any learning algorithm.

Wrapper method: A feature selection technique that evaluates subsets by training and testing a predictive model, directly optimising classifier performance.

Evolutionary algorithm: A nature-inspired optimisation process that iteratively refines a population of candidate solutions through selection, crossover and mutation.

Crowding distance: A diversity preservation measure in multi-objective algorithms that quantifies the density of solutions surrounding a candidate in objective space.

References

1. A PSO-based multi-objective multi-label feature selection method in classification. Scientific Reports (2017).
2. Approaches to Multi-Objective Feature Selection: A Systematic Literature Review. IEEE Access (2020).
3. An evolutionary filter approach to feature selection in classification for both single- and multi-objective scenarios. Knowledge-Based Systems (2023).
4. Feature Selection Using Artificial Gorilla Troop Optimization for Biomedical Data: A Case Analysis with COVID-19 Data. Mathematics (2022).
5. A Binary Multi-Objective Chimp Optimizer With Dual Archive for Feature Selection in the Healthcare Domain. IEEE Access (2021).