Evolutionary Optimization Algorithms in Continuous Domains

Evolutionary optimization algorithms constitute a class of derivative-free techniques inspired by principles of natural selection and genetics, tailored to optimise continuous real-valued functions. Central operators include selection, mutation and recombination, with populations of candidate solutions evolving over successive generations. Continuous domains require real-valued representations and adaptive variation mechanisms—most prominently evolution strategies (ES) and genetic algorithms (GA)—to navigate complex, multimodal landscapes. Advances in information-geometric optimisation and natural gradient frameworks have endowed these methods with invariance to parametrisation and facilitated efficient search in non-Euclidean spaces. Covariance Matrix Adaptation (CMA) has emerged as a pre-eminent self-adaptive ES, dynamically adjusting search distributions to learn problem structure. Recent trends encompass surrogate modelling to mitigate evaluation costs, hybridisation with gradient techniques for accelerated convergence and extensions into functional spaces, where reproducing kernel Hilbert spaces enable optimisation over infinite-dimensional manifolds. These algorithms have found broad application in engineering design, machine learning, robotics and control, demonstrating robustness to noise, scalability to high-dimensional problems and resilience against poorly conditioned landscapes. The global significance of these methods lies in their capacity to deliver near-optimal solutions without explicit derivative information, offering a versatile toolkit for complex real-world problems.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Research from all publishers

Contemporary studies have extended Covariance Matrix Adaptation Evolution Strategy into reproducing kernel Hilbert spaces, formulating a black-box framework that treats optimisation as a functional search. By representing policies or design variables as Gaussian processes, the adapted algorithm updates both mean functions and covariance operators, enabling optimisation in rich non-parametric spaces and demonstrating efficacy on functional benchmarks and reinforcement learning tasks.

A novel Proximal Evolutionary Strategy integrates layer-wise Covariance Matrix Adaptation with surrogate models and gradient-based local search to advance continuous control in deep reinforcement learning. This hybrid algorithm achieves reduced sample complexity and enhanced computational efficiency relative to established evolutionary and gradient-based methods, outperforming state-of-the-art policy-gradient techniques on a suite of continuous control challenges.

Analyses of modern neuro-evolutionary strategies reveal that evolutionary algorithms can match or surpass gradient-based reinforcement learning across diverse continuous control benchmarks. The study highlights the robustness of evolutionary approaches to hyper-parameter settings and parameter scaling, and underscores the complementary nature of reward functions in evolutionary and gradient-descent frameworks, prompting a reassessment of comparative efficacy.

Evolutionary Optimization Algorithms in Continuous Domains publication trend

The graph below shows the total number of articles in evolutionary optimization algorithms in continuous domains across all publications each year (not limited to Nature Index journals).

Technical terms

Evolutionary Algorithm: A derivative-free optimisation method that evolves a population of solutions through selection, mutation and recombination.

Evolutionary Strategy (ES): A subclass of evolutionary algorithms emphasising self-adaptation of mutation parameters and step sizes in continuous domains.

Genetic Algorithm (GA): An evolutionary algorithm that applies crossover and mutation operators to binary or real-valued representations to explore the search space.

Covariance Matrix Adaptation (CMA): A self-adaptive mechanism in ES that adjusts the covariance of the search distribution to capture problem structure.

Reproducing Kernel Hilbert Space (RKHS): An infinite-dimensional function space enabling non-parametric representation of solutions via kernel functions.

Surrogate Model: An approximate model of the objective function used to reduce computational expense by guiding the optimisation process.

References

1. Information-Geometric Optimization with Natural Selection. Entropy (2020).
2. A covariance matrix adaptation evolution strategy in reproducing kernel Hilbert space. Genetic Programming and Evolvable Machines (2019).
3. Proximal evolutionary strategy: improving deep reinforcement learning through evolutionary policy optimization. Memetic Computing (2024).
4. Efficacy of Modern Neuro-Evolutionary Strategies for Continuous Control Optimization. Frontiers in Robotics and AI (2020).