Surrogate-assisted global and distributed local collaborative optimization algorithm for expensive constrained optimization problems

Abstract

This paper presents a surrogate-assisted global and distributed local collaborative optimization (SGDLCO) algorithm for expensive constrained optimization problems where two surrogate optimization phases are executed collaboratively at each generation. As the complexity of optimization problems and the cost of solutions increase in practical applications, how to efficiently solve expensive constrained optimization problems with limited computational resources has become an important area of research. Traditional optimization algorithms often struggle to balance the efficiency of global and local searches, especially when dealing with high-dimensional and complex constraint conditions. For global surrogate-assisted collaborative evolution phase, the global candidate set is generated through classification collaborative mutation operations to alleviate the pre-screening pressure of the surrogate model. For local surrogate-assisted phase, a distributed central region local exploration is designed to achieve intensively search for promising distributed local areas which are located by affinity propagation clustering and mathematical modeling. More importantly, a three-layer adaptive selection strategy where the feasibility, diversity and convergence are balanced effectively is designed to identify promising solutions in global and local candidate sets. Therefore, the SGDLCO efficiently balances global and local search during the whole optimization process. Experimental studies on five classical test suites demonstrate that the SGDLCO provides excellent performance in solving expensive constrained optimization problems.

Introduction

Evolutionary algorithms (EAs) such as Differential evolution (DE)¹, Teaching-Learning-Based Optimization (TLBO)², Particle swarm optimization (PSO)³, Grey wolf optimizer (GWO)⁴have shown powerful search abilities in solving black-box problems where the gradient information of any input cannot be obtained. These algorithms not only handle single-objective optimization problems⁵but also demonstrate significant advantages in multi-objective optimization problems (MOOs)⁶, especially when multiple conflicting objectives need to be optimized simultaneously. As the problem dimensions increase, evolutionary algorithms continue to maintain strong competitiveness in high-dimensional optimization problems⁷. Specifically, various real-world engineering problems, such as the design of hybrid renewable energy systems⁸, task scheduling for airships⁹, the open vehicle routing problem¹⁰, and microgrid design¹¹, have been successfully solved using EAs. These problems are usually unconstrained optimization problems or with bound constraints. In fact, there are many real-world engineering cases involving complex constraints, such as computational fluid dynamics¹², computational electromagnetics¹³, and antenna design¹⁴. Zhou et al.¹⁵proposed an epsilon constraint handling method that simultaneously considers objective function values and constraint violations, and integrated it into a multi-objective evolutionary algorithm based on decomposition (MOEA/D)¹⁶. An adaptive fuzzy penalty method for mixed constraints optimization problems was proposed in¹⁷. Ang et al.¹⁸employed PSO with the feasibility rule-based constraint handling method to handle constraint optimization problems. The relevant literatures about constrained optimization problems are thoroughly reviewed in¹⁹. It can be easily concluded that incorporating classical constraint handling techniques, such as penalty function method²⁰, feasibility rules²¹, stochastic ranking method²², ensemble methods²³, into EAs for biasing the search directions towards feasible region has received wide-spread attention.

However, time-consuming simulations are required to obtain the objective and constraints of many complex engineering cases, and such kind of problems where both the objective and constraints are expensive to evaluate are defined as expensive constrained optimization problems (ECOPs). As we know, EAs usually consume 5000D where Dis the dimension of the solved problem functions evaluations to obtain satisfied or feasible solutions, which is computational unaffordable for ECOPs within an acceptable design cycle. Hence, effective surrogate models such as Kriging^{24,25,26,27,28}, RBF^29,30,31,32, SVM³³, etc., which can effectively approximate simulation problems, are used to replace unnecessary time-consuming simulation evaluations in evolutionary algorithms. This significantly accelerates the convergence speed of the algorithm. Such surrogate model-assisted evolutionary algorithms (SAEAs), widely applied in handling expensive constrained optimization problems, demonstrate excellent performance.

Generally, there are three key components in affecting the performance of SAEAs for ECOPs, such as constraint handling techniques, surrogate construction methods, and surrogate management strategies. Hence, the research progresses of SAEAs are summarized as follows from the perspectives of the three components.

Firstly, appropriate constraint-handling techniques (CHTs) can bias the search of the SAEAs towards the feasible region, thus the feasibility and convergence can be well maintained by designing effective combination between SAEAs and CHTs. Currently, several CHTs such as penalty function method²⁰, feasible rules²¹and the stochastic ranking method²² are widely employed in SAEAs. Specifically, the penalty function method guides the algorithm into the feasible region by incorporating the degree of constraint violation into the objective function, thereby penalizing the objective value of infeasible solutions. Li et al.³⁴explored methods for handling expensive inequality constraints within a dynamic surrogate-based optimization framework, in which the inequality constraints are handled by three different ways, such as constraining expected improvement (EI) function, penalizing surrogate prediction, and penalizing objective function. Multiple local surrogates and multiple penalty functions are designed in³⁵ where the constraints of each subproblem are penalized by adopting different penalty coefficients and search areas. Lu et al.³⁶ proposed an innovative Bayesian optimization method that effectively handles unknown equality and inequality constraints using exact penalty functions. However, penalty function method is sensitive to the penalize factor, and the value of the penalize factor is highly dependent on problems. In contrast, the advantage of stochastic ranking lies in its ability to effectively balance constraint feasibility and objective optimization through probabilistic ranking, thereby reducing the reliance on precisely designing penalty factors. Miranda-Varela et al.³⁷ studied the relationship between surrogate models and different constraint-handling techniques, and different combinations between surrogate-assisted DE and different CHTs are designed to test the performances. To address constrained combinatorial optimization problems, Gu et al.³⁸proposed an adaptive stochastic ranking strategy for developing an efficient surrogate-assisted multi-objective particle swarm optimization algorithm. A stochastic ranking-based surrogate-assisted evolutionary algorithm was proposed in³⁹to deal with offline data-driven optimization problems. Moreover, feasibility-based constraint handling techniques first prioritize feasible solutions by sorting them based on objective function values, and then sort infeasible solutions according to the degree of constraint violation, thereby guiding the algorithm into the feasible region. In⁴⁰, a classification collaborative mutation operation was designed to fully utilize information from both subpopulations and guide the evolutionary direction of the algorithm towards the feasible region. Wang et al.⁵ proposed a differential evolution algorithm that combines global and local surrogate models which provides an effective solution for addressing expensive constrained optimization problems, particularly those involving inequality constraints.

Secondly, the surrogate construction methods in SAEAs mainly refers to using different surrogates such as Kriging, RBF and SVM to provide guidance. Kriging is a machine learning method based on probabilistic statistics theory, which can be viewed as a stochastic process model used for interpolation. Zhang et al.⁴¹proposed a two-stage optimization method that combines the Kriging model, in which a new Pareto dominance relationship is created by incorporating probability distribution information derived from the Kriging model. By integrating a Kriging surrogate model, a multi-stage evolutionary algorithm designed in⁴² successfully enhances the efficiency and accuracy in solving expensive multi-objective optimization problems. Song et al.⁴³proposed a Kriging-assisted two-archive evolutionary algorithm where the information is exchanged between the two archives and the predictive capabilities of the Kriging model is used to guide the search process. By contrast, RBF is an approximation function based on the distance between input data and center points, and RBF model is widely used in SAEAs due to its excellent modeling efficiency. A distributed RBF-assisted differential evolution algorithm in⁴⁴ was designed to combine the RBF model with a distributed computing framework. Bai et al.⁴⁵ proposed a method that combines surrogate models with clustering-based sampling which effectively improves solution quality in high-dimensional expensive black-box optimization problems. Chen et al.⁴⁶ introduced an evolutionary algorithm based on the RBF surrogate model which combined the RBF model with the evolutionary algorithm. Support Vector Machine (SVM) is a supervised learning algorithm primarily used for classification, and the core idea of SVM is to find the optimal hyperplane that maximizes the margin between classes, thereby improving classification accuracy and robustness. Horaguchi et al.⁴⁷explored a classification-based multi-objective evolutionary algorithm (MOEA) for high-dimensional expensive optimization problems, incorporating dimensionality reduction techniques to improve optimization efficiency. A design optimization method that combines sequential radial basis functions and support vector machines was proposed in⁴⁸ which provides an effective solution for handling expensive objective functions. Liau et al.⁴⁹ proposed a decomposition-based enhanced multi-objective evolutionary algorithm that utilizes Support Vector Machine to predict and optimize the search process.

Thirdly, surrogate management strategies mainly refer to the combination ways between surrogates and evolutionary algorithms, and they can be generally categorized into three types: individual-based strategy, generation-based strategy, and population-based strategy. For the first one, the surrogate model is used to provide predictions for many candidate solutions, and the identified several promising solutions are evaluated by the truly expensive evaluation. Lin et al.⁵⁰ proposed a classification model-based assisted preselection and environment selection approach which provides an effective framework for solving expensive optimization problems. Regis et al.⁵¹ used the RBF model to find the most promising individual and improve the current overall best individual by finding the global minimum of the surrogate model. Wang et al.⁵proposed a differential evolution algorithm where both global and local surrogates were collaboratively employed. The generation-based strategy refers to using surrogate models for fitness evaluation in some generations of the evolutionary process, while using the truly expensive function or other surrogate models for fitness evaluation in other generations. The purpose of this approach is to reduce computational costs by decreasing the number of evaluations with the surrogate model, while retaining the ability to use the truly expensive function for precise evaluation at critical moments, thus balancing computational efficiency and optimization accuracy. In⁵², the global RBF model is restarted every few generations and the population is re-initialized at each restart using the best sample points. Zheng et al.⁵³ adopted a noise-based model switching strategy to select a surrogate model that meets the requirements from GP and RBFN for each generation. In different stages of the algorithm, Yu et al.⁵⁴ used coarse Gaussian process model and the fine radial basis function model respectively to assist the differential evolution algorithm. For population-based strategies, the selection and management of surrogate models are dynamically adjusted based on the current state of the population. This involves dynamically choosing suitable surrogate models to support global exploration or local optimization, and updating the models in real-time. Li et al.⁵⁵generated and updated surrogate by different techniques for different sub-populations and selected the suitable surrogates for each sub-population. In⁵⁶, a variable search region-based adaptive dynamic surrogate-assisted evolutionary computation method was introduced. As the current local optimal solution moves, the local search space is continuously transformed and the surrogate model is reconstructed. Couckuyt et al.⁵⁷ created a subpopulation for each surrogate model and dynamically selected the best model type based on the data available to date.

Also, there are roughly three kinds of SAEAs in solving ECOPs, i.e., global surrogate-assisted^40,52, local surrogated-assisted³⁵, and the collaborative between global and local surrogates⁵. For the first category, the surrogate is built by using all the evaluated solutions, then the search of EAs can be guided globally. This kind of algorithm provide powerful abilities in locating unexplored promising regions, but they are unable to achieve intensively local exploit on specific local regions. To alleviate this limitation, some algorithms introduce perturbation strategies that apply small random perturbations to the current solution, thereby exploring a larger solution space. Perturbation helps maintain diversity in local search and prevents premature convergence^58,59. However, it should be noted that these local surrogate-assisted optimization algorithms (especially those incorporating perturbations) may still not fully address complex ECOPs where many local optima exist in their landscapes. Therefore, for balancing the global exploration and local exploitation, the collaborative algorithms between global and local surrogates are widely devised where the global surrogate helps locate new promising regions and local surrogate accelerates the convergence in these located regions.

Based on the discussions, the existing SAEAs have indeed enhanced the optimization performance on solving ECOPs, but they cannot explicitly consider the differences of coverage regions for different solutions when executing local searches, or they don’t consider the similarity of surrogates constructed for different solutions in local searches. Thus, the corresponding local regions or surrogates of many local searches may be extremely similar, since their corresponding central solutions are closer to each other as the iterations progresses. Therefore, how to efficiently allocate well-distributed local searches during the optimization process is one of the research focuses of this paper. Moreover, these existing SAEAs mainly select the truly offspring through designing single-layer screening strategy constructed based on predicted objective or constraint values, in which the diversity among different offspring individuals may be ignored to a certain extent, thus the possibility of falling into local optima may not be effectively reduced. Therefore, two important search directions emerged in our mind, i.e., how to effectively design suitable fitness function on evaluating the quality of candidate solutions where the diversity and objective or constraint values can be comprehensively considered, and how to design effective multi-layer screen strategy to select promising candidate solutions layer by layer.

Therefore, this paper designs a surrogate-assisted global and distributed local collaborative optimization (SGDLCO) algorithm for expensive constrained problems. Specifically, for the aforementioned first search focuses, a distributed central region local exploration (DCRLE) is designed to diversify local searches where the affinity propagation clustering⁶⁰ and mathematical modeling are utilized to identify well-distributed central solutions. By minimizing redundant local searches for similar solutions or surrogates, the efficiency of exploitation is substantially improved. Subsequently, for the second point, the fitness functions such as diversity-based objective and constraints are designed to predict the overall quality of each candidate offspring individuals. Moreover, a three-layer adaptive screening strategy (TLAS) where the diversity, feasibility and overall quality are sequentially considered is proposed to progressively identify the truly promising offspring individual, thus the possibility of premature or trapping into local optima may be decreased since three key indicators are well balanced for each generation. Furthermore, for further improving the quality of candidate offspring individuals, a classification-cooperation mutation operation is employed in this paper to achieve the collaborative mutation between good and bad solutions.

Based on the considerations mentioned above, the main contributions of this paper are summarized as follows:

• 1) In the global surrogate-assisted collaborative evolution phase, the global candidate set is generated through classification collaborative mutation operations (CCMO) to alleviate the pre-screening pressure of the surrogate model. In CCMO, the positive guidance information of better solutions and the negative guidance information of worse solutions are fully utilized to improve the quality of candidate solutions.

• 2) In the local surrogate-assisted phase, a distributed central region local exploration (DCRLE) is designed to generate local candidate set for achieving intensively search in promising distributed local areas which are located by affinity propagation clustering and mathematical modeling. Thus, the redundant local searches for similar central solutions or surrogates can be largely alleviated, and the efficiency can be maintained by allocating specific local searches on well-distributed central solutions.

• 3) A three-layer adaptive selection strategy, which incorporates the diversity among different offspring individuals into the screening process, is adopted to progressively select promising candidate offspring solutions. Concretely, two diversity-based fitness functions where the feasibility, diversity and convergence are effectively balanced are formulated to screen candidate solutions from the global or local candidate sets.

• 4) Overall and systematic experiments on benchmark problems such as IEEE CEC2006⁶¹, and IEEE CEC2010⁶²and IEEE CEC 2017⁶³ have shown that the proposed SGDLCO provides highly competitive performance.

The structure of this paper is as follows: In Sect. 2, the problem statement and relative background techniques are presented. Section 3 describes the proposed SGDLCO. The experimental results are analyzed and summarized in Sect. 4. Conclusions are drawn in Sect. 6.

Problem statement and relative background techniques

Problem statement

The expensive constrained optimization problem with inequality constraints studied in this paper can be represented as:

$$\begin{gathered} minimize{\text{ }}f(\operatorname{x} ) \hfill \\ subject\, to{\text{ }}{g_j}(\operatorname{x} ) \leqslant 0{\text{ }}j=1,2,...,q \hfill \\ \end{gathered}$$

(1)

where $f(x)$ and ${g_j}(x)$ donates the objective and j-th inequality constraint functions where their responses can only be obtained by calling computationally expensive simulations, and q is the number of inequality constraints. ${x} =({{x} _1},{x_2},....,{{x} _D})$ is the decision vector, D is the number of decision variables, ${L} =({l_1},{l_2},...,{l_D})$ and ${U} =({u_1},{u_2},...,{u_D})$ are the lower and upper bounds of design variables. The degree of constraint violation of $\operatorname{x}$ can be computed as follows:

$${G_j}(x)=\hbox{max} (0,{g_j}(x)),j=1,...,p$$

(2)

Afterward, the overall constraint violation function can be formulated as:

$$G(x)=\sum\nolimits_{{j=1}}^{p} {{G_j}(\operatorname{x} )}$$

(3)

 

Therefore, the objective of the optimization problem is to find the optimal solution that minimizes the objective function while satisfying the constraint violation function $G(\operatorname{x} )$.

Relative background techniques

Differential evolution (DE)

DE is a population-based optimization algorithm that relies on swarm intelligence theory⁶⁴. It maintains the population by generating new individuals through mutation, selection, and crossover operations. By performing operations on the population of candidate solutions, DE explores the solution space through information exchange among different individuals in the population.

DE utilizes $NP$ parameter vectors of dimension D, treating them as the population for each generation. Each individual is represented as:

$${\operatorname{x} _i}=\left( {{\operatorname{x} _{i,1}},...,{\operatorname{x} _{i,D}}} \right)$$

(4)

For each individual in the population ${{x} _i}=\left( {{{x} _{i,1}},...,{{x} _{i,D}}} \right)$, the mutant vector is generated as follows:

(1) DE/rand/1

$${v_i}={\operatorname{x} _{r1}}+F \cdot \left( {{\operatorname{x} _{r2}} - {\operatorname{x} _{r3}}} \right)$$

(5)

(2) DE/best/1

$${v_i}={\operatorname{x} _{best}}+F \cdot \left( {{\operatorname{x} _{r1}} - {\operatorname{x} _{r2}}} \right)$$

(6)

(3) DE/best/2

$${v_i}={\operatorname{x} _{best}}+F \cdot \left( {{\operatorname{x} _{r1}} - {\operatorname{x} _{r2}}} \right)+F \cdot \left( {{\operatorname{x} _{r3}} - {\operatorname{x} _{r4}}} \right)$$

(7)

(4) DE/current-to-best/1

$${v_i}={\operatorname{x} _i}+F \cdot \left( {{\operatorname{x} _{best}} - {\operatorname{x} _i}} \right)+F \cdot \left( {{\operatorname{x} _{r1}} - {\operatorname{x} _{r2}}} \right)$$

(8)

In the above formula, $r1$, $r2$, $r3$, and $r4$ are four completely different random numbers which is selected from $[1,i) \cup (i,NP]$and $NP$ must be greater than 4. ${x_{best}}$ presents the optimal solution in the population. F is a scaling factor that amplifies the deviation variable. By performing crossover operation to increase the diversity of perturbed parameter vectors, the trial vector becomes:

$${u_{i,j}}=\left\{ {\begin{array}{*{20}{l}} {{v_{i,j}}{\text{ }} if\, ran{d_j} \leqslant CR\, or\, j={j_{rand}}} \\ {{x_{i,j}}{\text{ }}otherwise} \end{array}} \right.,{\text{ }}j=1,...,n$$

(9)

where $ran{d_j}$ presents the j-th estimate of a random number generator producing values between $[0,1]$, ${j_{rand}}$ is a randomly chosen integer from $\left\{ {1,...,n} \right\}$, $CR$ represents the crossover operator, which ranges from $[0,1]$.

Radial basis function (RBF)

RBF is a function whose output value depends solely on the distance between the input and a certain center point. RBF approximates complex nonlinear relations by introducing a linear combination o.

f local basis functions in the input space. RBF demonstrates excellent universal approximation capabilities and performs exceptionally well in handling nonlinear problems. Given the existing data $\{ {\operatorname{x} _i},{\operatorname{y} _i}|{\operatorname{x} _i} \in R,i=1,...,N\}$, the RBF approximation can be expressed as follows:

$$\hat {f}\left( \operatorname{x} \right)={\operatorname{w} ^T}\varphi =\sum\limits_{{i=1}}^{N} {{\omega _i}\phi \left( {\left\| {\operatorname{x} - {\operatorname{x} _i}} \right\|} \right)}$$

(10)

where ${\operatorname{x} _i}$ represents the computational center point of the ith basis function among all basis functions, $\left\| \bullet \right\|$ represents the distance between the input $\operatorname{x}$ and ${\operatorname{x} _i}$, N is the number of center points. ${\omega _i}$ is the weight coefficient, $\Phi$ is the Gram matrix which can be defined in Eq. (11).

$$\:{\Phi\:}=\left[\begin{array}{ccc}\varphi\:\left(\parallel{\text{x}}_{1}-{\text{x}}_{1}\parallel\right)&\:\cdots\:&\:\varphi\:\left(\parallel{\text{x}}_{1}-{\text{x}}_{N}\parallel\right)\\\:\vdots&\:\ddots\:&\:\vdots\\\:\varphi\:\left(\parallel{\text{x}}_{N}-{\text{x}}_{1}\parallel\right)&\:\cdots\:&\:\varphi\:\left(\parallel{\text{x}}_{N}-{\text{x}}_{N}\parallel\right)\end{array}\right]$$

(11)

Affinity propagation clustering (APC)

Affinity Propagation Clustering (APC) is an advanced unsupervised clustering algorithm developed by Frey and Dueck in 2007⁶⁰. This technique automatically determines the number of clusters by identifying cluster centers through iterative message passing. The algorithm operates on a similarity matrix, where each element represents the negative squared distance between data points. There are two kinds of messages involved in APC, they are responsibility $r(i,k)$ and availability $s(i,k)$. $r(i,k)$ can be computed as follows:

$$r(i,k) \leftarrow s(i,k) - \mathop {\hbox{max} }\limits_{{k^{\prime} \ne k}} \{ a(i,k^\prime)+s(i,k^{\prime})\}$$

(12)

where $s(i,k)$ is the similarity between i and k, and $a(i,k^{\prime})$ is the availability of $k^{\prime}$.

$a(i,k)$ can be computed as follows:

$$a(i,k) \leftarrow \hbox{min} \{ 0,r(k,k)+\sum\limits_{{i^{\prime} \notin \{ i,k\} }} {\hbox{max} \{ 0,r(i^{\prime},k)\} } \}$$

(13)

These two messages are iteratively updated till convergence, then the clustering result can be obtained by:

$${\hat {c}_i}=\arg \mathop {\hbox{max} }\limits_{k} \{ a(i,k)+r(i,k)\}$$

(14)

Proposed algorithm

Most existing self-adaptive evolutionary algorithms (SAEAs) use traditional differential evolution mutation operations to generate offspring individuals when dealing with expensive optimization problems with inequality constraints. This method has a significant degree of blindness in producing offspring individuals. Therefore, this paper employs the classification cooperation mutation operation (CCMO) during the global search phase to generate offspring individuals, thereby effectively utilizing limited computational resources. As the algorithm progresses, the search space gradually narrows. The range of local search often becomes excessively repetitive, which can significantly waste computational resources. Therefore, this paper designs a distributed central region local exploration (DCRLE) to avoid the redundancy in local search. The existing SAEAs primarily select truly promising offspring through a single-layer screening strategy based on designed predictive objectives or constraint values. However, the diversity among different offspring individuals may be somewhat overlooked, which could limit the effectiveness of reducing the likelihood of falling into local optima. While the current SAEAs have improved optimization performance for solving ECOPs by incorporating specific local search strategies, the center solutions corresponding to local searches for each population individual become increasingly similar as iterations progress, and thus many redundant local searches for similar solutions or surrogates are designed in these SAEAs. Therefore, to filter genuinely potential offspring solutions, a fitness function based on diversity, feasibility, and overall quality is designed to assess the overall quality of each candidate offspring individual. Additionally, a three-layer adaptive screening strategy (TLAS) is proposed, which sequentially considers diversity, feasibility, and overall quality. To diversify local searches, this paper introduces a distributed center region local exploration, and the affinity propagation clustering⁶⁰ and mathematical modeling are designed to identify well-distributed center solutions. This significantly reduces redundant local search iterations for similar solutions or alternatives.

SGDLCO Method

The flow chart of SGDLCO is shown in Fig. 1. In SGDLCO, the global and local surrogate-assisted phases are collaboratively executed where the global surrogate is employed to provide predictions for the global candidate solution sets generated by the classification collaborative mutation operation, and the local surrogate is used to build the distributed local search problems constructed for well-distributed central solutions selected based on affinity propagation clustering and mathematical modeling. More importantly, for effectively balancing feasibility, diversity and convergence, the three-layer adaptive screening strategy is designed to screen high-promising solutions from global and local candidate sets which are respectively generated in global and local surrogate-assisted phases. Thus, the proposed algorithm maintains efficient collaborative cooperation between global and local searches at each generation.

Fig. 1

Flowchart of SGDLCO.

The main framework of SGDLCO is shown in Algorithm 1. The initial population $Pop$is generated by Latin hypercube sampling (LHS)⁶⁵. Then, the objective function and constraint functions of all individuals in the initial population are evaluated, and the obtained results are stored in the database $DB$. For global surrogate-assisted collaborative evolution phase, the global RBF model is constructed by using all the individuals in $DB$. For each parent solution, the offspring candidate ${V_k}$​ is produced through the classification cooperation mutation operation (CCMO), as detailed in Sect. 3.2. $Gx$ is selected by applying the three-layer adaptive screening (TLAS) as described in Sect. 3.3 for candidate set ${V_k}$. Subsequently, $Gx$ is evaluated by the real objective and constraint functions, and the $DB$ is updated. $Pop$ is updated by selecting the top $NP$ individuals from the merged population of $Pop$ and $Gx$. During local surrogate-assisted phase, the candidate set $LV$ is generated by the distributed central region local exploration (DCRLE) as explained in Sect. 3.4. Similarly, the TLAS is used to select $Lx$, and then update $DB$ and $Pop$. The above procedure is repeated until the maximum number of fitness evaluations is reached.

Algorithm 1 SGDLCO method
Input:	Population size:$NP$. The maximum number of function evaluations:$MaxFEs$. The database:$DB$.
Output:	The optimized solution:${{\text{x}}_{best}}$.
1.	Generate the initial population,$Pop=\left\{ {{\operatorname{x} _1},{{\text{x}}_2},...,{{\text{x}}_{NP}}} \right\}$, by Latin hypercube sampling.
2.	Evaluate the initial population and archive them into $DB$, and set $FEs=NP$.
3.	While$FEs \leqslant MaxFEs$
4.	// The global surrogate-assisted collaborative evolution phase
5.	Construct global RBF models, i.e., ${\hat {f}^G},\hat {g}_{1}^{G},...,\hat {g}_{t}^{G}$, for objective and constraints by using all the individuals in$DB$.
6.	Generate candidate set, i.e., ${V_k}$ by the classification cooperation mutation operation (CCMO) shown in Algorithm 2.
7.	Conduct Algorithm 3 (called Three-Level Adaptive Screening, TLAS) to obtain the offspring set, i.e., $Gx$ from $GU=\left\{ {{{\text{u}}_1},...,{{\text{u}}_{NP}}} \right\}$.
8.	Evaluate $Gx$, and store them into $DB$, and $FEs=FEs+{N_{Gx}}$. //${N_{Gx}}$ donates the number of individuals in $Gx$.
9.	Select the best $NP$ individuals such as $Pop$ from the merged population based on FR, i.e., $MPop=Pop \cup Gx$.
10.	// The local surrogate-assisted phase
11.	Generate candidate set, i.e., $LV$ by the distributed central region local exploration (DCRLE) as explained in Algorithm 4.
12.	Conduct Algorithm 3 (called Three-Layer Adaptive Screening, TLAS) to obtain the offspring set, i.e., $Lx$ from $LV$.
13.	Evaluate $Lx$, and store them into $DB$, and $FEs=FEs+{N_{Lx}}$. //${N_{Lx}}$ donates the number of individuals in $Lx$.
14.	Select the best $NP$ individuals such as $Pop$ from the merged population based on FR, i.e., $MPop=Pop \cup Lx$.
15.	End While
16.	Select the best solution ${{\text{x}}_{best}}$ from $Pop$ based on FR.

Classification Cooperation Mutation Operation (CCMO)

Most of the existing SAEAs for handling expensive optimization problems with inequality constraints adopt traditional mutation operations of evolutionary algorithms to generate offspring individuals, which introduces a significant degree of randomness. Therefore, the classification-collaboration mutation operation designed in⁴⁰ is adopted in this section to design the mutation operation. For visualizing the search behavior of the designed CCMO, the classic DE/best/2 in DE is used as an example to illustrate the randomness of classical mutation. The DE/best/2 primarily generates mutated individuals by adding four random differential vectors to the current best individual, thereby creating a mutation direction. In this context, ${\text{x}}_{{best}}^{{}}$ represents the current optimal individual in the optimization process, while ${\text{x}}_{{r1}}^{{}}$, ${\text{x}}_{{r2}}^{{}}$, ${\text{x}}_{{r3}}^{{}}$, and ${\text{x}}_{{r4}}^{{}}$ are four individuals randomly selected from the current population, ${F_1}$ and ${F_2}$ are the scaling factors.

As shown in Fig. 2(a), the entire area inside the black line represents the overall design domain, FR and IFR represent feasible and infeasible regions respectively. Within this design domain, the area inside the purple line is the feasible region, while the remaining areas are infeasible region. The current population consists of 15 individuals, all represented as black dots. Thus, we first carry out the initial step of the DE/best/2 mutation by randomly selecting four different differential vectors. As shown in Fig. 2(b), the four green dots correspond to these four differential vectors. After determining the positions of the four differential vectors, it is necessary to further establish the selection order of these vectors and the values of the scaling parameters to generate mutated individuals. Due to the randomness of the mutation strategy, there are a total of 24 possible selection orders based on permutations. For simplicity, we will only show the visualization of the four cases corresponding to the limited rules, such as the first differential vector term consisting of ${\text{x}}_{{r1}}^{{}}$ and ${\text{x}}_{{r2}}^{{}}$, and the second differential vector term is composed of ${\text{x}}_{{r3}}^{{}}$ and ${\text{x}}_{{r4}}^{{}}$. Then the four cases are shown as below:

• (1) in Fig. 3(a), the mutation operation can be rewritten as: ${v_i}={\operatorname{x} _{best}}+{F_1} \cdot \left( {{\operatorname{x} _{r1}} - {\operatorname{x} _{r2}}} \right)+{F_2} \cdot \left( {{\operatorname{x} _{r4}} - {\operatorname{x} _{r3}}} \right)$ 

• (2) in Fig. 3(b), the mutation operation can be rewritten as: ${v_i}={\operatorname{x} _{best}}+{F_1} \cdot \left( {{\operatorname{x} _{r2}} - {\operatorname{x} _{r1}}} \right)+{F_2} \cdot \left( {{\operatorname{x} _{r4}} - {\operatorname{x} _{r3}}} \right)$ 

• (3) in Fig. 3(c), the mutation operation can be rewritten as: ${v_i}={\operatorname{x} _{best}}+{F_1} \cdot \left( {{\operatorname{x} _{r2}} - {\operatorname{x} _{r1}}} \right)+{F_2} \cdot \left( {{\operatorname{x} _{r3}} - {\operatorname{x} _{r4}}} \right)$ 

• (4) in Fig. 3(d), the mutation operation can be rewritten as: ${v_i}={\operatorname{x} _{best}}+{F_1} \cdot \left( {{\operatorname{x} _{r1}} - {\operatorname{x} _{r2}}} \right)+{F_2} \cdot \left( {{\operatorname{x} _{r3}} - {\operatorname{x} _{r4}}} \right)$

In Fig. 3, the red dashed arrows indicate the diversity of mutation directions resulting from different values of the scaling factors. The gray shaded areas correspond to the regions occupied by all mutated individuals. From these four subfigures, it can be observed that only the case shown in Fig. 3(d) leads all mutated individuals into the feasible region, while the mutated individuals in the other three cases are far from the feasible region. Therefore, in traditional mutation operations, the randomness in the selection of differential vectors leads to significant blindness in the mutation directions. When the surrogate model has some degree of prediction error, it becomes inefficient in filtering out a few high-potential individuals from a large pool of candidate offspring. Therefore, ensuring that each mutation is based on the situation in Fig. 3(d) to produce these high-potential individuals is the key focus of this section. In summary, this can be achieved by adjusting the selection rules for the four differential vectors in DE/best/2 to consistently obtain high-potential mutated individuals from the initial population shown in Fig. 2(b). Based on the visualization and the mutation formula in Fig. 3(d), we can derive two conclusions:

• (1) x_r1 and x_r3 are relatively close to the feasible region, and they produce a positive guidance effect.

• (2) x_r2 and x_r4are relatively far from the feasible region, and they produce a negative guidance effect

Therefore, for fully utilizing the positive guidance information of better solutions and the negative guidance information of worse solutions, the population is classified into two subpopulations based on feasibility rules in²¹, in which the first subpopulation consists of better solutions and the other is composed of the rest solutions. After this, the candidate set is generated for each parent solution by collaborative cooperation between the two subpopulations, and a high-promising candidate individual is selected from this candidate set based on RBF predictions. The pseudo-code of classification cooperation mutation operation (CCMO) is shown in Algorithm 2.

In Algorithm 2, the advantage of selecting DE/best/2 as the mutation operator is that it combines information from both the better and worse solutions in the current population. This enables the mutation process to move towards promising regions indicated by the best solutions while avoiding unfavorable areas. Therefore, DE/best/2 is well-suited to construct the CCMO shown in lines 5–6 of Algorithm 2. Specifically, two different solutions such as ${\text{x}}_{{r1}}^{B}$ and ${\text{x}}_{{r3}}^{B}$, which are randomly selected from $BPop$, are utilized to determine the positive guidance locations of CCMO; while ${\text{x}}_{{r2}}^{W}$ and ${\text{x}}_{{r4}}^{W}$ are selected from $WPop$ to locate the negative guidance. However, incorporating massive current greedy information to mutation may lead the algorithm into local optima more easily. Hence, different cooperative ways are employed to adjust the evolution of the population based on its current state, which is shown in lines 4–11. Concretely, the current greedy information brought by the better solutions or the best solutions is gradually diluted when the algorithm presents instability or stagnation, i.e., the number of failures reaches the predetermined value.

Algorithm 2 Classification Cooperation Mutation Operation (CCMO)
Input:	The population: $Pop$.Threshold for determining which mutation strategy to use: $Fv$.The count of failed attempts to update the population’s best solution: $T\_fail$.
Output:	${V_k}$: Candidate set.
1.	Divide the $Pop$ into two subpopulations based on FR where the $BPop$ contains the top half individuals, and the $WPop$ contains the rest.
2.	For$i=1:NP$
3.	For$j=1:NC$
4.	If$T\_fail<Fv$
5.	Randomly select two distinct individuals ${\text{x}}_{{r1}}^{B}$ and ${\text{x}}_{{r2}}^{B}$ from $BPop$, ${\text{x}}_{{r3}}^{W}$ and ${\text{x}}_{{r4}}^{W}$ from $WPop$.
6.	${w_j}={{\text{x}}_{best}}+{F_1}({\text{x}}_{{r1}}^{B} - {\text{x}}_{{r3}}^{W})+{F_2}({\text{x}}_{{r2}}^{B} - {\text{x}}_{{r4}}^{W})$. // ${F_1}={{\left( {1+rand(0,1)} \right)} \mathord{\left/ {\vphantom {{\left( {1+rand(0,1)} \right)} 2}} \right. \kern-0pt} 2}$, ${F_2}={{\left( {1+rand(0,1)} \right)} \mathord{\left/ {\vphantom {{\left( {1+rand(0,1)} \right)} 2}} \right. \kern-0pt} 2}$
7.	Else If$T\_fail>Fv{\text{ }}\& {\text{ }}T\_fail \leqslant 2Fv$
8.	${w_j}={{\text{x}}_{best}}+{F_1}({{\text{x}}_{r1}} - {{\text{x}}_{r2}})+{F_2}({{\text{x}}_{r3}} - {{\text{x}}_{r4}})$, ${{\text{x}}_{r1}}$, ${{\text{x}}_{r2}}$, ${{\text{x}}_{r3}}$, and ${{\text{x}}_{r4}}$ are four distinct individuals selected from $Pop$.
9.	Else If$T\_fail>2Fv$
10.	${w_j}={{\text{x}}_{r1}}+{F_1}({{\text{x}}_{r2}} - {{\text{x}}_{r3}})+{F_2}({{\text{x}}_{r4}} - {{\text{x}}_{r5}})$, ${{\text{x}}_{r1}}$, ${{\text{x}}_{r2}}$, ${{\text{x}}_{r3}}$, ${{\text{x}}_{r4}}$ and ${{\text{x}}_{r5}}$ are five distinct individuals selected from $Pop$.
11.	End If
12.	Obtain ${u_j}$ by conducting binomial crossover on ${w_j}$ and ${{\text{x}}_i}$.
13.	End For
14.	Select the best one such as ${v_k}$ from ${U_k}$ based on FR by using the global RBF predictions on ${U_k}$, and ${V_k}={V_k} \cup \{ {v_k}\}$.// ${U_k}=\left\{ {{u_1},...,{u_{NC}}} \right\}$.
15.	End For

As the algorithm progresses, the proportion of feasible solutions in the population gradually increases, while the proportion of infeasible solutions gradually decreases. The situation in which all the solutions are infeasible is selected as the example to clearly present the trajectory of mutation vectors, the tendency of movement of mutant vectors is shown in Fig. 4. In Fig. 4, the population $Pop$ is divided into $BPop$ and $WPop$. $BPop$ includes the top half solutions of $Pop$, and $WPop$ is composed of the remaining part. It can be easily found that the $BPop$ is closer to the feasible region compared to $WPop$. This indicates that the generated candidate solutions should be moved towards the positive directions located by $BPop$, and the negative directions of $WPop$ can be also used as guidance to further move the candidate solutions towards feasible region. Specifically, ${\operatorname{x} _{r1,g}}$ and ${\operatorname{x} _{r3,g}}$ are two randomly selected solutions from $BPop$, ${\operatorname{x} _{r2,g}}$ and ${\operatorname{x} _{r4,g}}$ are two randomly selected solutions from $WPop$. The line connecting ${\operatorname{x} _{r1,g}}$ to ${\operatorname{x} _{r2,g}}$, and the line connecting ${\operatorname{x} _{r3,g}}$ to ${\operatorname{x} _{r4,g}}$ represent the direction and length of ${F_1} \cdot ({\operatorname{x} _{r1,g}} - {\operatorname{x} _{r2,g}})$ and ${F_2} \cdot ({\operatorname{x} _{r3,g}} - {\operatorname{x} _{r4,g}})$ respectively. Hence, All the possible cases that ${F_1} \cdot ({\operatorname{x} _{r1,g}} - {\operatorname{x} _{r2,g}})$ can be extended are on the red dotted line between ${\operatorname{x} _{r1,g}}$ and ${\operatorname{x} _{r2,g}}$. The red dotted line between ${\operatorname{x} _{r3,g}}$ and ${\operatorname{x} _{r4,g}}$ contains all the possible cases that ${F_2} \cdot ({\operatorname{x} _{r3,g}} - {\operatorname{x} _{r4,g}})$ can be extended. Therefore, the parallelogram area formed by vertices $\operatorname{w} _{{i,g}}^{1}$, $\operatorname{w} _{{i,g}}^{2}$, $\operatorname{w} _{{i,g}}^{3}$, and $\operatorname{w} _{{i,g}}^{4}$ represents the region where the candidate offspring solutions are located. Hence, although all the solutions in current population are infeasible, all of the generated candidate solutions are moved into feasible region. This directly demonstrate that CCMO is capable of generating high-promising candidate solutions, then the screen pressure of RBF can be greatly deceased.

Fig. 2

Randomly select four individuals from the initial population.

Fig. 3

Four cases of DE/best/2.

Fig. 4

The tendency of movement of mutant vectors when all solutions are infeasible.

Three-layer adaptive screening (TLAS)

During the algorithm’s iterations, the selection strategy often becomes too stringent and only allows the optimal solutions to survive, the diversity among different offspring individuals may be ignored to a certain extent. This excessive selection pressure reduces the population’s diversity, increasing the risk of falling into local optima. When the population tends to consist of highly similar individuals, the algorithm is prone to getting stuck in local optima and fails to effectively find global optima. This situation can lead to a homogenized search process, resulting in premature convergence. Therefore, maintaining a certain level of diversity is crucial to promote a more comprehensive search and innovation, avoiding premature convergence to local optima. By introducing diversity maintenance mechanisms, the adaptability of the algorithm can be enhanced, increasing the chances of finding better solutions. To alleviate the selection pressure on the population, as shown in Algorithm 3, a three-layer adaptive screening strategy (TLAS), which incorporates the diversity among different offspring individuals into the screening process, is adopted to progressively select promising candidate offspring solutions.

Specifically, TLAS sequentially considers the diversity, feasibility, and overall quality of each individual, and selects offspring based on these three indicators.

The first layer of the screening function is shown in lines 2–3 of Algorithm 3. The minimum distance calculated by Eq. (15) represents the shortest distance between an individual and all other individuals in $TDB$. For each individual in the candidate $CS$, the minimum distance $Dist({\operatorname{x} _i})$ from the individual to all individuals in the total database $TDB$ except itself is calculated. As the iterations progress, the individuals in the population gradually become homogeneous. In fact, the individual with the biggest minimum distance to all other individuals indicates that it is located in a relatively isolated region in the design space. This means that the individual exhibits greater diversity from other individuals. Therefore, the individual with the biggest $Dist({\operatorname{x} _i})$ is selected. It represents relatively unique positions in the design space, which helps prevent the algorithm from falling into local optima.

$$Dist({\operatorname{x} _i})=\mathop {\hbox{min} }\limits_{{{\operatorname{x} _i} \in CS}} \left\| {{\operatorname{x} _i} - TDB} \right\|$$

(15)

The second layer of the screening function is shown in line 4 of Algorithm 3. After selecting the individual with diversity as described above, the individual with the best predicted value in the current population is selected based on FR. Such individual plays a role in guiding the evolution direction of the population, and the region where the individual resides represents a more promising area in the sample space.

Given the condition that the individual selected based on diversity may not has better quality measured by the predicted objective and constraints, and the individual selected based on the combined quality of the predicted objective and constraints cannot reflect the differences between individuals in the population. Therefore, the third layer screening strategy is designed for selecting the individual with overall quality as shown in lines 7–8 of Algorithm 3.

A mathematical model for screening individuals based on their overall quality is designed, as shown in Eq. (16):

$$\begin{gathered} {{\hat {f}}_{div}}({\operatorname{x} _i})=\hat {f}({\operatorname{x} _i})/Dist({\operatorname{x} _i}) \hfill \\ {{\hat {g}}_{j,div}}({\operatorname{x} _i})={{\hat {g}}_{j,div}}({\operatorname{x} _i})/Dist({\operatorname{x} _i}) \hfill \\ \end{gathered}$$

(16)

where $\hat {f}({\operatorname{x} _i})$ and ${\hat {g}_{j,div}}({\operatorname{x} _i})$ respectively represents the predicted values of objective and j-th constraint functions for ${\operatorname{x} _i}$. Then${\hat {f}_{div}}({\operatorname{x} _i})$ donates diversity-based objective, which represents the combined quality between objective and diversity; ${\hat {g}_{j,div}}({\operatorname{x} _i})$ donates diversity-based constraint, which calculates the combined quality between constraints and diversity. Then the FR is employed to select a high-promising solution in terms of the overall quality of diversity, feasibility and diversity.

In summary, TLAS select individuals from the current population based on their diversity, feasibility, and comprehensive quality, aiming to maintain the quality of population individuals while generating a more exploratory population. Consequently, TLAS significantly increases the chances of finding global optimal solutions in the solution space rather than getting trapped in local optima.

Algorithm 3 Three-Layer Adaptive Screening (TLAS)
Input:	Candidate set: $CS$($GU$ or $LV$). The Database: $DB$. The lower bound: $xl$. The upper bound: $xu$. The dimensions of an individual: n. The number of individuals which is selected: ${N_1}$.
Output:	Offspring set: $OS$($Gx$ or $Lx$).
1.	Construct the merged database, i.e., $MDB=DB \cup CS$, and initialize $OS=\emptyset$.
2.	For each ${{\text{x}}_i}$ in $CS$, $TDB=MDB\backslash \left\{ {{{\text{x}}_i}} \right\}$ and calculate the minimum distance $Dist\left( {{{\text{x}}_i}} \right)=\mathop {\hbox{min} }\limits_{{{x_i} \in CS}} \left\| {{{\text{x}}_i} - TDB} \right\|$.
3.	Store an individual such as ${\text{o}}{{\text{x}}_{dist}}$ from $CS$ with the biggest $Dist\left( {{\text{o}}{{\text{x}}_{dist}}} \right)$ into $OS$.
4.	Archive the best one such as ${\text{o}}{{\text{x}}_{best}}$ into $OS$ from $CS$ based on FR by using the RBF predictions on $CS$.
5.	$CS=CS\backslash \left\{ {{\text{o}}{{\text{x}}_{best}}} \right\}$, and randomly select an individual such as ${\text{o}}{{\text{x}}_{rand}}$ from $CS$, and $OS=OS \cup \left\{ {{\text{o}}{{\text{x}}_{rand}}} \right\}$.
6.	Delete the similar individuals in $OS$ within the minimum distance threshold $Dis\_eps$.//$Dis\_eps=min(\sqrt {0.00{1^2}n} ,5{e^{-4n}}min(xu-xl))$.
7.	Calculate the diversity-based objective for each ${\operatorname{x} _i}$ in $CS$, i.e., ${\hat {f}_{div}}({{\text{x}}_i})={{\hat {f}({{\text{x}}_i})} \mathord{\left/ {\vphantom {{\hat {f}({{\text{x}}_i})} {Dist({{\text{x}}_i})}}} \right. \kern-0pt} {Dist({{\text{x}}_i})}}$.
8.	Calculate the diversity-based constraints for each ${\operatorname{x} _i}$ in $CS$, i.e., ${\hat {g}_{j,div}}({{\text{x}}_i})={{{{\hat {g}}_{j,div}}({{\text{x}}_i})} \mathord{\left/ {\vphantom {{{{\hat {g}}_{j,div}}({{\text{x}}_i})} {Dist({{\text{x}}_i})}}} \right. \kern-0pt} {Dist({{\text{x}}_i})}},\forall j \in \{ 1,...,t\}$.
9.	Select the top ${N_1} - {N_{OS}}$ individuals from $CS$ based FR by using the diversity-based objective and constraints and store them into $OS$.

Distributed Central Region Local Exploration (DCRLE)

As the algorithm progresses, the search scope tends to concentrate on a small range of more promising regions. Thus, the corresponding local regions or surrogates of many local searches may be extremely similar, since their corresponding central solutions are closer to each other, resulting in a limited exploration of new solution spaces. This limitation not only reduces the diversity of the algorithm but also increases the risk of local optima, as the algorithm may repeatedly evaluate similar solutions while neglecting other potentially high-quality solutions. As iterations continue, an excessive focus on these similar regions can lead to a decline in the algorithm’s innovative capacity, making it difficult to escape the current local optimum, thereby affecting the overall search efficiency and effectiveness. Therefore, designing effective mechanisms to expand the search range and encourage the exploration of more diverse solution areas will help enhance the algorithm’s performance and increase the likelihood of finding global optima. Therefore, as shown in Algorithm 4, distributed central region local exploration (DCRLE) is designed to effectively allocate uniformly distributed local searches. It utilizes affinity propagation clustering and mathematical modeling to identify uniformly distributed central solutions. However, we found that during the progression of the algorithm, two different clustering situations are shown in Figs. 5 and 6. We take g6 from CEC2006 for an example. In these two figures, each point represents an individual and the different clusters generated by APC are distinguished by points of different colors. The polygons formed by connecting points of the same color represent the areas covered by each cluster. x and y represent two different dimensions of the design variables. Figure 5 represents the clustering situation at the initial stage of the algorithm where affinity propagation clustering divides the population into four clusters, represented by the four polygons in Fig. 5. However, as the algorithm progresses, the population tends to exhibit over-clustering due to low diversity, as shown in Fig. 6. When the number of iterations reaches 573, as indicated in the lower left corner of Fig. 6, each point represents its own cluster, with only one point in each cluster. This occurs because they lack sufficient similarity or shared features to form larger clustering structures, and they do not have enough similarity with other points to form larger clusters. At this point, if we directly perform local searches on each cluster, it will lead to excessive redundancy in the search areas. Then the mathematical modeling method shown in lines 1–7 of Algorithm 4 is used to reselect the central points for well-distributed local searches. For a detailed elaboration, the step-by-step introduction is shown below.

Step 1 in line 1, the cluster results of APC provide effective guidance to arrange the well-distributed local search.

Step 2 in lines 2–8, when the population demonstrates an excessive number of clusters resulting from high similarity among individuals, three central individuals are selected or generated based on different distributed information to ensure that the central regions of local searches designed around these individuals are diversified.

Step 3 in line 10, to determine the local search scope for each cluster, the K nearest points to each cluster center ${\text{c}}{{\text{x}}_k}$ in the database DB are stored into $M{X_k}$. K is calculated as follows.

$$K=\hbox{max} ((D+1)(D+2)/2,100)$$

(17)

where D represents the number of dimensions.

Step 4 in line 11, to avoid irreversibility of the Gram matrix in constructing RBF, similar individuals within the distance threshold $Dis\_eps$ are removed in each cluster. Then the local RBF models are constructed using the remaining individuals for each cluster where the compute of $Dis\_eps$ is consistent with the formula in line 6 of Algorithm 3.

Step 5 in lines 12–18, the inner optimization problem for conducting local search on each cluster is formulated as follows.

$$\begin{gathered} minimize{\text{ }}\hat {f}(\operatorname{x} ) \hfill \\ subject to{\text{ }}{{\hat {g}}_j}(\operatorname{x} ) \leqslant 0{\text{ }}j=1,2,\cdots,q \hfill \\ {\text{ }}x{L_i}<{\operatorname{x} _i}<x{U_i},i=1,\cdots,D \hfill \\ {\text{ }}{{\hat {g}}_{dis}}(\operatorname{x} ) \leqslant 0 \hfill \\ \end{gathered}$$

(18)

where $\hat {f}(\operatorname{x} )$ and ${\hat {g}_j}(\operatorname{x} )$ are the RBF surrogates built with the K nearest points for the original objective and the j-th constraint, respectively. $x{U_i}$ and $x{L_i}$ respectively represents the upper and lower bounds of i-th dimension in the current local search scope. ${\hat {g}_{dis}}(\operatorname{x} )$ donates the distance constraint calculated based on the diversity threshold which is computed in Eq. (19).

$${\hat {g}_{dis}}({\operatorname{x} _i})=Dis\_eps - Dist({\operatorname{x} _i})$$

(19)

Step 6 in lines 19–21, the current population in inner optimization is regenerated when the current population trapped in a local infeasible region, i.e., all the individuals are infeasible and their constraint violations show extremely small differences.

Step 7in lines 22–33, based on the recommendation in CoDE⁶⁶, three different combinations of mutation operations and parameters are employed to generate diverse offspring solutions.

Step 8 in line 36, the best solution $l{v_k}$ is obtained by completing the inner optimization for each central solution.

Fig. 5

The situation of normal APC.

Fig. 6

The situation of excessive APC.

Algorithm 4 Distributed Central Region Local Exploration (DCRLE)
Input:	$Pop$, $DB$, $NP$, total evaluation times: $totalFEs$, K, $Dis\_eps$.
Output:	$LV$
1.	The $Pop$ is clustered into np clusters, such as $C{X_1},...,C{X_{np}}$ based on APC.
2.	If$np \geqslant \left\lfloor {0.2NP} \right\rfloor$ // Re-select three individuals based on the distribution of $Pop$ when the clustered number is large.
3.	Reset $np=3$, for each individual in $Pop$, calculate the minimum distance $Dist\left( {{{\text{x}}_i}} \right)=\mathop {\hbox{min} }\limits_{{{\operatorname{x} _i} \in Pop}} \left\| {{{\text{x}}_i} - DB} \right\|$.
4.	Select an individual such as ${\text{c}}{{\text{x}}_1}$ and ${\text{c}}{{\text{x}}_2}$ with the maximum and minimum i.e., $Dist\left( {{\text{c}}{{\text{x}}_1}} \right)$ and $Dist\left( {{\text{c}}{{\text{x}}_2}} \right)$.
5.	Obtain the mean individual of $Pop$, i.e., ${\text{c}}{{\text{x}}_{3,j}}={{\sum\limits_{{i=1}}^{{NP}} {{{x} _{i,j}}} } \mathord{\left/ {\vphantom {{\sum\limits_{{i=1}}^{{NP}} {{\operatorname{x} _{i,j}}} } {NP}}} \right. \kern-0pt} {NP}},\forall j \in \left\{ {1,...,n} \right\}\forall {x_i} \in Pop$ .
6.	Else If
7.	Obtain the mean individual of $C{X_k}$ for each dimension, i.e., ${\text{c}}{{\text{x}}_{3,j}}={{\sum\limits_{{i=1}}^{{NP}} {{\operatorname{x} _{i,j}}} } \mathord{\left/ {\vphantom {{\sum\limits_{{i=1}}^{{NP}} {{\operatorname{x} _{i,j}}} } {NP}}} \right. \kern-0pt} {NP}},\forall j \in \left\{ {1,...,n} \right\}\forall {x_i} \in C{X_k}$.
8.	End If
9.	For$k=1:np$
10.	Store the K nearest individuals to ${\text{c}}{{\text{x}}_k}$ into $M{X_k}$ from $DB$ and $M{X_k}=M{X_k} \cup N{P_k}$.
11.	Delete the similar individuals in $M{X_k}$ and construct local RBF models, i.e., ${\hat {f}^k},\hat {g}_{1}^{k},...,\hat {g}_{t}^{k}$, for objective and constraints by using the remaining individuals in $M{X_k}$.
12.	While$FEs \leqslant totalFEs$
13.	Initialize the population $TX$.
14.	For each ${\text{t}}{{\text{x}}_i}$in $TX$
15.	Calculate the minimum distance $Dist\left( {{\text{t}}{{\text{x}}_i}} \right)=\mathop {\hbox{min} }\limits_{{{\text{t}}{{\text{x}}_i} \in TX}} \left\| {{\text{t}}{{\text{x}}_i} - DB} \right\|$.
16.	Predict the objective and constraints for ${\text{t}}{{\text{x}}_i}$ by local RBF models.
17.	Evaluate the distance constraint for ${\text{t}}{{\text{x}}_i}$, i.e., ${g_{dis}}=Dis\_eps-Dist({\text{t}}{{\text{x}}_i})$.
18.	End For
19.	If all the individuals are infeasible and their constraint violations show extremely small differences
20.	Regenerate the population such as $TX$, and predict them by local RBF models.
21.	End If
22.	For each ${\text{t}}{{\text{x}}_i}$ in $TX$
23.	If$rand(0,1) \leqslant 1/3$
24.	${\text{t}}{{\text{v}}_i}={\text{t}}{{\text{x}}_i}+F({\text{t}}{{\text{x}}_{best}} - {\text{t}}{{\text{x}}_i})+F({\text{t}}{{\text{x}}_{r1}} - {\text{t}}{{\text{x}}_{r2}})$, ${\text{t}}{{\text{x}}_{r1}}$ and ${\text{t}}{{\text{x}}_{r2}}$ are two distinct individuals selected from $TX$, and $F=0.6$.
25.	Else If$rand(0,1)>2/3$
26.	${\text{t}}{{\text{v}}_i}={\text{t}}{{\text{x}}_i}+F({\text{t}}{{\text{x}}_{r1}} - {\text{t}}{{\text{x}}_i})$, ${\text{t}}{{\text{x}}_{r1}}$ is a distinct individual selected from $TX$, and $F=0.8$.
27.	Else
28.	${\text{t}}{{\text{v}}_i}={\text{t}}{{\text{x}}_i}+F({\text{t}}{{\text{x}}_{r1}} - {\text{t}}{{\text{x}}_2})$, ${\text{t}}{{\text{x}}_{r1}}$ and ${\text{t}}{{\text{x}}_{r2}}$ are two distinct individuals selected from $TX$, and $F=1.0$.
29.	End If
30.	Obtain ${\text{u}}{{\text{v}}_i}$ by conducting binomial crossover on ${\text{t}}{{\text{v}}_i}$ and ${\text{t}}{{\text{x}}_i}$.
31.	If${\text{u}}{{\text{v}}_i}$ is better than ${\text{t}}{{\text{x}}_i}$ based on FR.
32.	${\text{t}}{{\text{x}}_i}={\text{u}}{{\text{v}}_i}$.
33.	End If
34.	End For
35.	End While
36.	Store the best one ${\operatorname{lv} _k}$ from $TX$, and archive ${\text{l}}{{\text{v}}_k}$ into $LV=\left\{ {{\text{l}}{{\text{v}}_1},...,{\text{l}}{{\text{v}}_{np}}} \right\}$.
37.	End For

Experimental studies

Experimental settings

The performance of the proposed algorithm SGDLCO is verified by using thirteen widely used inequality-constrained problems from CEC2006, as well as 10-dimensional and 30-dimensional inequality-constrained problems from CEC2010 and CEC2017. The main characteristics of these thirteen benchmark problems from CEC2006 are listed in Table 1. And the main characteristics of other test problems from CEC2017 are listed in Table A1 in the Appendix. The maximum number of fitness evaluations $MaxFEs$ is set to 1000 for all the test problems. All experimental results are obtained over 25 independent runs in Matlab R2023a. To systematically evaluate and compare the performance of SGDLCO with other different algorithms in solving specific optimization problems, the effective rate (ER) metric shown in Eq. (19) is used to measure the ratio of effective evaluations to total evaluations. If at least one feasible solution is found during one evaluation, then that evaluation is considered effective. In Eq. (20), ${N_{effective}}$ and ${N_{\hbox{max} }}$ respectively represent the number of effective evaluations and the total number of evaluations. Additionally, in the following table. “Mean” and “Std” respectively represent the average and standard deviation of the objective function values obtained among all effective runs.

$$ER=\frac{{{N_{effective}}}}{{{N_{\hbox{max} }}}}$$

(20)

Table 1 The main characteristics of thirteen benchmark problems from CEC2006.

Comparison with four evolutionary algorithms

Four excellent evolutionary algorithms, i.e., C²oDE⁶⁷, CORCO⁶⁸and FROFI⁶⁹, DeCODE⁷⁰ are selected for comparison with SGDLCO. C²oDE utilizes three different trial vector generation strategies to balance diversity and convergence, while achieving a balance between constraints and the objective function. It performs better or at least comparably to other state-of-the-art methods on multiple benchmark test functions. CORCO proposes a new constrained optimization evolutionary algorithm which utilizes the correlation between constraints and the objective function for the first time, and it balances this correlation through a correlation index. FROFI introduces a novel replacement mechanism and mutation strategy, and effectively utilizes objective function information to alleviate the excessive bias of known feasibility rules, thereby enhancing its robustness. DeCODE utilizes the decomposition-based multi-objective optimization method to solve constrained optimization problems by transforming them into bi-objective optimization problems and decomposing them into scalar optimization subproblems.

The experimental results (objective function values) of all these algorithms on the 13 test problems from CEC2006 are listed in Table 2. The effective rate of all these algorithms on the 13 test problems from CEC2006 are listed in Table 3. Tables 4 and 5 list the test results of each algorithm on six 10-dimensional and 30-dimensional problems in CEC2010, respectively, while Tables 6 and 7 list the test results of each algorithm on seven 10-dimensional and five 30-dimensional problems in CEC2017, respectively. In these tables, t-test are employed, as shown in Table 3, where “Win” indicates the number of test problems where SGDLCO outperforms the algorithm, “Tie” indicates the number of test problems where SGDLCO performs comparably to the algorithm, and “Loss” indicates the number of test problems where SGDLCO performs worse than the algorithm.

In Table 2, on nine test problems (g1, g2, g4, g6, g7, g10, g16, g18, g19, g24), SGDLCO is able to achieve solutions that are significantly better than those of the other four algorithms. This indicates that within 1000 evaluation attempts, SGDLCO is able to find optimal solutions that are significantly better than those of the other four algorithms. In Table 3, for eight test problems (i.e., g01, g07, g10, g16, g18) with smaller feasibility ratio, SGDLCO is able to achieve a 100% ER on these test problems. This means that, compared to these four algorithms, SGDLCO is always able to find at least one feasible solution when dealing with these test problems.

As shown in Table 4, SGDLCO performs excellently on the majority of problems, outperforming other algorithms on the C01, C07, C08, C13, and C14 problems. As shown in Tables 5 and 7, for C13 problem with a low feasibility rate, SGDLCO consistently finds feasible solutions in every run, whereas other algorithms fail to obtain feasible solutions. As shown in Table 6, SGDLCO achieved better or comparable results than other algorithms on all problems. As shown in Table 8, SGDLCO performs excellently on the C01, C02, C04, C05, C13, and C22 problems, outperforming other algorithms on these problems. As shown in Table 9, for C13 and C22 problems with low feasibility rates, SGDLCO consistently finds feasible solutions in every run, whereas other algorithms fail to obtain a feasible solution. Similar results can also be seen in Tables 10 and 11. Therefore, SGDLCO significantly outperforms the other four algorithms across various test problems, demonstrating its superiority and stability in handling inequality-constrained optimization problems.

Table 2 Experimental function values among SGDLCO, C²ODE, CORCO, FROFI, and DeCODE on thirteen test problems from CEC2006.

Table 3 Effective rates among SGDLCO, C²ODE, CORCO, FROFI, and DeCODE on thirteen test problems from CEC2006.

Table 4 Experimental function values among SGDLCO, C²ODE, CORCO, FROFI, and DeCODE on six 10-D test problems from CEC2010.

Table 5 Effective rates among SGDLCO, C²ODE, CORCO, FROFI, and DeCODE on six 10-D test problems from CEC2010.

Table 6 Experimental function values among SGDLCO, C²ODE, CORCO, FROFI, and DeCODE on six 30-D test problems from CEC2010.

Table 7 Effective rates among SGDLCO, C²ODE, CORCO, FROFI, and DeCODE on six 30-D test problems from CEC2010.

Table 8 Experimental function values among SGDLCO, C²ODE, CORCO, FROFI, and DeCODE on seven 10-D test problems from CEC2017.

Table 9 Effective rates among SGDLCO, C²ODE, CORCO, FROFI, and DeCODE on six 10-D test problems from CEC2017.

Table 10 Experimental function values between SGDLCO, C²ODE, CORCO, FROFI, and DeCODE on five 30-D test problems from CEC2017.

Table 11 Effective rates among SGDLCO, C²ODE, CORCO, FROFI, and DeCODE on five 30-D test problems from CEC2017.

Comparison with four SAEAs

Many SAEAs are developed to solve expensive constrained optimization problems. Seven out of these algorithms, i.e., GLoSADE⁵, SADE-CV_SR³⁷, SADE-CV_FR³⁷, SParEA⁷¹, MPMLS³⁵, SA-TSDE⁷²and SACCDE⁴⁰provide competitive performance than others on solving classical test suites. Therefore, they are employed to compare with SGDLCO on three widely used test suites, i.e., IEEE CEC2006⁶¹, IEEE CEC2010⁶²and IEEE CEC2017⁶³. To be consistent with the three comparison methods, we suppose that both the objective and constraints are simultaneously evaluated by running one expensive simulation. The comparison results of these algorithms on benchmark problems from IEEE CEC2006, IEEE CEC2010 and IEEE CEC2017 are respectively shown in Tables 12, 13 and 14.

In Table 12, SGDLCO is capable of obtaining near-optimal solutions for the majority of relatively simple problems in IEEE CEC2006 and outperforms these comparison algorithms on most functions. This indicates that SGDLCO has the ability to quickly locate the global optimum for simple problems. Furthermore, in Table 13, for relatively complex problems such as those in IEEE CEC2010, SGDLCO also significantly outperforms these comparison algorithms on most functions, which indicates that SGDLCO has a significant performance advantage on CEC2010 test suites. In Table 14, SGDLCO has a significant advantage over only four of the seven algorithms such as GLoSADE, SADE-CV_SR, SADE-CV_FR, and MPMLS, and SGDLCO has comparable performance with the other three algorithms. This indicates that the performance of SGDLCO varies across different test suites and the performance of SGDLCO on CEC2017 does not seem to be significant. In order to analyze the overall performance of the SGDLCO algorithm on the three test suites more clearly, the statistical results of the overall performance of the algorithm against the three test sets are presented in Sect.4.4. In addition, the detailed comparative analysis results with each algorithm are as follows:

• (1) For GLoSADE, in CEC2006, SGDLCO achieves better results on seven out of thirteen problems. In CEC2010, SGDLCO outperforms GLoSADE on six problems and achieves comparable results on two problems. Additionally, in CEC 2017, SGDLCO achieves better results on ten out of twelve problems.

• (2) For SADE-CV_SR and SADE-CV_FR, in CEC2006, SGDLCO achieves better results on eleven out of thirteen problems. In CEC2010, SGDLCO outperforms both SADE-CV_SR and SADE-CV_FR on ten problems. Additionally, in CEC 2017, SGDLCO achieves better results on eleven out of twelve problems compared to both algorithms.

• (3) For SParEA, in CEC2006, SGDLCO achieves better results on six out of thirteen problems and had similar results on five problems. In CEC2010, SGDLCO outperforms SParEA on six problems. Additionally, in CEC 2017, SGDLCO achieves better results on three out of twelve problems and had similar results on seven problems with SParEA. It is worth noting that while the t-test results were similar on some problems, such as C22 in Table 15 and C04 in the six row of Table 14, SGDLCO consistently obtained significantly better average values for these specific problems compared to SParEA.

• (4) For MPMLS, in CEC2006, SGDLCO achieves better results on seven out of thirteen problems. In CEC2010, the performance of SGDLCO is significantly better than MPMLS on five problems, and SGDLCO obtains comparable results than MPMLS on five problems. In CEC2017, SGDLCO outperforms MPMLS on six problems, and SGDLCO obtains comparable results than MPMLS on four problems.

• (5) For SA-TSDE, in CEC2006, SGDLCO achieves better results on eight out of thirteen problems and obtains similar results on four problems. In CEC2010, the performance of SGDLCO is significantly better than SA-TSDE on six problems and SGDLCO obtains comparable results than SA-TSDE on four problems. In the twelve problems tested in CEC2017, SGDLCO outperforms SA-TSDE on four problems, and SGDLCO obtains comparable results on another four.

• (6) For SACCDE, in CEC2006, SGDLCO achieves better results on five out of thirteen problems, and SGDLCO has similar results than SACCDE on five problems. In the twelve problems tested in CEC2010, SGDLCO achieves better results on six problems, and SGDLCO obtains comparable results on six problems. In CEC2017, SGDLCO outperforms SACCDE on three problems, and SGDLCO obtains comparable results than SACCDE on six problems. This indicates that in the three test suites, SGDLCO outperforms or performs comparably to SACCDE in most problems.

Table 12 Comparison results (function values) among the compared eight algorithms on test problems from IEEE CEC 2006.

Table 13 Comparison results (function values) among the compared eight algorithms on six 10-D and 30-D test problems from IEEE CEC 2010.

Table 14 Comparison results (function values) among the compared eight algorithms on seven 10-D and five 30-D test problems from IEEE CEC 2017.

Wilcoxon signed rank test for comparison

To further compare the overall performance of SGDLCO with these classical algorithms on the aforementioned test problems, we employed the Wilcoxon signed rank test⁷³ to comprehensively assess their performance differences. All Wilcoxon signed rank test results are listed in Table 15. Here, different dimensions of the same function (e.g., 10D C01 and 30D C01) are considered as distinct problems.

In Table 15, it can be observed that SGDLCO shows a significant improvement over SADE-CV_SR, SADE-CV_FR with a level of significance level $\alpha =0.01$, over GLoSADE with $\alpha =0.02$, over SA-TSDE with $\alpha =0.{\text{0}}5$, over SACCDE with $\alpha =0.{\text{10}}$, over SParEA with $\alpha =0.20$, and over MPMLS with $\alpha =0.25$.

Table 15 The Wilcoxon signed rank test results for mean value comparison.

Effectiveness of some strategies in SGDLCO

In this subsection, five 10-D problems in CEC2017 are chosen to discuss the effectiveness of several strategies proposed in this paper. The maximum number of fitness evaluations $MaxFEs$ is set to 1000, and the rest parameters about SGDLCO are set the same as that suggested in subsection 4.1.

(1) Effectiveness of classification cooperation mutation operation

To verify the effectiveness of the classification cooperation mutation operation (CCMO) proposed in SGDLCO, a variant called SGDLCO_noCCMO is introduced for detailed comparison. SGDLCO_noCCMO replaces the classification cooperation mutation operation (CCMO) in SGDLCO with the DE/best/2 method as shown in Eq⁸.

Figure 7 lists the comparison results of SGDLCO and SGDLCO_noCCMO on the five 10-D test problems from CEC2017. From Fig. 7, it can be seen that SGDLCO achieved better or comparable results to SGDLCO_noCCMO except for C05. This demonstrates that CCMO effectively utilizes information from all individuals to obtain more accurate solutions. Figure 8 lists the standard deviations of the operational results of SGDLCO and SGDLCO_noCCMO on the five 10-dimensional test problems from CEC2017. From the table, we can see that SGDLCO exhibits more stable results than SGDLCO_noCCMO on most problems.

Fig. 7

Comparison results (mean values) among SGDLCO and SGDLCO-noCCMO on five 10-D test problems from IEEE CEC 2017.

Fig. 8

Comparison results (std values) among SGDLCO and SGDLCO-noCCMO on five 10-D test problems from IEEE CEC 2017.

(2) Effectiveness of three-layer adaptive screening

To maintain population diversity during the selection process and avoid the population getting trapped in local optima, this study constructs a variant of SGDLCO called SGDLCO_noTLAS. SGDLCO_noTLAS replaces TLAS in SGDLCO with FR.

Figure 9 presents the experimental results of SGDLCO and SGDLCO_noTLAS on five selected problems from CEC2017. From Fig. 9, we can see that SGDLCO achieves better results than SGDLCO_noTLAS. These experimental results indicate that TLAS effectively maintains the diversity of the population, enabling effective exploration of the solution space and enhancing the quality of individuals within the population. Therefore, TLAS significantly increases the chances of finding the global optimum in the solution space, rather than getting stuck in local optima. Figure 10 lists the standard deviations of the operational results of SGDLCO and SGDLCO_noTLAS on the five 10-dimensional test problems from CEC2017. From the table, we can see that SGDLCO exhibits more stable results than SGDLCO_noTLAS in 3 out of the 5 problems.

Fig. 9

Comparison results (mean values) among SGDLCO and SGDLCO-noTLAS on five 10-D test problems from IEEE CEC 2017.

Fig. 10

Comparison results (std values) among SGDLCO and SGDLCO-noTLAS on five 10-D test problems from IEEE CEC 2017.

(3) Effectiveness of distributed central region local exploration

To verify the effectiveness of the DCRLE proposed in this paper, a variant of SGDLCO, called SGDLCO_noDCRLE, was introduced. SGDLCO_noDCRLE replaces the DCRLE in SGDLCO with the local surrogate-assisted search phase detailed introduced in GLoSADE. The core difference between the local surrogate-assisted search phase in GLoSADE and the DCRLE is that the first approach performs complete local search processes for each population individual, while the DCRLE only performs local searches for selected individuals with well-distribution and potentiality.

Figure 11 lists the test results of SGDLCO and SGDLCO_noDCRLE on five problems from CEC2017. It can be seen that SGDLCO achieved better results on 4 out of 5 tested problems, with C20 achieving comparable results between the two. And from Fig. 12 we can observe that SGDLCO achieves more stable results than SGDLCO-noDCRLE on all problems except C20. This indicates that SGDLCO has a stronger search capability compared to SGDLCO_noDCRLE. This suggests that DCRLE can effectively allocate uniformly distributed local searches, concentrating the search range on a smaller, more promising area, thereby making more efficient use of limited computational resources.

Fig. 11

Comparison results (mean values) among SGDLCO and SGDLCO-noDCRLE on five 10-D test problems from IEEE CEC 2017.

Fig. 12

Comparison results (std values) among SGDLCO and SGDLCO-noDCRLE on five 10-D test problems from IEEE CEC 2017.

Conclusion

This paper introduces a surrogate-assisted global and distributed local collaborative optimization algorithm for expensive constrained optimization problems, named SGDLCO. The algorithm operates in two phases during each iteration: global surrogate-assisted collaborative evolution and distributed local surrogate-assisted search. In the global surrogate-assisted collaborative evolution phase, to fully utilize the positive guidance information from better solutions and the negative guidance information from poorer solutions, and to alleviate the pre-screening pressure on the surrogate model, the global candidate set is generated through classification cooperative mutation operation. In the distributed local surrogate-assisted phase, to effectively allocate uniformly distributed local searches, this paper designs a distributed central region local exploration method that uses affinity propagation clustering and mathematical modeling to identify uniformly distributed central solutions. For different clustering situations, targeted mathematical modeling methods are used to reselect central points for well-distributed local searches. Additionally, distance constraints are considered during the local search process to maintain search diversity and prevent the algorithm from getting trapped in local optima. To alleviate the selection pressure on the population, this paper designs a three-layer adaptive selection strategy. Two diversity-based fitness functions are formulated to screen candidate solutions from global or local candidate sets. This approach ensures the quality of the population while generating a more exploratory population, thereby significantly increasing the probability of TLAS finding the global optimum in the solution space and avoiding local optima. Compared to other algorithms, SGDLCO demonstrates significant advantages in handling expensive constraint optimization problems. It reduces the blind exploration in the global search stage and effectively avoids excessive redundancy in the local search. This significantly increases the likelihood of finding the global optimum and prevents the algorithm from getting trapped in local optima.

Furthermore, the design philosophy and methodology of SGDLCO provide a solid foundation for future algorithm extensions and improvements, particularly in addressing multi-objective and high-dimensional problems, where it is expected to achieve superior performance. Therefore, how to effectively extend the current proposed SGDLCO for solving these problems is an important future work.