Introduction

China’s cold regions are extensive, covering approximately 75% of the country’s total land area, primarily across the western and northern territories1,2. In these regions, hazardous rock mass consist of fractured rock bodies and associated structural systems. Typically, intersected by multiple sets of discontinuities and situated on steep slopes, they often exist in a state of limit equilibrium or instability under external loads3. The stability of these rock mass has attracted increasing attention because of the severe impact of environmental factors—specifically frost heave and freeze-thaw cycles—which significantly increase the risk of geological hazards such as collapses and landslides. Consequently, accurately predicting the long-term stability and evolution of hazardous rock mass is critical for ensuring the safety and functionality of engineering structures on cold-region slopes. While existing research has yielded valuable insights into deformation mechanisms and stability assessment through theoretical analysis, field monitoring, physical model testing, and numerical simulation4,5,6, these conventional methods face limitations. These methods often rely on idealized boundary conditions and require highly precise physical and mechanical rock mass parameters. Moreover, physical tests are constrained by high costs and limited reproducibility. As a result, substantial discrepancies often arise between predicted outcomes and actual monitoring data, particularly within the complex, dynamic geological environments of cold regions7,8.

To overcome the limitations of traditional computational methods for analyzing the stability of hazardous rock mass in cold regions, neural network approaches have been increasingly adopted in recent years. Characterized by strong nonlinear mapping capabilities and fault tolerance, neural networks are well suited for addressing the high uncertainty, nonlinearity, and spatiotemporal variability inherent in rock deformation prediction9,10.

Among these, Backpropagation (BP) neural networks are widely used in slope stability prediction because of their superior nonlinear fitting performance11,12. Unlike traditional methods that rely on sliding surface identification, mechanical modeling, or numerical simulation, BP networks efficiently evaluate slope stability by learning complex nonlinear relationships between geological parameters (e.g., slope height, angle, lithology, pore pressure, and rainfall) and the stability coefficient derived from historical or experimental data. However, BP network performance is highly sensitive to initial weights and structure, rendering them prone to local optima and limiting their effectiveness with dynamic time series data. To address time-varying features in slope stability evolution—such as rainfall-induced pore pressure changes and displacement—the Long Short-Term Memory (LSTM) model employs gating mechanisms to effectively capture temporal trends in monitoring data13,14. This capability enables dynamic prediction of future safety factors or failure probabilities, facilitating the construction of sliding time window models that support continuous early warning systems. Nevertheless, LSTM networks involve complex structures, require computationally intensive training, and demand high data completeness and quality. In contrast, Generalized Regression Neural Networks (GRNN)—feedforward networks based on radial basis functions, offer advantages such as noniterative training and rapid computation. These features make them suitable for small-sample datasets and slope monitoring scenarios involving high-dimensional variables15,16. Despite these benefits, GRNNs are highly sensitive to smoothing factors, possess limited generalizability, and are vulnerable to outliers, all of which compromise prediction robustness. Genetic Algorithms (GA) are frequently integrated with BP, GRNN, or limit equilibrium methods to optimize network parameters or slide surface search paths. The strong global optimization capabilities of these methods enhance both the prediction accuracy and computational efficiency. However, GA performance depends heavily on fitness function design and often suffers from low search efficiency and premature convergence17,18. These findings suggest that while integrating neural networks with intelligent optimization algorithms has great potential for slope stability prediction, balancing accuracy, generalization, and interpretability remains a critical challenge under specific engineering conditions.

Although single neural networks have demonstrated utility in predicting slope stability, their accuracy remains constrained. They are susceptible to local optima, sensitive to initialization parameters, and struggle to accommodate complex geological conditions and highly uncertain monitoring data—factors critical to engineering applications. Consequently, integrating neural networks with intelligent optimization algorithms (e.g., the GA, PSO, and SSA) has become a prevalent strategy to increase the convergence speed and global search capabilities. However, most existing hybrid models address general slope conditions, leaving their adaptability to the specific “freeze‒thaw‒gravity” coupling mechanism in cold regions insufficiently discussed. The complex interactions among freezing depth, temperature fluctuations, and rock parameters create a unique solution space that generic models often fail to capture19,20,21,22,23. Therefore, the core contribution of this work lies in comparatively evaluating different optimization strategies applied to hazardous rock mass under freeze-thaw conditions; it involves evaluating and identifying a suitable algorithmic strategy applied to the environmental variables affecting hazardous rock mass in Northwest China24,25.

This study analyzes typical engineering cases of hazardous rock mass in the cold regions of Northwest China, identifies six key parameters that significantly influence stability, and constructs both standalone and hybrid intelligent models for comparative analysis. The predictive performance of these models is systematically evaluated in terms of computational accuracy, robustness, and adaptability. Furthermore, the advantages and limitations of each model under various application scenarios are elucidated, providing a reference for stability assessment and early warning systems in cold regions.

Cases and analysis of hazardous rock mass in cold regions

Key parameters and normalization processing

Compared with hazardous rock mass in temperate regions, those in cold regions are subject to intense freeze-thaw cycles. Pore water within a rock structure undergoes repeated phase transitions due to periodic temperature fluctuations. The frost heaving forces generated during freezing drive crack propagation along penetrating discontinuities and accelerate the mechanical deterioration of intact rock. During the thawing phase, meltwater infiltrates deeper into the rock mass, increasing the internal moisture content and intensifying frost damage in subsequent cycles. This freeze-thaw coupling mechanism induces long-term, cumulative degradation of stability and serves as a primary driver for the progressive failure of rock mass in cold regions. The parameter selection for this study adhered to three fundamental principles. First, priority was given to indicators highly sensitive to stability changes to ensure the timely detection of potential instability. Second, parameters are required to exhibit strong physical independence to minimize information redundancy and coupling interference. Finally, data accessibility was ensured to facilitate efficient, cost-effective acquisition during practical monitoring. Based on these principles and prior research26,27, six key parameters were selected to construct the stability prediction model: cohesion (c), freezing depth (H), lowest temperature (T), freezing load (P), sunshine duration (S), and foot of slope displacement (D).

The physical mechanisms underlying these parameters within the coupling model are defined as follows: (a) Sunshine duration (S) is a thermal driver: S is not merely a meteorological index but a critical factor determining the thermal boundary conditions of the rock mass. Solar radiation creates significant temperature gradients between the rock surface and the interior. In cold regions, this thermal differential directly regulates the frequency and amplitude of effective freeze-thaw cycles (e.g., daytime thawing and nighttime freezing). Consequently, it generates thermal stress and exacerbates fatigue damage in the rock structure, serving as a quantitative environmental driver for mechanical degradation. (b) Foot of slope displacement (D) as a state indicator: Unlike external environmental loads, D functions as a “state variable” characterizing the cumulative damage and evolutionary stage of the hazardous rock mass. Rock instability is typically a progressive failure process; thus, historical displacement trends implicitly encapsulate information regarding internal crack propagation and structural deterioration that environmental factors alone cannot capture. In this study, cumulative displacement is utilized as a historical input feature to assist the neural network in learning the mapping relationship between the “deformation state” and the stability margin rather than treating it as the prediction target itself. This approach adheres to time series forecasting logic—using historical evolution to predict future stability trends—thereby ensuring causal clarity.

Taking a typical sliding hazardous rock mass in the cold region of western China as an engineering case study, field investigations reveal a widespread distribution of such masses along steep cliffs descending approximately 160 m from the mountain summit. These formations are shaped primarily by the combined effects of tectonic activity and unloading fractures. The specific rock mass under investigation is located on the lower section of the southern cliff face, with overall dimensions of approximately 3.5 m (height) × 8.5 m (width) × 1.5 m (thickness). Composed of thick-bedded sandstone, the rock mass features well-developed, highly penetrating unloading fractures that serve as the primary structural planes controlling potential sliding. Given the availability of comprehensive and continuous monitoring data for this site, six key control parameters were selected for analysis, with the corresponding data trends presented in Fig. 1.

Fig. 1
figure 1figure 1

Parameters of the hazardous rock mass as a function of the number of monitoring sessions.

Full size image

As illustrated in Fig. 1, with the accumulation of monitoring sessions (corresponding to freeze-thaw cycles), the parameters exhibit two distinct temporal behaviors. The first group—comprising cohesion, freezing depth, and foot of slope displacement—follows a monotonic trend, evolving gradually over time. The second group—which includes the lowest temperature, freezing load, and sunshine duration—exhibits irregular fluctuations characterized by high variability and the absence of a discernible trend. To eliminate bias resulting from disparate scales and value ranges and to increase the accuracy and interpretability of subsequent analyses, all the parameters were normalized. Given the distinct evolutionary characteristics of these variables, a group normalization strategy was adopted, processing trend-dominated and fluctuation-dominated parameters separately. The normalized results are presented in Fig. 2.

Fig. 2
figure 2

Parameters after normalization.

Full size image

Parameter correlation and gray relational analysis

Prior to modeling, analyzing parameter correlations is essential to prevent feature multicollinearity from compromising prediction performance. The correlation coefficient, R (X, Y), is calculated via the following equation:

$$R\left( {M,N} \right)=\frac{{{\text{Cov}}\left( {M,N} \right)}}{{\sqrt {{\text{Var}}\left[ M \right]{\text{Var}}\left[ N \right]} }}$$
(1)

where M and N are two independent features; R (M, N) is the correlation coefficient of M, N; Cov(M, N) is the covariance; and Var[M] and Var[N] are the variances of M and N, respectively.

The correlation coefficients between different parameters can be calculated via Eq. (1), and the results are shown in Table 1.

Table 1 Correlation matrix of parameters.
Full size table

The stability coefficient (i.e., the factor of safety), often referred to as the stability coefficient, is derived from limit equilibrium theory. Specifically, it is defined as the ratio of the maximum resisting force (or moment) to the driving force (or moment) acting along a potential slip surface28. To further investigate the relationships between key influencing factors and the stability of hazardous rock mass, Grey Relational Analysis (GRA) was employed to quantitatively assess the degree of correlation.

$${r_i}=\frac{1}{n}\sum\limits_{{k=1}}^{N} {\left[ {\frac{{\Delta (\hbox{min} )+\beta \Delta (\hbox{max} )}}{{{\Delta _i}(k)+\beta \Delta (\hbox{max} )}}} \right]}$$
(2)

where ri is the correlation of key factors, n is the number of factors, Δmin is the minimum difference of the second level, Δmax is the maximum difference of the second level, and β is the discrimination coefficient. The corresponding results are shown in Fig. 3.

Fig. 3
figure 3

Histogram of correlations between different parameters and stability coefficients.

Full size image

As illustrated in Fig. 3, the correlation coefficients between the input parameters and the stability coefficient all exceed 0.5, with the majority clustering at approximately 0.8. This indicates a strong statistical association between the selected parameters and the calculated stability coefficient. Notably, the coefficients for cohesion, lowest temperature, and freezing load are significantly greater than those for the other variables, suggesting that these factors play a more influential role within the studied dataset. This observation aligns with the characteristic stability evolution of hazardous rock mass subjected to freeze-thaw cycles in cold regions. Consequently, the six selected key parameters are confirmed to be representative and suitable for the predictive analysis of rock mass stability.

Hybrid intelligent algorithm model and its construction

Principle of algorithm

While deep learning models have achieved state-of-the-art performance across various domains, their success is predominantly predicated on the availability of massive datasets. In the context of hazardous rock mass in cold regions, the acquisition of large-scale, high-quality monitoring data is challenging because of harsh environmental conditions and significant monitoring costs, resulting in a distinct “small-sample” problem. Consequently, complex deep models are prone to overfitting when trained on such limited datasets. In contrast, BP neural networks—functioning as universal approximators—combined with global optimization algorithms (e.g., metaheuristics) represent a highly practical solution for geotechnical engineering. Therefore, this study employs the SSA and PSO—which represent distinct evolutionary mechanisms—to optimize a conventional BP network, aiming to identify the most robust strategy for geotechnical prediction under small-sample conditions.

The BP algorithm is a training method for multilayer feedforward neural networks that is based on gradient descent optimization and serves as a cornerstone of artificial intelligence29,30. Its fundamental principle involves iteratively adjusting network weights and biases via an error backpropagation mechanism. This process minimizes the discrepancy between the predicted outputs and target values, effectively capturing complex nonlinear mapping relationships. The BP algorithm proceeds in three primary stages: forward propagation, error calculation, and backpropagation. During forward propagation, the input data traverse successive hidden layers and undergo linear weighting and nonlinear activation to yield the model’s final prediction. The activation value of the j-th neuron in the i-th layer is expressed as Eq. (3).

$$\alpha _{j}^{l}=f\left( {\sum\limits_{i} {\omega _{{ji}}^{l}\alpha _{i}^{{l - 1}}+b_{j}^{l}} } \right)$$
(3)

where ωjil is the connection weight, bjl is the bias, and f (-) is the activation function.

On the basis of the error between the predicted output and the true label, the loss function is subsequently calculated as defined in Eq. (4).

$$E=\frac{1}{2}\sum\limits_{i} {{{\left( {{\gamma _k} - \gamma _{k}^{ * }} \right)}^2}}$$
(4)

where E is the total loss value; γk and γk* are the true and predicted values, respectively; and k is the number of neurons.

The error backpropagation process updates the parameters by propagating the gradient of the loss function with respect to the weights through each layer via the chain rule. For the jth neuron in the output layer, its error term satisfies the following equation:

$$\delta _{j}^{l}=f\left( {z_{j}^{l}} \right)\sum\limits_{i} {\omega _{{kj}}^{{l+1}}\delta _{k}^{{l+1}}}$$
(5)

where δjl is the error of the jth neuron of the hidden layer.

The Long Short-Term Memory (LSTM) network is a specialized Recurrent Neural Network (RNN) architecture designed to address long-term dependency challenges in time series modeling31,32. By incorporating a gating mechanism, LSTM effectively mitigates the vanishing and exploding gradient problems common in standard RNNs. This capability significantly enhances the retention of long-term sequential information, rendering the model particularly suitable for predicting the temporal evolution of slope stability. Structurally, an LSTM cell comprises four key components: the forget gate, input gate, output gate, and cell state. Its core mechanism relies on the selective retention or discarding of information via these gates. Specifically, at time step $t$, given the current input xt, the previous hidden state ht−1, and the previous cell state ct−1, the forget gate determines the fraction of information to be preserved from the previous state, as defined in Eq. (6).

$${f_t}=\sigma \left( {{W_f}\cdot \left[ {{h_{t - 1}},{x_t}} \right]+{b_f}} \right)$$
(6)

where σ (-) is the sigmoid activation function.

Second, the input gate controls how much of the current input is written:

$$\left\{ \begin{gathered} {i_t}=\sigma \left( {{W_i}\cdot \left[ {{h_{t - 1}},{x_t}} \right]+{b_i}} \right) \hfill \\ {c_t}=\tanh \left( {{W_c}\cdot \left[ {{h_{t - 1}},{x_t}} \right]+{b_c}} \right) \hfill \\ \end{gathered} \right.$$
(7)

where tanh (-) is the hyperbolic tangent function.

Finally, the cell state and output are updated:

$$\left\{ \begin{gathered} {c_t}={f_t} \odot {c_{t - 1}}+{i_t} \odot {{\tilde {c}}_t} \hfill \\ {h_t}={o_t} \odot {c_{t - 1}}+\tanh \left( {{c_t}} \right) \hfill \\ \end{gathered} \right.$$
(8)

where \(\odot\) is the Hadamard product of the output gate vector.

The Generalized Regression Neural Network (GRNN) is a feedforward architecture that is based on probability density estimation and serves as an extension of the Radial Basis Function (RBF) network33,34. Owing to its simple structure and rapid convergence, the GRNN obviates the need for complex iterative training. Its robustness in handling small-sample, nonlinear problems makes it particularly advantageous for engineering tasks involving multifactorial coupling, such as slope stability analysis. Structurally, the network comprises four layers: input, pattern, summation, and output. Fundamentally, it estimates the conditional probability density function from historical data to facilitate nonparametric regression, with the network output defined by Eq. (9).

$$\hat {Y}\left( X \right)=\frac{{\sum\nolimits_{{i=1}}^{n} {{Y_i}{\kern 1pt} {\kern 1pt} \cdot {\kern 1pt} {\kern 1pt} {\kern 1pt} \exp \left( { - \frac{{{{\left( {X - {X_i}} \right)}^{^{T}}}\left( {X - {X_i}} \right)}}{{2{\psi ^2}}}} \right)} }}{{\sum\nolimits_{{i=1}}^{n} {\exp \left( { - \frac{{{{\left( {X - {X_i}} \right)}^{^{T}}}\left( {X - {X_i}} \right)}}{{2{\psi ^2}}}} \right)} }}$$
(9)

where Xi is the input vector of the ith training sample, Yi is the target output value corresponding to the input vector of the ith training sample, and ψ is the smoothing factor.

The GA is a stochastic global optimization method inspired by natural selection and evolutionary genetics. By emulating the biological principle of “survival of the fittest,” the algorithm iteratively searches the solution space to approach an optimum through processes such as population initialization, fitness evaluation, selection, crossover, and mutation35,36. GA has demonstrated robust capabilities in addressing nonlinear, multiobjective, and high-dimensional problems. Consequently, it has been widely applied in geotechnical engineering, particularly in slope stability prediction, inverse analysis, and safety factor optimization. During the selection phase, the likelihood of an individual being chosen for reproduction is directly proportional to its fitness value. Denoting the selection probability of the i-th individual as Pi, this relationship is expressed as:

$${p_i}=\frac{{f\left( {{x_i}} \right)}}{{\sum\nolimits_{{j=1}}^{m} {f\left( {{x_j}} \right)} }}$$
(10)

where f (xi) is the fitness of the ith individual and where m is the total number of individuals in the population.

After the selection process is complete, the GA proceeds to the crossover phase, where the selected individuals generate new offspring through genetic recombination. Suppose that the crossover point is k and consider two parent individuals xi and xj. At crossover point k, the gene sequences of the two individuals are exchanged, resulting in the formation of new offspring, as expressed in the following equation:

$${x_{{\text{new}}}}=\left( {{x_{i1}},{x_{i2}},{x_{i3}}, \cdots ,{x_{ik}},{x_{j\left( {k+1} \right)}},{x_{j\left( {k+2} \right)}},{x_{j\left( {k+3} \right)}}, \cdots ,{x_{jn}}} \right)$$
(11)

where xnew is the offspring individual.

The GA subsequently enters the mutation phase, where specific genes within newly generated individuals are randomly altered with a low probability. This mechanism introduces genetic variation, thereby enhancing population diversity. Fundamentally, the mutation operation serves to prevent the algorithm from becoming trapped in local optima and to augment global search capabilities by injecting novel gene combinations into the solution space. The mutation process is mathematically described by Eq. (12):

$$x_{{nk}}^{\prime }={x_{nk}}+\zeta$$
(12)

where xnk is a newborn individual and ζ is a small randomized amount of variation.

Network parameter optimization strategy based on SSA/PSO

In practical slope stability analysis, standalone algorithms often suffer from limitations such as low computational efficiency, susceptibility to local optima, noise sensitivity, and complex parameter tuning. To mitigate these challenges and enhance model performance, prediction frameworks can be augmented with adaptive parameter tuning strategies and metaheuristic algorithms, notably the Sparrow Search Algorithm (SSA) and Particle Swarm Optimization (PSO)37,38,39,40. Prior research has demonstrated that hybrid models incorporating the SSA or PSO are highly effective and robust in predicting slope stability coefficients within complex environments41,42. Specifically, when coupled with predictive models, the SSA and PSO exhibit distinct advantages over standalone approaches, particularly in terms of global search capability, convergence speed, and prediction stability. While the SSA excels in global exploration, PSO is particularly effective in escaping local optima. Consequently, hybrid strategies that leverage these algorithms effectively integrate global search with local refinement. This synergy enhances accuracy and efficiency for high-dimensional, nonlinear problems, making such models well suited for slope stability prediction.

The Sparrow Search Algorithm (SSA) draws inspiration from the foraging and antipredatory behaviors of sparrows, achieving efficient global optimization through the synergistic interaction of three functional groups: discoverers, followers, and vigilantes. The algorithm’s core framework integrates global exploration, local exploitation, and adaptive regulation. Within the population, individuals with higher fitness values function as discoverers, assuming a primary leadership role. Possessing superior search capabilities, they are responsible for locating food sources and guiding the positional updates of the rest of the population. The followers adjust their trajectories on the basis of the discoverers to converge toward the optimal solution. Furthermore, the algorithm incorporates a vigilance mechanism to enhance environmental adaptability and effectively prevent premature convergence to local optima. The mathematical formulations governing these behavioral mechanisms are presented in Eqs. (13)–(15).

$$X_{{i,j}}^{{t + 1}} = \left\{ {\begin{array}{*{20}c} {X_{{ij}} \exp \left( { - \frac{i}{{a \cdot iter_{{\max }} }}} \right)} & {R_{2} < ST} \\ {X_{{ij}} + QL} & {R_{2} \ge ST} \\ \end{array} } \right.$$
(13)
$$X_{{i,j}}^{{t + 1}} = \left\{ {\begin{array}{*{20}c} {Q\exp \left( {\frac{{X_{{{\text{worst}}}} - X_{{i,j}}^{t} }}{{i^{2} }}} \right)} & {i > n/2} \\ {X_{P}^{{t + 1}} + \left| {X_{{ij}} - X_{P}^{{t + 1}} } \right|A^{ + } L} & {otherwise} \\ \end{array} } \right.$$
(14)
$$X_{{i,j}}^{{t + 1}} = \left\{ {\begin{array}{*{20}c} {X_{{{\text{best}}}}^{T} + \varphi \left| {X_{{ij}}^{t} - X_{{{\text{best}}}}^{t} } \right|} & {if\;f_{i} > f_{g} } \\ {X_{{i,j}}^{{t + 1}} + K\frac{{\left| {X_{{i,j}}^{t} - X_{{{\text{worst}}}}^{t} } \right|}}{{(f_{i} - f_{w} ) + \varepsilon }}} & {if\;f_{i} = f_{g} } \\ \end{array} } \right.$$
(15)

where itermax is the maximum number of iterations, a is a random number between [0, 1], Q is a random number obeying a normal distribution, L is a 1×d dimensional matrix, the range of R2 is [0, 1] for the warning value, and the range of ST is [0.5, 1] for the safety value. Xworst is the worst position of the sparrow in the d-dimension for the tth iteration, Xp is the optimal position of the sparrow in the d-dimension for the t + 1th iteration, A denotes a matrix of order 1 × d, and A+=AT(AAT)−1. Xbest is the current global optimal position; φ and K are step control parameters; φ is a normally distributed random number obeying a mean of 0 and variance of 1; K is [0,1] and is the direction of movement of the sparrow; ε is a very small number that prevents the denominator from being 0; and fi, fg, and fw are the current individual, optimal, and worst fitness values of the sparrow, respectively.

Particle Swarm Optimization (PSO) is a population-based metaheuristic algorithm inspired by the social foraging behavior of bird flocks or fish schools. It efficiently navigates the solution space through information sharing and synergistic interactions among particles. Each particle represents a candidate solution, and its trajectory is updated on the basis of the combined influence of its own historical best position (personal best) and the population’s global best position (global best). This mechanism achieves a dynamic balance between individual cognition and social collaboration. PSO is characterized by structural simplicity, minimal parameter requirements, and rapid convergence. During each iteration, the velocities and positions of the particles are adjusted on the basis of their current states and global information, with the swarm guided by the optimal individual, as defined in Eq. (16). To increase population diversity and prevent premature convergence to local optima, PSO incorporates a stochastic perturbation mechanism, which is functionally analogous to the “vigilantes” in the SSA. Owing to its robust global search capabilities, PSO has been extensively applied to complex multivariate optimization problems in geotechnical engineering, including parameter identification, safety factor back-analysis, and critical slip surface identification.

$$V_{i}^{{t+1}}=\mu V_{i}^{t}+{\lambda _1}{R_1}\left( {{P_i} - X_{i}^{t}} \right)+{\lambda _2}{R_2}\left( {G - X_{i}^{t}} \right)$$
(16)

where µ is the inertia weight and where λ1 and λ2 are the learning factors.

As outlined previously, traditional neural networks relying on gradient descent are highly susceptible to local optima and sensitive to parameter initialization. Therefore, this study leverages the robust global optimization capabilities of the SSA and PSO to optimize the initial weights and biases of the BP neural network. The construction of the hybrid model proceeds as follows: First, the network topology is defined as I-H-O (representing the node counts for the input, hidden, and output layers, respectively). Each individual (sparrow or particle) in the SSA or PSO algorithm encodes a vector representing a candidate set of initial weights and biases. The population is randomly initialized to generate multiple sets of potential network parameters. Second, these parameters are assigned to the BP network, and the mean squared error (MSE) of the training data is selected as the fitness function. A lower MSE indicates superior network performance and, consequently, higher fitness for the individual. In the SSA-BP model, the sparrow population is categorized into discoverers, followers, and vigilantes on the basis of fitness values. These groups execute global search and local exploration, dynamically updating their positions according to Eqs. (13)-(15). Conversely, in the PSO-BP model, particles update their velocities and positions on the basis of both personal and global historical optima, as defined in Eq. (16). Upon reaching the maximum iteration count or satisfying the error tolerance, the global optimal position is output and decoded into the optimal initial weights and biases for the BP network. Finally, the BP algorithm performs secondary training and fine-tuning to yield the final prediction model. This mechanism effectively resolves the instability caused by random initialization, achieving seamless integration of global optimization and local gradient descent.

In this study, the architecture of the BP neural network is defined by the six identified influencing factors (I = 6) and the single stability coefficient output (O = 1). Based on empirical formulas, the number of hidden layer nodes is set to 12, resulting in a 6-12-1 topology. Consequently, the dimensionality of the search space (i.e., the individual dimension in SSA or PSO) is calculated as M = 97. The specific parameter configurations for the hybrid algorithms are as follows: (1) SSA-BP Model: The population size is set to 50, with a maximum of 100 iterations. The proportion of discoverers is set to 20%, and the proportion of vigilantes (sparrows sensing danger) is also set to 20%. The safety threshold is fixed at 0.8 to balance global exploration and local exploitation. (2) PSO-BP Model: The population size and maximum number of iterations are consistent with the SSA settings. The acceleration coefficients (learning factors) are both set to 1.5, and the inertia weight decreases linearly from 0.9 to 0.4 to increase the convergence accuracy in the later stages. (3) BP neural network: The learning rate is set to 0.01, the target error is 0.001, and the activation functions for the hidden and output layers are the hyperbolic tangent sigmoid (tansig) and linear (purelin) functions, respectively.

Model test results and predictive analysis

Error evaluation and comparison of different algorithms

To rigorously evaluate the predictive performance, 12 distinct models—comprising both standalone intelligent algorithms and multiple hybrid optimization frameworks—underwent multiple rounds of iterative training and testing. This study utilized a representative case study of a slip-type hazardous rock mass in a cold region. Figures 4, 5 and 6 illustrate the comparative analysis between the model predictions and the measured field data. These visualizations serve to assess the accuracy and generalization capability of the different algorithms in predicting rock mass stability.

Fig. 4
figure 4

Comparison of the predicted and actual values of the stability coefficients of the hazardous rock mass for a single algorithmic model.

Full size image
Fig. 5
figure 5

Comparison of the predicted and actual values of the stability coefficients of the hazardous rock mass for the hybrid algorithm model including the SSA.

Full size image
Fig. 6
figure 6

Comparison of the predicted and actual values of the stability coefficients of the hazardous rock mass for the hybrid algorithm model including PSO.

Full size image

Figures 4, 5 and 6 visualize the goodness-of-fit between the predicted values of different models and the measured stability coefficients via scatter plots, where the diagonal line (y = x) represents the line of perfect concordance. Figure 4 reveals that the data points for the standalone models exhibit significant dispersion around the diagonal. The majority of sample points deviate from the ideal regression line, indicating that the models have a limited ability to explain the stability of hazardous rock mass in cold regions. The coefficients of determination (R2) are generally low (ranging from a minimum of 0.8124 to a maximum of 0.8956). This suggests that standalone models struggle to fully capture the complex nonlinear fluctuations induced by coupled freeze-thaw effects. In contrast, the predictive accuracy substantially improves following the integration of intelligent optimization algorithms. As illustrated in the relevant plots, the scatter distribution for the PSO-optimized models becomes noticeably more compact, yielding a significant increase in R2 compared with that of the standalone models. However, minor deviations persist in regions characterized by extreme values. Notably, Fig. 5 demonstrates that SSA-optimized models (particularly SSA-BP) exhibit superior convergence. The predicted points cluster tightly along the y = x line, indicating a high degree of goodness-of-fit. Quantitative analysis confirms that the SSA-BP model achieves a coefficient of determination (R2) of 0.9789, significantly surpassing both the PSO and standalone BP models. This high R2 value not only validates the SSA’s effective capability in global optimization and local minima avoidance but also indicates close numerical agreement between the predicted and observed stability coefficients for the investigated cases between geological factors (e.g., freezing depth and cohesion) and stability coefficients. Consequently, the predictive performance of the models follows the order R²(SSA) > R²(PSO) > R²(Single).

To provide a comprehensive quantitative assessment of predictive efficacy, three standard statistical metrics were employed: the mean absolute error (MAE), mean squared error (MSE), and root mean squared error (RMSE). Fundamentally, the minimization of these metrics corresponds to superior model fit and enhanced prediction accuracy. As illustrated in Fig. 7, hybrid models consistently exhibit reduced error margins across all three indicators compared with traditional standalone algorithms. This highlights the superior performance of hybrid frameworks in terms of both accuracy and stability. Notably, hybrid models incorporating the SSA yield significantly lower error values than those utilizing PSO. This finding corroborates the goodness-of-fit trends observed in Figs. 4, 5 and 6, further confirming the efficacy of the SSA’s global search capability in optimizing model performance. In contrast, standalone models (e.g., BP and GA) demonstrate elevated error metrics, reflecting their inherent limitations in handling highly nonlinear problems. Overall, hybrid models augmented by metaheuristic optimization strategies exhibit greater accuracy and robustness in predicting slope stability in cold regions, rendering them highly suitable for parameter modeling and safety assessment in complex environments.

Fig. 7
figure 7

Evaluation metrics for different algorithmic models.

Full size image

The superior performance of the SSA-BP model relative to that of PSO-BP is attributable to fundamental differences in their evolutionary mechanisms. In standard PSO, particles exhibit a strong tendency to converge rapidly toward the current global best position. While efficient, this aggressive convergence often leads to premature convergence and stagnation in local optima, particularly within the high-dimensional, multimodal search spaces characteristic of rock mass stability problems. In contrast, the SSA introduces a dynamic vigilante mechanism (typically accounting for 10%-20% of the population). These individuals possess the capacity to relocate stochastically on the basis of a safety threshold. Specifically, upon detecting a “danger signal,” sparrows at the population’s periphery migrate rapidly toward safer zones (approaching the current best position), whereas those in the central region execute random walks to disrupt potential stagnation. This dynamic behavior significantly bolsters population diversity and the ability to escape local optima, particularly during the later stages of iteration. Consequently, the SSA is more effective in identifying superior initial weights and biases for the BP neural network.

Prediction and impact analysis of the stability coefficient

Building upon the systematic error analysis presented in Fig. 8, the reliability of the hybrid models in predicting the stability coefficients of hazardous rock mass in cold regions has been empirically validated. To assess generalizability further, the optimal hybrid framework was subsequently applied to a secondary case study involving a typical hazardous rock mass within the same engineering region. The resulting predictions were benchmarked against reported literature values, as illustrated in Figs. 8, 9 and 10, thereby rigorously verifying the model’s robustness and accuracy.

Fig. 8
figure 8

Prediction of the stability coefficients of hazardous rock mass for single algorithm modeling.

Full size image
Fig. 9
figure 9

Prediction of the stability coefficients of hazardous rock mass with the hybrid algorithm model including the SSA.

Full size image
Fig. 10
figure 10

Prediction of the stability coefficients of hazardous rock mass with the hybrid algorithm model including PSO.

Full size image

The comparative analysis presented in Figs. 8, 9 and 10 reveals distinct performance hierarchies among the evaluated models. Consistently, hybrid intelligent algorithms outperform standalone models in terms of prediction accuracy. As visualized in Fig. 9, the predictions generated by the standalone models deviate markedly from the literature benchmarks. This discrepancy is attributed primarily to inherent limitations in nonlinear modeling, the inability to identify multifactorial coupling mechanisms, and deficiencies in global parameter optimization. Furthermore, standalone models exhibit heightened sensitivity to hyperparameter selection and are prone to overfitting or underfitting during training, which limits their generalizability to unseen data. Crucially, these models fail to adequately capture the complex interactions between internal rock mass structures and external disturbance factors. Consequently, the resulting predictions diverge from actual stability characteristics, yielding significant discrepancies when compared against established literature values.

In contrast, hybrid models incorporating evolutionary optimization mechanisms significantly enhance global search capabilities and dynamic adaptability. This effectively overcomes the limitations inherent to standalone models in addressing high-dimensional, nonlinear problems. As illustrated in Fig. 10, the SSA-integrated model demonstrates enhanced stability and consistency when processing multiconstraint, high-dimensional, and complex data. This performance is attributed to the SSA’s robust global optimization and adaptive weight adjustment strategies. Consequently, the alignment with literature benchmarks is notably improved, highlighting the model’s efficacy in capturing stability variation patterns within complex geological settings. Conversely, as shown in Fig. 11, the PSO-integrated model exhibits commendable computational efficiency while maintaining high prediction accuracy, making it particularly suitable for scenarios demanding rapid response and real-time decision-making. However, pronounced fluctuations at specific data points suggest a susceptibility to premature convergence (stagnation in local optima) during early stages, insufficient particle diversity, or suboptimal parameterization. In summary, the SSA-based model is preferable for high-precision, long-term stability assessment, whereas the PSO-based model excels in rapid evaluation and engineering decision support. Future research should investigate synergistic nested optimization mechanisms that combine the SSA and PSO, integrated with adaptive parameter tuning and dynamic feedback strategies. Such advancements aim to further enhance model generalizability and stability, driving the field of hazardous rock mass assessment in cold regions toward greater accuracy and efficiency.

An examination of Figs. 2 and 3 reveals that three critical parameters—lowest temperature, freeze load, and sunshine duration—exhibit significant temporal volatility and lack distinct cyclical patterns. Consequently, these parameters were selected to simulate stability under extreme scenarios (representing the most favorable and unfavorable combinations). These predictions were generated via the optimal hybrid framework identified in the previous analysis, with the results depicted in Fig. 11.

Fig. 11
figure 11

Prediction results with different parameters.

Full size image

According to the analysis of Fig. 11, the stability coefficient of the hazardous rock mass progressively decreases over the monitoring period (rather than with the “monitoring frequency”). This indicates a continuous degradation in stability driven by the combined effects of freeze-thaw cycles and environmental disturbances. Among the influencing factors, the freeze load has the most profound impact. In the worst-case scenarios (most unfavorable conditions), the sharpest reduction in the stability coefficient is induced, establishing it as the dominant driver of stability deterioration. Conversely, the lowest temperature and sunshine duration exert moderate influences. While both contribute to stability loss under adverse conditions, the impact of the lowest temperature is marginally stronger than that of sunshine duration. Notably, under best-case scenarios (optimal conditions), the degradative impact of these factors is significantly attenuated. This suggests that a rock mass retains a degree of resilience when subjected to favorable climatic and loading environments. Consequently, the results identify the freeze load as the most influential factor within the investigated cases. Therefore, future monitoring and engineering interventions should prioritize the tracking of freeze load trends. Simultaneously, multifactorial synergistic strategies—incorporating temperature and sunshine duration—should be implemented to increase prediction accuracy and ensure slope safety in cold regions.

Crucially, the stability of hazardous rock mass results from the complex interplay of multiple coupled factors. For example, “Lowest temperature” directly dictates the “Freezing Depth,” whereas “Sunshine Duration” modulates the freeze-thaw cycles that progressively degrade “Bond Strength.” Although the sensitivity analysis in the previous section isolated single factors to observe individual trends, the proposed SSA-BP model inherently captures these complex nonlinear interactions through the connection weights and activation functions of its hidden layers. Consequently, the “Worst/Best Combination” analysis presented in Fig. 11 does not merely represent a linear summation of individual effects; rather, it demonstrates the model’s robust predictive capability under extreme coupled scenarios. This confirms that the model has successfully learned the synergistic mechanisms governing the six input features.

Finally, on the basis of the trade-off between computational cost and prediction accuracy, this study proposes a differentiated application strategy for hazardous rock mass engineering in cold regions: (a) For large-scale geological surveys where computing resources are constrained (e.g., via portable inspection devices), the PSO-BP model is recommended. Its rapid convergence and lower computational overhead allow engineers to quickly identify potential high-risk points within massive datasets, satisfying the demand for time-sensitive preliminary assessments. (b) For specific rock mass identified as high risk, the SSA-BP model is the optimal choice for long-term early warning systems. In this scenario, the primary constraint is reliability rather than real-time calculation speed (given the slow kinetics of the freeze-thaw process). The SSA-BP model’s superior ability to escape local optima minimizes prediction error, effectively mitigating the risk of safety incidents caused by model underestimation, thus meeting the stringent requirements for fine-grained, long-term stability management.

Limitations

First, the data used in this study rely on specific hazardous rock mass in the cold regions of Northwest China, which are predominantly composed of sandstone. Consequently, the applicability of the proposed model to other geological conditions or rock types requires further validation.

Second, although the hybrid algorithm mitigates the issues associated with small datasets, the total number of monitoring samples remains limited. The generalization ability of the model could be further improved with larger-scale, long-term monitoring datasets.

Finally, the model currently considers six primary influencing factors. Other potential environmental triggers, such as sudden seismic activity or extreme fluctuations in groundwater levels, were not included in this framework and represent a direction for future research. The comparative superiority of different hybrid models should be interpreted as case-dependent rather than universally optimal.

Conclusions

  1. (1)

    GRA reveals that the freeze load, lowest temperature, and rock cohesion strength exhibit the strongest correlations with the stability coefficient, identifying them as the critical determinants of stability. Mechanistically, cyclic variations in the freeze load induce substantial stress fluctuations along structural planes, triggering localized displacement and structural fatigue. Concurrently, low-temperature conditions exacerbate freeze-thaw deterioration, progressively degrading the mechanical properties of rock. Consequently, the synergistic effect of these factors drives a sustained decline in stability. Quantitative sensitivity analysis further demonstrated that under the most unfavorable freeze load conditions, the stability coefficient decreases by up to 25%. This magnitude significantly exceeds the impact of other variables, empirically indicating that the freeze load plays a dominant role in the studied hazardous rock mass.

  2. (2)

    Comparative analysis demonstrates that hybrid models that integrate swarm intelligence strategies significantly outperform standalone algorithms in predicting hazardous rock mass stability. A quantitative evaluation via the MAE, MSE, and RMSE indicates that hybrid models consistently yield minimized error margins across diverse test scenarios. By systematically optimizing weight initialization and parameter search trajectories, mechanisms such as the SSA and PSO effectively mitigate the inherent limitations of standalone algorithms, specifically their sensitivity to initial conditions and susceptibility to stagnation in local optima. Consequently, hybrid frameworks demonstrate a superior capacity to capture the nonlinear coupling among key environmental factors, significantly enhancing modeling robustness for complex engineering datasets.

  3. (3)

    Comparative analysis reveals distinct optimal use cases for each hybrid framework. The SSA-integrated model demonstrates superior efficacy in high-precision, long-term stability prediction. Its adaptive weight adjustment mechanism effectively resolves high-dimensional and strongly nonlinear data structures, achieving a goodness-of-fit (R2) exceeding 0.90—a performance that is consistently higher than that of the compared models under identical data conditions. Conversely, the PSO-integrated model is better suited for real-time monitoring and decision support scenarios where computational speed is paramount. While PSO entails a marginal risk of premature convergence (local optima entrapment), it offers a computational efficiency approximately 40% higher than that of traditional BP models while still achieving a notable reduction in prediction error across most test datasets.