Developing a novel hybrid model based on GRU deep neural network and Whale optimization algorithm for precise forecasting of river’s streamflow

Abstract

Streamflow contemplates a fundamental criterion to evaluate the impact of human activities and climate changes on the hydrological cycle. In this study, a novel innovative deep neural network (DNN) structure by integrating a double Gated Recurrent Units (GRU) neural network model with a multiplication layer and meta-heuristic whale optimization algorithm (WOA) (i.e., hybrid 2GRU×–WOA model) is developed to improve the prediction accuracy and performance of mean monthly Chehel-Chai River’s streamflow (CCRSF_m) in Iran. The Pearson’s correlation coefficient (PCC) and Cosine Amplitude Sensitivity (CAS) as feature (input) selection process determine the only precipitation (P_m) as the most effective input variable among a list of on-site potential climate time series parameters recorded in the study area. Thanks to a well-proportioned layer network structural framework in the suggested hybrid 2GRU×–WOA model, it leads to an appropriate total learnable parameter (TLP) compared to standard individual GRU and Bi-GRU as the benchmark models developed in the comparable meta-parameters. This hybrid model under the optimal meant meta-parameters tuned i.e., coupling a state activation functions (SAF) of tanh-softsign, dropout rate (P-rate) of 0.5, numbers of hidden neurons (NHN) of 70, outperforms with an R² of 0.79, NSE of 0.76, MAE of 0.21 (m³/s), MBE of -0.11(m³/s), and RMSE of 0.36 (m³/s). Hybridizing the 2GRU× model with WOA algorithm causes to increase in the value of R² by 6.8% and reduce in the value of RMSE by 20.4%. Comparatively, standard individual GRU and Bi-GRU models result in an R² of 0.59 and 0.66, NSE of 0.55 and 0.6, MAE of 0.91 and 0.53 (m³/s), MBE of 0.047 and − 0.06 (m³/s), RMSE of 1.29 and 0.83 (m³/s), respectively.

Introduction

Background and literature review

During recent eras, due to quick populace growth, industrialization, urbanization, and increasing civic water demands, providing water is a crucial undertaking for lawmaking^1,2,3. It necessitates a strong evaluation of available and future water supplies and the influences of climate and environmental changes on socio-hydrologic systems⁴. Devotion to this anxiety has been amplified lately because of water crises.

Climate changes and anthropogenic doings have brought on a perceptible intensification in periodic surface hydrologic extreme occasions such as an increase in frequency and intensity of universal temperature, rainfall, streamflow, droughts, and floods in the world in the twenty-first century^5,6.

In the prior eras, hydrologists made many investigations to reply to the next question “What occurs to precipitations?”. Streamflow has been regularly pondered as an inclusive response factor to evaluate watershed climatology, hydrology, and other catchment features. Precise spatiotemporal streamflow forecasting as a periodic feature of the atmospheric hydro-meteorological factors are very central matters for hydrologists in the water-related sectors such as regional cascade planning and managing of water resources, irrigation, municipal sustainable development, hydropower generation systems, optimum reservoir operation, agricultural planning, flooding control and risk scrutiny; social security and catastrophe hindrance^7,8,9.

Streamflow forecasting for the long lead time is still a challenging mission as analyzing the river manners for operative objectives. For these purposes, hydrological modeling methods (HMMs) as the recognized worthy framework have been extensively utilized since the mid-1970s to scrutinize, comprehend, and forecast several complex natural procedures of periodic hydrology applications, for instance, streamflow^10,11. In a wider perspective, HMMs dependent on offering solutions at diverse levels of computational intricacy are categorized into two chief kinds: (I) Process-Driven and (II) Data-Driven techniques^3,12.

Adopting process-driven models for hydrological phases entails sophisticated intellectual mathematical formulas and a substantial quantity of geographical multi-source calibration data to assure a satisfactory rate of model exactness^13,14. Accordingly, hydrologists have to exploit the data-driven models as an appropriate substitute to assess the intricate hydrological process. Data-driven models by applying artificial intelligence (AI) techniques try to attain a potential relation among various multidimensional dynamics predictors–target dynamics variables with exceptionally complex unbalanced trends without any previous hypothesis or information on the fundamental physical latent features and relationships among them in estimation catchments^15,16. These flexible and robust approaches have been developed to obviate the troubles of numerical tactics application, costly and timewasting process of large-scale atmospheric hydro-meteorological data records in monitoring and assessing various periodic hydrological parameters in different complex geo-spatiotemporal environments and climatic regimes¹⁷. These techniques have shown admirable competence in assessing multivariate spatiotemporal byzantine and nonlinear univariate time series hydrological events and hydraulic variables in complex environs and climate change such as modeling the periodic groundwater level variations¹⁸, daily air temperature¹⁹, reservoir inflows discharge²⁰, discharge coefficient of diverse weirs^21,22, dimensions of flow separation zone²³, drought forecasting²⁴.

As far as this, to predict streamflow in different environs and hydro-climatic conditions, abundant water science engineers have developed diverse kinds of traditional single data-mining models e.g. Artificial Neural Networks (ANN), Genetic Programming, and Adaptive Neuro-Fuzzy Inference Systems (ANFIS)^25,26,27, Non-Linear Autoregressive Moving Average with Exogenous Input Polynomial model²⁸, Autoregressive-Moving Average, Autoregressive (AR) Moving, and Multivariate Adaptive Regression Splines (MARS) models^29,30,31, Gaussian Process Regression model^32,33, Functional Linear models³⁴, Regression Tree models³⁵, Online Sequential Extreme Learning Machine models^36,37,38, and ensemble and stochastic conceptual data-driven methods³⁹, Empirical Random Forest Family’s model⁴⁰, Bayesian Model Averaging⁴¹, Support Vector Regression (SVR) model^42,43,44. Soo et al. compared the ability of five machine learning models (MLMs), including K-Nearest Neighbors, Support Vector Machine (SVM), Random Forest (RF), ANN, and Long Short-Term Memory (LSTM) in forecasting in Klang River Basin, Malaysia⁴⁵. They announced that within the methods used, RF − III presented superior performance with Symmetric Mean Absolute Percentage Error and Median absolute percentage error amounts of 0.36 and 0.37, respectively.

Of data-driven models, Gated Recurrent Units (GRU) neural network was presented by Cho et al. as a reformed kind of LSTM, the most popular version of deep neural networks (DDNs)⁴⁶. It can effectively satisfy the innate gradient vanishing problem in the usual neural networks and the temporal data using pertinent interior gates^47,48. GRU is contemplated as the leading approach with noteworthy advancement in real-time forecasting of the geological nonlinear rainfall-runoff time series process more effectively with delay times of more than a few months of the watershed. Generally, DDNs can accomplish higher forecasting precision than physical process-based approaches in most situations as they are able to analyze deeper structures and mine high-dimensional data. They have been utilized to assess streamflow process in different watersheds by the majority of researchers^49,50,51.

The multivariate complex time series hydrological variables particularly streamflow, consist of naturally both perpetual stochastic and deterministic elements. Hence, an unimpeachable real-time appraisal contemplates a strenuous and time-consuming concern as a consequence of their extreme long-term non-stationarities and uncertainties latent in the spatiotemporal input-target data, randomization, human interventions, and highly indeterminate nonlinear characteristic accompanied by multipart interactions/forms within atmospheric elements^52,53. For these reasons, the applicability of the traditionally used individual regression-based and data-mining tools is almost ineffectual and has generally bumped into serious predicaments such as high spatial-temporal fluctuations depending on severe uncertainties, nature resolutions, weights fit-tuning, etc^54,55,56.

Taking all the together, to impede the all above-mentioned difficulties and interludes, different strong and leading-edge hybrid signal pre-processing (mode decomposition) and bio-inspired optimization-based data-mining models for large-scale data analytical as an appropriate alternative are being developed to improve noticeably predictions’ talent and efficiency of ordinary models. The hybrid strategies employ the incorporation of two or further data integration and modelling modus operandi prompting feasible to prominently increase the exactness of forecasted streamflow data.

The nature-inspired metaheuristic optimization-based algorithms bring forth improve the ability of standalone predictive models by incorporating different optimization algorithms by realizing close-optimum results within a rational timeframe for the estimation parameter, while concurrently could diminish the computational convergence time period⁵⁷. Outdated customary optimization algorithms have limitations including single-based solutions, complications in indefinite search spaces, and converging to local optima^15,16,58. To date, numerous investigators have designed metaheuristic algorithms to address these limitations. On this point, the whale optimization algorithm (WOA) presented by Mirjalili & Lewis is a robust and reliable bio-inspired metaheuristic optimization algorithm motivated by the intelligence and social life manners of humpback whales⁵⁹. This algorithm is characterized by the bubble-net hunting tactic and is applicable in global optimization machine learning and data mining problems by imitating physical or biological phenomena.

Hitherto, different hybrid algorithms have been developed by adjusting and optimizing the simulation factors and broader choice of the membership function to enhance the accuracy of predicting streamflow in different regions and hydro-climatic conditions. For example, different hybrid meta-heuristic optimization-based algorithms including the Shuffled Frog Leaping, Particle Swarm Optimization (PSO), Ant Colony Optimization, Gray Wolf Optimization (GWO) algorithms⁶⁰, hybrid Genetic Algorithm with SVR and Bayesian Additive Regression Tree models⁶¹, hybrid Improved Complete Ensemble Empirical Mode Decomposition with Adaptive Noise algorithm and GRU model with improved GWO algorithm⁶², hybrid Gravitational Search, PSO and GWO optimization algorithms with the extreme learning machine (ELM) model⁶³, hybrid optimally pruned ELM (OP-ELM), least square support vector machine, seasonal auto regressive moving average, MARS, and M5 model tree⁶⁴, hybrid SVR and generalized regression neural network models with seasonal and trend decomposition algorithms⁶⁵, hybrid fuzzy information granulation with SVR model⁶⁶. A summary of the different hybrid models developed for predicting streamflow in different areas is presented in Table 1.

Table 1 Hybrid models for forecasting streamflow over different time scales.

Motivation for this study

This research intends to estimate a long-term time series of mean monthly Chehel-Chai River’s streamflow (CCRSF_m) using climatic datasets from Sep 1990 to Aug 2020 by GRU deep learning, MLMs. To do so, first of all, the general single GRU layer network as the benchmark model, the general single Bi-directional GRU (Bi-GRU) layer network, and the double GRU coupled with a multiplication layer (i.e., 2GRU× model) network models are developed. Then, the ideal one of these initially designed meant models (based on performance evaluation metrics calculated) is intentionally hybridized with meta-heuristic whale optimization algorithm (WOA) (i.e., hybrid 2GRU×–WOA model) to further improve the prediction accuracy of CCRSF_m. Therefore, we do not limit our investigation only to the conventional deep learning (DL) network structure. Since the WOA algorithm mostly presents a steady and fast convergence rate and can identify optimum solutions in lower populations with the lesser opportunity of local trapping modes, it is used as nature-inspired optimization algorithm.

As the aptitude of these models relies on the kind and rate of some meta-parameters, realizing a fitting optimum pattern is a demanding and bewildering undertaking. Hence, various scenarios are adopted by tuning diverse meta-parameters in the construction of suggested models and the WOA algorithm.

The literature review shows that there has been little research operating different layer structures of the GRU model for time series streamflow forecasting. As far as authors know, amongst the current computational intelligence-system literature centering on streamflow prediction, only commonplace and wide-ranging simple DNN architectural structures have been focused on. The novelty of this research is the development of a leading-edge and robust, unique hybrid 2GRU×–WOA model with different analytical layer network structures, for the first time to predict more complex natural phenomena such as time series CCRSFm oscillations patterns.

Study area and data description

Chehel-Chai River is one of the main branches of Gorganrood River and is located in Golestan Province, northern Iran. The Chehel-Chai watershed is situated inside the city border of Minoodasht with an area of 256,830 (m²), a mean slope of 46%, a maximum and minimum elevation of 2570 and 190 m above sea level, and a moist environment. The mean yearly precipitation and temperature were reported as 750 mm and 15.4 (°C), respectively. Forest (60%) and rain-fed lands (39%) shape the chief surrounding ground cover in this area^76,77. Figure 1 shows Golestan Province and the location of the observation station generated using QGIS 3.40.

Fig. 1

Location map of Chehel-Chai basin in Golestan Province, Iran.

In the current study, to forecast time series mean monthly Chehel-Chai River’s streamflow (CCRSF_m) (m³/s), 360 monthly atmospheric datasets documented from Sep 1990 to Aug 2020 by Lazoreh climatic observation station are exploited. The historic climatic parameters operated include monthly mean, maximum, minimum, absolute minimum, and absolute maximum air temperature (T_ave, T_max,T_min,T_mina,T_maxa), precipitation (P_m), evapotranspiration (ET_m) gotten from the IMO (Iran Meteorological Organization). Table 2 provides some descriptive statistics indices of the variables used in the study area and period.

Table 2 Statistical indices of variables recorded in the study area.

Research objectives

The main contributions of the present work are as follows:

• 1. Identify the most effective variables on CCRSF_m, among a list of on-site potential climate parameters recorded through feature selection techniques.

• 2. Development of standalone and hybrid GRU-based neuro-evolution time series paradigms optimized by WOA nature-inspired metaheuristic algorithm for precise forecasting of CCRSF_m vacillations rhythm.

• 3. Determine the optimal spectrum of aimed meta-parameters in GRU-based models developed and WOA optimization algorithm for better configuration and lessening the impact of overfitting/underfitting problems.

• 4. Assess and compare the accuracy of modeling with counterparts in the validation stage to differentiate the attributes of the best-developed model in offering better reliable and consistent performance using some comparison plots and statistical metrics.

Feature selection process

Because the performance of any modeling is influenced mainly by an apt selection of input variables for the precise prediction of target, unfit selections could adversely affect the effectiveness of any methodology. So, in this section, existing large-dimensional potential hydro-climatic data sets recorded in the studied region are evaluated to recognize the most effective input variables for predicting CCRSF_m as the model target variable. In this context, the variables of extreme importance are selected using Pearson’s correlation coefficient (PCC) and Cosine Amplitude Sensitivity (CAS) as linear and nonlinear representative data analysis methods.

The CAS data inquiry for the variables presented in Table 3 is done by altering each input variable at a fix ratio and holding the other input variables constant as follows⁷⁸:

$${R_{ij}}=~\frac{{\mathop \sum \nolimits_{{k=1}}^{n} {I_{ik}}{O_{jk}}}}{{\sqrt {\mathop \sum \nolimits_{{k=1}}^{n} I_{{ik}}^{2}} ~.~\sqrt {\mathop \sum \nolimits_{{k=1}}^{n} O_{{jk}}^{2}} ~}}$$

(1)

where, I_i and O_j are input and output parameters, respectively, and n is total number of datasets. The R_ij value [0,1] shows the strength of the relationship within the input and target parameters. The values of PCC and CAS methods are presented in Table 3.

Table 3 PCC and CAS values versus CCRSF_m.

According to Table 3, as a result of the insignificant amount of PCC and CAS data analysis methods for T_min, T_max, T_ave, T_mina, T_maxa, ET_m, their effects on predicting CCRSF_m by suggested models can be disregarded. Hence, merely P_m can be considered as the most influential and important input parameter. To conclude, the equation for the prediction of CCRSF_m can be formulated as follows:

$$CCRS{F_m}=f\left( {{P_m}} \right)$$

(2)

Figure 2 (A and B) shows the time series plots of CCRSF_m and P_m recorded by Lazoreh climatic observation station for the studied time that show the seasonality of data. Since the parameters have a temporal pattern, the monthly scale is used as a parameter.

Fig. 2

Time series graphs of variables used in Eq. 2 between Sep 1990-Aug 2020 (360 months) in the Chehel-Chai River watershed: (A) CCRSF_m, (B) Precipitation (P_m).

Methodology

Due to the unstable, intricate, and nonlinear relationship in Eq. 2, only precise and robust approaches are enabled to analyze CCRSF_m. In this context, a sequential dataset of 360 monthly hydro-climatic observations covering the period from September 1990 to August 2020 is used in modeling process. The datasets are normalized to zero mean and unit variance as advised by Lawrence et al.⁷⁹. The normalized datasets are divided into two subclasses. One limited 70% of the data (252 monthly observations) are consecutively applied in calibrating the predictive models. And, the lasting 30% (108 samples) are set aside to be applied serially in validation, without randomization. This process warranted that the data be on a uniform scale, so discrete variable sensitivity did not complicate the results.

GRU and Bi-GRU neural networks

Recurrent Neural Networks (RNNs) assimilate previous info to cross for forecasting the future state of a variable using input data with certain dependencies by enforcing a memory cell containing an unfolded loop cell. However, for large-scale data, its learning process meets with disappearing gradients in the backpropagation training algorithm over time⁸⁰. To overcome this problem, LSTM adds intentionally hidden units to the memory cell of RNNs, so that it maintains information over long periods thanks to its unparalleled progressive structure named, Credit Assignment Paths (CAPs)^80,81,82,83.

GRU neural network is capable of learning long-term relationships and assessing highly nonlinear historical information if the data scale is not too vast. Both GRU and LSTM neural network models act in an alike way with an analogous central framework. GRU is an adaptive neural network since it is quicker in computing, simpler in learning, less condensed in construction, and has fewer learnable factors with a distinguished inherent proficiency^84,85. Figure 3 illustrates the working order of the system and the interior memory cell of GRU, where rt, x_t, and h_t are the reset gate of GRU, input variable, and hidden state at time t, respectively. W_r and U_r denote weight matrices for the input data and hidden state, respectively.

Fig. 3

Internal structure and mechanism of GRU memory cell.

Against LSTM, GRU does not include isolated memory cells, as a substitute, it employs a separate h_t to dispense data over time steps. Furthermore, the input and forget gates are incorporated with an update gate (z), and r_t is straightly applied to h_t−1 to obtain h_t (the candidate state). In this system, the memory cell learns at time t by the input at time t and the output at the prior time step (t-1). The instruction of GRU is defined by the subsequent computations⁴⁶:

$${z_t}=\sigma ({W_z}{x_t}+{U_z}{h_{t - {\text{1}}}}+{b_z})$$

(3)

$${r_t}=\sigma ({W_r}{x_t}+{U_r}{h_{t - {\text{1}}}}+{b_r})$$

(4)

$${\tilde {h}_t}=\tanh (W\tilde {h}{x_t}+U\tilde {h}({r_t} \times {\tilde {h}_{t - 1}})+b\tilde {h})$$

(5)

$${h_t}=\left( {1 - {z_t}} \right) \times {h_{t - 1}}+{z_t} \times {\tilde {h}_t}$$

(6)

In these equations, tanh and σ are the hyperbolic tangent and logistic sigmoid functions, respectively. The sign “×” and b denotes the element-wise multiplication and bias vector, respectively. These factors are learnable sets. Attributable to the impact of σ, whole gates are a vector within (0, 1). When the r_t is locked, GRU is influenced just by x_t and z_t controls the information dimension of h_t−1 can be passed into h_t⁴⁶.

The GRU model only considers the effect of the prior information on the succeeding information without regarding the correlation sides in time series predicting⁸⁶. The Bi-GRU network model with several gates in the memory cells is based on different forms of the general one-directional GRU; nonetheless, reiterating elements within the hidden layer are more intricate. It includes forward and backward GRU to manage the input-output current inside the network and extract features in-depth by forward and reverse historic sequence computations^87,88. The model construction is displayed in Fig. 4. Last of all, the output is calculated by the following formulation⁸⁶:

$${h_t}=\overrightarrow {{h_t}} \odot \overleftarrow {{h_t}}$$

(7)

where, $\overrightarrow {{h_t}} ~and$ $\overleftarrow {{h_t}}$ are the outputs of the forward and backward GRU, respectively.

Fig. 4

Structure of Bi-GRU neural network model.

Whale optimization algorithm (WOA)

Whales are considered the largest mammals with strong intellectual and emotional abilities, like humans, in the world. The whales lived habitually in groups, and the humpback whales have an exceptional hunting technique known as the bubble-net feeding technique. It enforces functional twisting movements to make a bubble-net raiding mechanism called bubble-net feeding. These bubbles are called double-loops and upward spirals. Humpback whales wish to forage for small fish or krill schools near the sea’s surface. It has been perceived that this hunting mode is done by forming typical bubbles sideways a loop or ‘9’-shaped track as exposed in Fig. 5^59,89,90.

Fig. 5

Bubble-net nourishing method of humpback whales⁵⁹.

The WOA was presented by Mirjalili & Lewis⁵⁹ and is considered a well-known swarm intelligence algorithm inspired by the real special chasing tactics of humpback whales in nature, done by the unsystematic or finest search agent to hunt the prey. It is executed in three stages: (i) Siege chasing; (ii) Operation stage: The process of raiding the net bubble; (iii) Exploration stage: Chasing search. Some unsystematic solutions initiated WOA. In each reiteration, the search agents bring their situation up to date by the three operators. The WOA supposes that the optimum solution at the moment (the optimum response) is prey; so, it identifies prey and then encircles prey. When the agent of the optimum search is recognized, other search agents inform their place to the optimum search agent⁵⁹. The following formulas can describe this process:

$$\vec{D} = \left| {\vec{C}.~\vec{X}^{*} \left( t \right) - \vec{X}\left( t \right)} \right|$$

(8)

$$\vec {X}\left( {t+1} \right)=~{\vec {X}^*}\left( t \right) - \overrightarrow {A.} \vec {D}$$

(9)

where, t signifies the present repetition, $\vec {A}$ and $\vec {C}$ are the coefficient vectors, X^* is the position vector of the optimum solution attained at present, and $\vec {X}$ is the position vector. $\vec {D}$ also displays the space amid the hunt and ith whale. The vectors $\vec {A}$ and $\vec {C}$ are computed as follows:

$$\vec {A}~={\text{~}}2\vec {a}\cdot\vec {r} - \vec {a}$$

(10)

$$\vec {C}=2.~\vec {r}$$

(11)

where, $\vec {a}$ is linearly reduced from 2 to 0 over the progress of repetitions (in both exploration and exploitation stages) and $\vec {r}~$is a random vector in [0,1].

The bubble-net invading method contains two chief stages: (i) the encirclement process which whales drive to the water surface) denotes shrinking and includes decreasing $\vec {a}$ in the Eq. (10). The quantity of $\vec {A}$ reduces as $\vec {a}$ declines; and (ii) spiral updating of the whales’ locations is utilized to mimic the spiral activities of whales in the hunt boundary by computing the distance amid the hunt (X^*, Y^*) and the hunter (X, Y)⁵⁹:

$$\vec {X}\left( {t+1} \right)=~\overrightarrow {D^{\prime}} .{\text{~}}{e^{bl}}.~cos~\left( {2\pi l} \right)+{\vec {X}^*}\left( t \right)$$

(12)

$\overrightarrow {D^{\prime}}$= |${\vec {X}^*}\left( t \right) - \vec {X}\left( t \right)|$ describes the distance amid the ith whale and the hunt; b shows a coefficient defining the form of the logarithmic helix-formed movements; l shows a random quantity in [− 1,1]. The movement of the whales near the prey happens alongside the spiral-formed routes by wincing the loops. The following equation is presented to describe this process⁵⁹:

$$\vec {X}\left( {t+1} \right)=\left\{ {\begin{array}{*{20}{c}} {{{\vec {X}}^*}\left( t \right) - \overrightarrow {A.} \vec {D}~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~if~p<0.5} \\ {\overrightarrow {D^{\prime}} .~{e^{bl}}.~cos~\left( {2\pi l} \right)+{{\vec {X}}^*}\left( t \right)~~~~~~if~p \geqslant 0.5} \end{array}} \right.~~$$

(13)

where, p ∈ [0,1]; this permits one to catch the likelihood of retaining the spin mode so as to bring up-to-date the positions of the whales. In the exploration (searching) stage, the humpback whales’ quest for the prey is arbitrarily consistent with their position as matched to other whales⁵⁹. Thus, the whales bring their situations up-to-date, compliant with randomly chosen search factors instead of the premium search factor⁵⁹:

$$\vec{D} = \left| {\vec{C}.~\overrightarrow {{X_{{rand}} }} - \vec{X}} \right|$$

(14)

$$\vec {X}\left( {t+1} \right)=\overrightarrow {{X_{rand}}} - \overrightarrow {A.} \vec {D}$$

(15)

where, $\overrightarrow {{X_{rand}}}$ is a random site detected by the present population. Figure 6 offers pseudo-code of the WOA algorithm.

Fig. 6

Pseudo-code of the WOA algorithm (Mirjalili and Lewis, 2016).

Model development

As mentioned above, in this study, first, the general single GRU and Bi-GRU neural networks are used as benchmark models for performance comparison, and then a 2GRU× neural network model with sequence output mode is developed to estimate CCRSF_m in the study area. Figure 7A–C portrays the GRU-based layer network structure of models.

Fig. 7

Designed layer network structure of GRU-based models: (A) General single GRU model (1), (B) General single Bi-GRU model (2), (C) and 2GRU× model (3).

Finally, due to the smallest RMSE and highest R² values in Model 3 compared to Models 1 and 2, it is preferred to hybridize with the WOA algorithm (i.e., hybrid 2GRU×–WOA model (4)) to improve the prediction accuracy of CCRSFm further. In this model, WOA trained the bias and weights within layers of the 2GRU× model so that they could be updated in keeping with a proportion recognized by the premium WOA. Then, the training set is applied to the renovated bias and weights of the 2GRU× model, and the WOA optimizes them in each repetition by randomly dispensing mode. This hybrid model terminates as a maximum number of iterations are gotten or as the best solution is obtained for a certain number of iterations; if not, it proceeds with the next generation operation.

The capability and computation complexity of the hybrid 2GRU×–WOA model generally relies on using the suitable main deterministic factors of the WOA algorithm, including population size (PS), maximum number of iterations, total load demand, up-coefficient vector, and down-coefficient vector. Selecting the optimal factors of bio-inspired WOA algorithm is very imperative so that the optimal key deterministic parameters and pattern for 2GRU× model be achieved using the optimization process. Figure 8 defines the forecasting process of the hybrid 2GRU×–WOA model.

Fig. 8

Flowchart of integrated 2GRU× with whale optimization algorithm (WOA) (i.e., hybrid 2GRU×–WOA model (4)).

In these models, the amount of P-rate and the kind of SAF as meta-parameters are exactly tuned to realize a proper pattern and augment the skill of the models designed. Nonetheless, since there is no formal pre-instruction to identify suitable meta-parameters for DDN models with a given dataset, this process is pondered as a time-wasting and demanding task¹⁸. For the sake of this aim, several scenarios are adopted to realize valid values.

In all models, the Input layer feeds the time series P_m into the layers’ network structure. For a useful configuration and modification in the big dataset, the Dropout layer is operated to deter overfitting by passing over some hidden neurons with a prearranged option rate of P⁹¹. To tune the amount of P-rate as a meta-parameter, various values are tested. To strengthen fitting ability in learning long-term sequential datasets, the Fully Connected layer is utilized with input and output sizes of “auto” and 1, respectively. The Multiplication layer multiplies the inputs from various layers’ neural network element-wise. As the ending layer, the Regression Output layer is utilized to compute the “half-mean-squared-error loss” for regression objectives. To tune SAF, a diverse combination of tanh and softsign are distinctly utilized, and for NHU, numerous amounts are tested. In order to preclude the gradients disappearing and lessen the negative influence of padding drawbacks, a training process with 1000 maximum repetitions is set as recommended by Lin et al.¹⁸. More details about the function of layers in the all models developed were provided by⁹².

Performance evaluation metrics

In this modelling, the following statistical metrics are applied to liken the capability and performance of all models used in predicting the time series CCRSF_m:

$$\begin{gathered} {\text{Coefficient of Determination }}\left( {{R^2}} \right); \hfill \\ {R^2}=\frac{{\sum\nolimits_{{i=1}}^{N} {{{({O_i} - {\mu _o})}^2} - \sum\nolimits_{{i=1}}^{N} {{{({O_i} - {P_i})}^2}} } }}{{\sum\nolimits_{{i=1}}^{N} {{{({O_i} - {\mu _o})}^2}} }} \hfill \\ \end{gathered}$$

(16)

$${\text{Root Mean Square Error }}\left( {RMSE} \right);RMSE=\sqrt {\frac{{\sum\nolimits_{{i=1}}^{N} {{{({O_i} - {P_i})}^2}} }}{N}}$$

(17)

$${\text{Mean Bias Error }}\left( {MBE} \right);MBE=\frac{{\sum\nolimits_{{i=1}}^{N} {({P_i} - {O_i})} }}{N}$$

(18)

$${\text{Nash}}--{\text{Sutcliffe Efficiency }}\left( {NSE} \right);NSE=1 - \frac{{\sum\limits_{{i=1}}^{N} {{{\left( {{O_i} - {P_i}} \right)}^2}} }}{{\sum\limits_{{i=1}}^{N} {{{\left( {{O_i} - {\mu _o}} \right)}^2}} }}$$

(19)

$${\text{Mean Absolute Error }}\left( {MAE} \right);MAE=\frac{{\sum\nolimits_{{i=1}}^{N} {|{O_i} - {P_i}|} }}{N}$$

(20)

where N is the number of datasets, ${P_i}$ and ${O_i}$ are the predicted and observed CCRSF_m at time i, ${\sigma _o}$ and ${\sigma _P}$ are standard deviations of the observed and predicted the CCRSF_m. The ${\mu _p}$ and ${\mu _o}$ are the mean predicted and observed CCRSF_m. The best amount for Eqs. (16–20) are 1, 0, 0, 1, and 0, respectively.

Results and discussion

Validation of the models

In this simulation, numerous experiments are conducted to determine the optimal value of the main deterministic factors in the models developed. The characteristics and statistical results of all models used under the optimal scenario in the validation phase in forecasting CCRSF_m are shown in Tables 4 and 5, respectively. The models in their training stages are more precise than in their testing stages. The tanh-softsign pairing in the hidden layers of the 2GRU× and hybrid 2GRU×–WOA models brings forth learning more complicated nonlinear functions, and accordingly, it causes the models not to be as much open to the overfitting dilemma. Besides, the ideal amount of main deterministic factors for WOA algorithm in the hybrid 2GRU×–WOA model for the best solution in forecasting CCRSF_m is achieved as an up-coefficient vector of 0.25, a down-coefficient vector of 0.1, a total load demand of 0.05, a maximum number of iterations of 500 and a population size of 30.

Table 4 The characteristics and ideal meta-parameters of all models developed in forecasting CCRSF_m.

Table 5 The statistical results of models developed under the ideal meta-parameters in forecasting CCRSF_m.

According to Table 5, it can be concluded that hybridizing with the WOA algorithm advances noticeably the performance and ability of the 2GRU× model. This optimization algorithm augments the 2GRU× model training phase and achieves better efficiency in the predicting CCRSF_m. Additionally, the value of MBE shows that all models except the model 1 underestimated the corresponding measured values at the validation phase.

RMSE variations in the model 4 over the range of NHN used under the optimal hyper-parameter in the testing phase is displayed (Fig. 9). High NHN causes RMSE to grow as a result of overfitting, nevertheless, small NHN reduces network learning skill because of underfitting.

Fig. 9

RMSE variations in the model 4 over the range of NHN used under the optimal hyper-parameter in forecasting CCRSF_m in the testing phase.

Performance comparison

In this modelling, to compare the skill and efficiency of models used in capturing the time series CCRSFm, a substantial factor called TLP (Total Learnable Parameters) is exploited, as suggested by Lin et al.¹⁸ (Table 4). The TLP is considered a crucial criterion for the discriminating forecasting performances and practical capacities of DL-based models. Moreover, it can also assess tendencies toward under-/over-fitting effects.

Based on Tables 4 and 5, though Model 1 has the extreme TLP value, Model 4 results in the best performance and surpasses other models by capturing the time series CCRSF_m. The dominant reason for this is explainable by the high quantity of TLP in the model, 1 which led to an extremely unnecessary network capacity and accordingly, it prompts overfitting and hinders the optimization process. On the contrary, due to the lesser amount of TLP in model 2, it has a lesser network capacity, accordingly, leading to underfit. Generally, models 1 and 2 are not capable of monitoring and predicting time series CCRSFm in the study region due to poor performance. The model 4, thanks to the well-adjusted layer network structure and consequently TLP value, excelled over other models. It is owing to the well-proportioned TLP amount that it could hastily get the ideal weight sets − 1000 iterations in 33 s, while the model 1, for the uppermost TLP number, entails extra time to converge – 1000 iterations in 48 s.

Figure 10 matches pictographically the hydrograph plot for the measured and predicted temporal CCRSF_m by model 4 under the ideal meta-parameters in the validation phase. Along with this figure, model 4 owing to its inventive advanced layer’s network structure can agreeably estimate the distribution of measured sequential CCRSF_m and fit relatively the vacillations trend mostly in the peak and deepest values of CCRSF_m that prove an acceptable unanimity.

Fig. 10

Hydrograph plot of the measured and predicted time series CCRSF_m by models 1 and 4 under the ideal meta-parameters during the validation phase (108 months between September 2011– August 2020).

In terms of distribution criteria, scatter diagrams for the measured and predicted time series CCRSF_m by the models used during the validation stage are presented in Fig. 11A–D. By a visual judgment, it is noticeable that the predicted CCRSF_m by the hybrid 2GRU×–WOA model is generally near to the exact line (i.e., 1:1) for abundant data points with minor scattering compared to the GRU model. It validates a high steadiness and best-performing approach with a satisfactory R² of 0.79.

Fig. 11

Scatter plot for the measured and predicted CCRSF_m (m³/s) by the (A) GRU model (1), (B) Bi-GRU model (2), (C) 2GRU× model (3), and (D) hybrid 2GRU×–WOA model (4) in the validation stage.

A violin plot is presented to concomitantly match the performance and skill of all models and single out the best model used in the validation phase (Fig. 12). By a visual evaluation, it can be concluded that the hybrid 2GRU×–WOA model relatively better fits the distribution of the observed temporal CCRSF_m and could estimate to some extent exacter the peak and lowest values in comparison with the other models developed.

Fig. 12

Violin plot of the observed CCRSF_m against the predicted by the models developed under optimal meta-parameters in the validation phase.

Summary and conclusion

In this study, different layer structures of GRU-based deep learning framework were developed to estimate the forecasting CCRSF_m from Sep 1990 to Aug 2020 (360 months). In all models, to satisfy the long-period nonlinearity and non-stationary dilemmas, the seq2seq regression forecasting module is applied. The most worth mentioning outcomes of the modelling process are:

• 1. The PCC and CAS data analysis methods approved that the P_m was the most influential predictor variable on CCRSF_m to feed the models developed.

• 2. The training-stage forms of all models were more precise than their validation counterparts.

• 3. After several trials, the suggested hybrid 2GRU×–WOA model was accepted as the best-performing model by performance evaluation criteria to forecast CCRSF_m. The optimal P-rate, NHN, and SAF tuned for this model were obtained to be 0.5, 70, and tanh-softsign, respectively. Integrating the 2GRU× model with WOA algorithm caused to increase in the value of R² by 6.8% and reduced in the value of RMSE by 20.4%.

• 4. By comparing the model structures developed and relevant TLP values, it can be concluded that inserting the Multiplication layer led to a more suitable layer network structure and well-adjusted TLP. So, for achieving effective DL-based models, an apt network structural, NHN, and well-balanced TLP value should be applied.

• 5. In all models, growing P-rate value lessens convergence time. The model 1 for the high TLP quantity, necessitated more time to train – 1000 repetitions in 48 s.

The hybrid 2GRU×–WOA structure is a cutting-edge method as verified by its commendable accuracy and performance (verified statistically). It can therefore be employed as an intelligent smart model for monitoring and predicting time series river streamflow under different climatic conditions. This hybrid model is an easy-to-implement, cost-effective, dependable, and time-saving process. Its well-formed layer network structure prompted an apt response to TLP and engendered more precision than the standard GRU and Bi-GRU as the benchmark models in the same meta-parameters. Despite the advantages of a hybrid model, it has some constraints: it entails an extremely long period of detailed (i.e., regular measurements) precipitation data to predict CCRSFm in a study area, as the seq2seq regression module of forecasting was employed.

Even though this study evaluated the effects of different GRU-based model and WOA algorithm structures on forecasting CCRSF_m, upcoming studies could examine other methods. For instance, modeling could integrate DNN models with the most up-to-date optimization algorithms, such as the Puma Optimizer (PO) and Mountain-Gazelle Optimizer (MGO). The results should be equated to the outcomes of the current study to obtain the most effective technique.