Introduction

The moisture content in the atmosphere and large scale moisture convergence over India is increasing1 together with more rapid increase of the heat content of the upper ocean and sea surface temperature (SST) over the Indian Ocean (IO)2, largely associated with the increase in global mean temperature from anthropogenic activity3. Increasingly more unstable atmosphere with ample moisture supply from the warm IO assures that the frequency and intensity of daily ERE over India is increasing rapidly in recent decades4,5,6,7. In a gridded daily rainfall dataset, an ERE at a grid box could be defined as the events where daily rainfall exceeds 99.5 percentile of the rainfall distribution at the grid point4,8. The seasonal ERE frequency over a region is determined by the aggregations of such events across all grid boxes representing the region during a given Indian summer monsoon season. Over Central India (CI), rain events exceeding ~10 cm/day represent such events4, while over Northeast India (NEI), where mean rainfall is higher, events exceeding ~20 cm/day represent such events8. About 9 such events used to occur over CI during June-September (JJAS) in 1901 that increased to about 18 in 20101, consistent with significant increase of such events6. Over the NEI, such events have doubled from about 10 in 1920 to about 20 events in 20109. Flash floods, landslides and torrential rains associated with the extreme events kill thousands and displace millions of people and animals every year in India. The plains of CI, as well as the plains over NEI, are flood prone areas and floods alone account for more than $3 billion in economic losses every year in India6. Additionally, hydrological disasters from ERE also lead to food productivity loss in countries like India10. In the backdrop of the trend of the increase of rainfall extremes being nonlinear with rapid increase in recent decades1, the frequency of hydrological disasters associated with these events are expected to increase at a faster rate in the coming decades. Compounding with the hydrological disasters, rapid increase in humid heat stress (Humidex) in recent decades11 is going to make outdoor activity increasingly difficult, leading to productivity loss affecting the developing countries disproportionately. The socio-economic loss from climate extremes would make it untenable to maintain economic growth required for countries like India12,13.

The EREs are increasing not only over CI and NEI but also over the semiarid northwest India (NWI)14 together with everywhere in the tropics15. As a result, the socio-economic loss from hydrological disasters is increasing rapidly in the tropics. To minimize the accelerating socio-economic loss from increasing frequency and intensity of extreme events, seasonal prediction of extreme rainfall frequency (ERE) during summer monsoon season has become important but lacking. While the importance of forewarning of seasonal Tropical Cyclone Frequency (TCF) or accumulated cyclone energy (ACE)1 has been recognized and has led to development of skillful long-lead seasonal forecasts of TC frequency14,15,16,17,18,19,20,21,22,23, a similar recognition of the disaster potential of the accumulated seasonal EREs in the country and requirement of a seasonal prediction model has been lacking. Also, the ERE frequency during the Indian monsoon season over the country is about 36, the TC frequency in the North IO in a year is only about 5, which may have increased to about 7 in recent decades. Thus, the socio-economic loss due to ERE may be significantly higher than that associated with TC. Yet, research on seasonal prediction of ERE frequency over any of the monsoonal regions is lacking.

To fill this major gap in seasonal forecasting of ERE in tropics, here, we develop a seasonal prediction model for predicting ERE frequency and ERA 1-month in advance. However, the physical basis for predictability for seasonal prediction of ERE frequency and ERA needs to be established. As daily extreme events are instabilities on the background mean circulation and thermodynamics, we envision that the predictability of inter-annual variability of ERE frequency and ERA would also be governed by the slowly varying drivers that modulate the mean Indian monsoon. In a recent study24, we established that the simultaneous teleconnections from all three tropical ocean basins or G-ENSO is essential for the predictability of ISMR at any given season. We call it G-ENSO as against traditional ENSO defined based on Pacific SST. Using a simple empirical model, we demonstrated that the G-ENSO represented by the depth of 20° isotherm (D20) over 0° -360°E, 30°N-30°S is a better predictor of ISMR compared to traditionally used SST and unravelled that ISMR is highly predictable at 18-month lead (Fig. 2c of Sharma et al.24). We also show in Sharma et al.24 that the apparently non-intuitive result of high potential predictability of ISMR at 18-month lead (Fig. 2c of Sharma et al.24) is due to a unique phase locking of the growth of errors in the coupled ocean-atmosphere system with the monsoon annual cycle making forecast errors to oscillate in such a way to have a minimum at 18-month lead with respect to ISMR (Fig. 5d of Sharma et al.24). The feasibility of ISMR prediction at 18-month lead is also demonstrated using linear and deep learning models. Here, we argue that the physical basis for predictability and seasonal prediction of ERE frequency is the same as the basis for predictability and seasonal prediction of ISMR. The predictability of the seasonal mean Indian monsoon climate or ISMR comes from its association with slowly varying predictable global climate modes like the ENSO25,26,27,28 and Atlantic Multi-decadal Oscillation (AMO)29,30,31,32,33. ERE during summer monsoon arise from thunderstorms spawned in mesoscale convective clusters4. Therefore, the predictability of seasonal mean ERE frequency and ERA is going to come from their association with the ENSO and AMO. In the present study, we demonstrate that the association of the G-ENSO with seasonal ERE frequency and ERA are similar to those with ISMR. However, we also recognize that the seasonal ERE frequency could also be influenced by higher frequency variability associated with land-surface processes and Indian Ocean variability unrelated to the ENSO, such as the Indian Ocean Dipole Mode34. Therefore, interannual variability of ERE frequency could be significantly be different from that of ISMR and influence its predictability.

Recently, deep learning models have been found to be powerful tools to push the limit of useful seasonal prediction of ENSO35 and ISMR24 beyond that achieved by the state-of-the-art climate models. In the present study, we develop such a deep learning based convolutional neural network (CNN) model for seasonal prediction of ERE frequency and ERA over CI (74°E-86°E, 15°N-26°N) that demonstrates significantly higher skill compared to linear regression models.

Results

Potential Predictability of Seasonal ERE frequency and ERA

To demonstrate that the seasonal frequency of ERE and ERA during June-September over CI (Fig. 1a, b) are driven by the same global recharge-discharge oscillator (G-ENSO) represented by D20 over 0° -360°E, 30°S-30°N that modulates ISMR, we extend the applicability of our predictor discovery algorithm, developed in Sharma et al.24 for unravelling the long-lead predictability of ISMR, to estimate the potential predictability of the seasonal frequency of ERE and ERA. We generate D20-based predictors (Dp) up to 24-month leads by projecting appropriately lagged D20 anomalies on the statistically significant regions of the global correlation map between D20 and seasonal ERE frequency/ERA anomalies (see Methods). The patterns of lead-lag correlations between D20 and seasonal ERE frequency/ERA anomalies up to 24-month leads (Figure S1, S2) indicate that they are associated with different phases of evolution of the G-ENSO. While at one lead, the Pacific D20 anomalies may dominate, at other leads, the D20 anomalies over Indian Ocean and Atlantic Ocean also contribute significantly to the teleconnection influencing the interannual variability of the ERE frequency and ERA. Since the Dp at any lead is based on the projection on the global correlation map between D20 and seasonal frequency of ERE/ERA anomalies, the Dp predictor incorporates information from the G-ENSO. The correlation between Dp and seasonal ERE frequency/ERA as a function of lead time is shown in Fig. 1c. Figure 1c indicates that the correlation between Dp and seasonal ERE frequency/ERA is lower at short leads (1–4 months lead) and increases at longer leads, remaining almost consistent with a mean value of ~0.85 between 5-22 months lead. The correlation between Dp and seasonal ERE frequency/ERA differs from the correlation between Dp and ISMR (Fig. 2c of Sharma et al.24). This discrepancy arises because the correlation between the ISMR anomaly and the seasonal ERE frequency anomaly over CI from 1901 to 2022 is 0.17. Consequently, the interannual variability of ERE differs significantly from that of ISMR, likely due to high-frequency contributions from local processes. This difference partly explains the variations in the correlation of Dp with ISMR and seasonal ERE frequency.

Fig. 1: Variability and predictability of seasonal ERE frequency and ERA over CI during summer monsoon season.
figure 1

(a) generated from 1°x1° resolution grid boxes (b) generated from 2° x 2° resolution grid boxes. ERA and Frequency along with trend are shown in Figure S3. c Anomaly correlation coefficient (ACC) between D20 based predictor (Dp) and seasonal ERE frequency and ERA as a function of lead month. d Estimation of potential predictability in terms Potential ACC estimation of seasonal ERE frequency and ERA as a function of lead month.

Full size image

With the Dp predictors from the predictor discovery algorithm, we evaluate the potential skill of seasonal ERE frequency and ERA at lead times of up to 24 months by training a linear regression model with Dp as predictors and seasonal ERE frequency/ERA as predictands. The potential skill or potential anomaly correlation coefficient (ACC) is defined as the maximum correlation between observed and predicted seasonal ERE frequency/ERA generated from ‘perfect’ initial conditions. The seasonal ERE frequency/ERA is taken from 1960 to 2022. Given the availability of the D20 dataset from 1958 to 2022, we apply a cross-validation approach to ensure a sufficiently large training sample. In this method, a 5-year window is left out for hindcasting while the linear regression model is trained on the remaining years of Dp and seasonal ERE frequency/ERA. The process is repeated iteratively, with the 5-year hindcast window shifting sequentially from 1960–1964 to 1965–1969, and so on. In the final iteration, only three years of hindcasts are available due to the dataset’s time span. Following this, forecasts of seasonal ERE frequency and ERA are made between 1960 and 2022, and validated with observations (Fig. 1d). Figure 1d indicates that the potential skill of seasonal ERE frequency/ERA is higher at longer leads (5-22 months lead) than at shorter lead times (1-4 months lead) with potential ACC close to the correlation between Dp and seasonal ERE frequency/ERA (Fig. 1c) across all lead times. As explained in Sharma et al.24, the monsoon season is between June-September and at short leads of 1–3 months lead, the predictions initiated from March to May have to go through the ‘spring predictability barrier’ and the coupled system has larger event-to-event variability and smaller ‘signal-to-noise’ ratio in the system. This means that there is also more ‘noise’ or more ‘diversity’ in the G-ENSO-monsoon relationship for these leads. As shown in Sharma et. al.24, the growth of errors in the coupled system is ‘fast’ for initial conditions 1-3 months ahead of the monsoon season, giving rise to the ‘spring-predictability barrier’. Hence, the lower potential skill for 1-3 months lead is understandable. However, at longer leads, predictability is governed by ‘slow’ growing errors and leads to higher potential predictability36,37. Although the hindcast period is independent of the training period, the predictor Dp is derived from correlation maps constructed using the full dataset (predictor discovery algorithm), making it dependent on the complete data record. As a result, these hindcasts are generated with ‘perfect’ initial conditions and potential skill estimates suffer from some degree of built-in artificial skill or overfitting24,38.

In the absence of a forecast model for the seasonal ERE frequency and ERA, we currently focus on the development of such a model at one-month lead time. Since the potential skill of seasonal ERE frequency and ERA at one-month lead time exceeds 0.75, achieving the potential skill even at one-month lead would mark a significant advancement in the seasonal prediction of extreme event frequencies associated with ISMR. To realize the potential skill at one-month lead, we employ both linear and non-linear forecast techniques. In the following, the forecast techniques and their results are evaluated in more detail.

Forecasts and verification

Linear regression model forecasts

If a linear regression model could provide useful forecasts, there is no need for going to a more complicated deep learning model. To demonstrate the feasibility of realizing the potential skill on a set of completely independent hindcasts at one-month lead using linear regression model, the monthly D20 anomalies and the seasonal ERE frequency/ERA datasets are separated into two parts to form training and testing sets so that the testing dataset is neither used in the predictor discovery nor in the training of the linear regression model. The Dp for the training period is generated based on projection of the May D20 anomaly between 1960 and 2002 on the correlation map between May D20 and seasonal ERE frequency/ERA anomalies for the period 1960–2002. To keep the testing period completely independent from predictor discovery as well as model training, the Dp for testing period (2003-2022) are generated based on projection of May D20 anomaly between 2003 and 2022 on the correlation map between May D20 and seasonal ERE frequency anomalies for the period 1960-2002. Hence, the Dp predictors for both the training and testing periods are computed based on projections using correlation maps derived solely from the training period. The actual skill is defined as the correlation between observations and completely independent hindcasts generated by the model and shown in Fig. 2a, b. The skill of seasonal ERE frequency/ERA prediction between 2003 and 2022 using Dp at one-month lead (r = −0.08) comes to be far short of the potential skill of seasonal ERE frequency (Fig. 1d). While a long-lead forecast is not the objective of this study, we generate a set of completely independent hindcasts over the same 20-year period (2003-2022) at 6-month lead (Fig. 2a, b). The skill of r = 0.22 and r = 0.40 of seasonal ERE frequency and ERA, respectively, at 6-month lead represents an improvement of the linear regression model’s forecast over 1-month lead. However, the skill still remains significantly below the potential skill estimate at 1-month lead.

Fig. 2: Linear regression model’s forecast.
figure 2

a Independent hindcast of Normalized seasonal ERE frequency anomaly and (b) Normalized ERA anomaly forecast over CI between 2003 and 2022 generated by the linear regression model at 1-month lead (red) and 6-month lead (blue). The observed normalized seasonal ERE frequency and ERA anomalies generated from 1° x 1° grid boxes are presented by the black curve. c, d Correlation between May D20 and seasonal ERE frequency anomalies over the period 1960–2002 and 1960–2003, respectively. e Difference between the correlation maps shown in (c, d).

Full size image

The significant difference between the actual skill and potential skill is consistent with the large difference between the ‘perfect’ initial conditions and ‘real’ initial conditions (Fig. 2c–e). The Dp predictors are based on the projection of D20 anomaly on the correlation map between D20 and seasonal ERE frequency/ERA anomalies. Such correlation maps, shown in Fig. 2c, d, highlights the presence of small-scale correlation patterns embedded in the large-scale smooth pattern. The difference between the correlation maps for the period 1960–2002 and 1960–2003 (Fig. 2e) indicates that while the large-scale correlation pattern may not change significantly in one year, the small-scale correlation pattern changes significantly. Hence, the cumulative effect of the small-scale non-linear changes in the correlation maps between D20 and seasonal ERE frequency anomalies when the testing period is excluded in the predictor discovery (‘real’ initial conditions) and when the testing period is included in the predictor discovery (‘perfect’ initial conditions) leads to a significant change in the projection. This, in turn, results in a large difference between perfect and real initial conditions, making it challenging to develop a skillful forecast model for seasonal ERE frequency using linear techniques. Hence, a non-linear predictor discovery algorithm is warranted to further improve the actual skill.

Deep learning CNN model forecasts

We test the feasibility of improving the skill of forecasting seasonal ERE frequency and ERA at 1-month lead using non-linear predictor discovery technique. Hence, we developed a Convolutional Neural Network (CNN, Fig. 3), a class of deep learning model, guided by the physical association between G-ENSO and seasonal frequency of ERE/ERA. For the development of the physics-guided deep learning model (see Methods), we obtained D20, seasonal ERE frequency, and ERA anomalies from the historical outputs of Coupled Model Intercomparison Project Phase 6 (CMIP6) (Figure S4, S5). Due to the large number of training parameters, a sufficiently large training dataset is required to prevent overfitting. Overfitting occurs when a deep learning model learns patterns from the training data too well, including noise and random fluctuations, resulting in poor performance on unseen test data. This happens when the model becomes too complex relative to the amount and quality of training data or when it lacks appropriate regularization. Due to the limitation in the observational oceanic temperature data, a total of 35 ensemble members from 14 CMIP6 models between the period 1850 to 2014 are used for the training process of the deep learning model (Table S1). The biases of the CMIP6 models39 unwittingly provide a wide spectrum of possible predictor-predictand relationships that occur in observations for effectively training the models. This concept has been successfully used in extending lead of useful prediction of ENSO35,40 and Indian Ocean Dipole mode41 as well as in extending the skill of useful prediction of East Asian monsoon42. The forecast skill of the model is evaluated with observational D20 (seasonal ERE frequency/ERA) anomalies between 1958 and 1988 as the validation dataset and D20 (ERE frequency/ERA) between 1992 and 2022 as the testing dataset. The daily rainfall data available from most CMIP6 models is at a resolution poorer than 1° x 1°. As the CMIP6 ensemble members have varying resolutions, all the D20 (rainfall) datasets are interpolated to 2.5° x 1.25° (2° x 2°) along the longitudinal and latitudinal directions, respectively. As long training data from CMIP6 models is not available at 1° x 1° resolution, we also show in Fig. 1b, seasonal ERE frequency and ERA calculated from the daily rainfall data regridded to 2° x 2° resolution. While the seasonal ERE frequency and ERA decrease at lower resolution, the interannual variability remains unchanged as the correlation between seasonal ERE frequency for 1° x 1° and 2° x 2° resolution is more than 0.97 while that between ERA of 1° x 1° and 2° x 2° is 0.85 (Fig. 1a, b). The time series of both ERE frequency and ERA from observations (Fig. 1a, b) as well as simulated by CMIP6 models (Figure S4 and S5) have nonlinear trends and could be estimated by second or higher order polynomial fits.

Fig. 3
figure 3

Schematic diagram of the CNN architecture used for the seasonal forecast of ERE frequency and ERA over CI during June-September.

Full size image

During our training of the CNN model, we recognized that training the ERE on the May D20 anomaly map over the tropical basin (0°E-360°E, 30°S-30°N) at just one-month lag, namely D20 (D20-1m) is not sufficient to train it to provide useful forecasts of ERE at 1-month lead. We hypothesize that this is due to the existence of higher frequency interannual variability in the ERE time series. To overcome this problem, we decided to constrain the ERE to train retaining the evolutionary history of G-ENSO-ERE relationship for a period of 8 months. Considering 0 represents the year of prediction while -1 represents one year prior to the year of prediction, to forecast seasonal ERE frequency at one-month lead, tropical D20 anomalies from October(-1) to May(0) over 0°E-360°E, 30°S-30°N are used in the training. It is like finding conditional probability of ERE (t = +1) given D20-1m …. D20-8m. We find that this leads to significant improvement in the CNN model’s ability in forecasting anomalies of seasonal ERE frequency at 1-month lead. The predicted anomalies are normalized by dividing with standard deviation of the predicted period before comparing the results with the observational data. A detailed description of the CNN’s architecture and its components is provided in the Method section and in Text S1–S6.

An ensemble mean forecast of ten-members indicates that the CNN outperforms the linear model in making independent forecast at 1-month lead. The CNN predicts the normalized seasonal ERE frequency anomalies for the independent testing period 1992-2022 with a correlation (r) of 0.69 and root mean squared error (RMSE) of 0.77 (Fig. 4a) by learning the intricate non-linear relationship of seasonal ERE frequency with the tropical ocean dynamics as depicted by the D20 field from October(-1) to May(0) simulated by the CMIP6 models. Due to the significant similarities in the variation of seasonal ERE frequency and ERA, the CNN with the same model parameters trained on the relationship between ERA and D20 anomalies from October(-1) to May(0) could also predict the normalized ERA anomalies for the same testing period (1992-2022) with correlation skill of 0.61 and RMSE of 0.86 (Fig. 4b). The success of the CNN is also evident from its ability to provide meaningful seasonal ERE frequency and ERA forecast for the majority of the cases between 1992 and 2022 without employing observational dataset for training the CNN. Notably, all the seasonal ERE frequency and ERA events with normalized amplitude greater than 1 (1994, 2005, 2006, 2007, 2019) are successfully predicted by the CNN. The model is also successful in forecasting 3 out of 5 events with normalized seasonal ERE frequency less than -1. The CNN’s ability in discriminating among above normal events, normal and below normal events is verified using relative operating characteristics (ROC) analysis (Fig. 4c, d). The Area Under the Curve (AUC) in ROC measures the model’s ability to distinguish between different classes. An AUC of 1.0 indicates perfect classification, while AUC < 0.5 means the model performs no better than random guessing or unreliable classification. The AUC > 0.5 for the CNN across all categories suggests that the model has predictive skill and performs better than random classification. The points on the ROC curves indicate the various thresholds used in each of the three events. Normalized ERE frequency or ERA anomalies of 1.55, 1.35, 1.15, 0.95, 0.75, and 0.55 (−1.55, −1.35, −1.15, −0.95, −0.75, and −0.55) are taken as thresholds for the above (below) normal ERE frequency or ERA events, whereas Normalized ERE frequency or ERA anomalies between −0.05 to 0.05, −0.15 to 0.15, −0.25 to 0.25, −0.35 to 0.35, −0.45 to 0.45, and −0.55 to 0.55 are taken as thresholds for the normal ERE frequency or ERA events (see Methods).

Fig. 4: CNN model’s forecast.
figure 4

Ensemble mean forecast (red) and min-max spread (grey) of (a) Normalized seasonal ERE frequency anomaly and (b) Normalized ERA anomaly forecast over CI between 1992 and 2022 generated by the CNN model. Normalization is done by dividing the time series with their standard deviation. The observed normalized seasonal ERE frequency and ERA anomalies generated from 2° x 2° grid boxes are presented by the black curve. c ROC curve of the above normal (black), normal (red), and below normal (blue) ERE frequency events forecasted by the CNN model. (d) Same as (c) but for ERA forecast. The forecasts from all ten models are used for the generation of the ROC curve. The area under the curve (AUC) in all the three cases are indicated. The dashed line indicates zero ROC skill.

Full size image

Further verification of the categorical forecast of seasonal ERE frequency and ERA is done by estimating the Heidke Skill Score (HSS) and Accuracy (Table 1). The HSS and Accuracy of both seasonal ERE frequency and ERA indicates that the CNN’s prediction is better than random chance. Moreover, the usefulness of the forecasts is also indicated from the signal to noise ratio (S/N) greater than 3 indicating significantly greater signal strength than noise. Further, the positive mean squared skill score (MSSS) indicates the model’s capability in capturing variability about the climatological baseline values (Table 1), overcoming the generic problem of statistical models that often tends to predict the mean and fails to predict the extremes. A detailed description of HSS, S/N, and MSSS calculation is provided in Text S7–S9.

Table 1 Verification of ERA and F forecast using HSS, Accuracy, S/N, and MSSS
Full size table

Visualization of the Model’s Learning

To quantify the role of the global recharge-discharge oscillator associated with the seasonal frequency of Indian summer monsoon ERE, we estimated the integrated gradient (IG, Text S10) of the 2007 seasonal ERE frequency forecast made by the CNN (Fig. 5b). The absolute value of IG is a measure of the grid-wise influence of the input map on the CNN’s forecast. The input multiplied by the corresponding IG highlight the grid wise importance of the positive-negative D20 anomaly signal picked up by the CNN for making the prediction. The year 2007 was an excess ISMR year along with higher frequency of ERE over CI. The combined effect of Indian, Pacific, and Atlantic ocean dynamics in modulating the interannaul variation of ISMR as well as the extreme rainfall activity during ISMR is well established24,43,44,45,46,47,48. This is also evident from the feature selected by the CNN in making the ERE frequency forecast over CI (Fig. 5b). Figure 5b indicates that for 2007 forecast of the ERE frequency over CI, the CNN model monitors the development of positive (negative) D20 anomaly over the west (east) Indian ocean and negative D20 anomaly over the east equatorial Pacific and Atlantic Ocean from October 2006 to May 2007. Studies show that positive Indian Ocean dipole mode along with weak El-Nino and negative Atlantic zonal mode preconditions developed before the season results into strong ISMR which in turn could lead to the rise in the frequency of ERE48,49,50,51. Moreover, due to significant association of the off-equatorial D20 variations in the potential predictability of ERE frequency at any lead (Fig. 1d and Figure S1), the essential contribution of the off-equatorial dynamics of the Pacific and Atlantic Ocean is also indicated in Fig. 5b. Off-equatorial D20 variations are primarily linked to multi-decadal extra-tropical forcing such as Pacific Decadal Oscillation (PDO) and AMO52,53.

Fig. 5: Physical interpretation of the CNN model’s forecast.
figure 5

a Monthly normalized D20 anomaly input maps used for the forecast of ERE frequency during 2007. b Integrated Gradient (IG) maps associated with each individual month used in the forecast of ERE frequency during 2007. c Composite of October(2006) to May(2007) normalized D20 anomaly input maps used for the forecast of 2007 seasonal ERE frequency event. d Heat map of 2007 seasonal ERE frequency forecast by the CNN model. Only the values with over 95% confidence level based on Student’s t-test using the standard deviation of the IG and heat map during 1992–2022 are shaded in (b, d), respectively.

Full size image

We also analyze the model’s learning separately using heat map analysis (Fig. 5d, Text S11). Unlike IG, which presents the grid-wise contribution of each input month (Figs. 5a and 5b), the heat map represents the overall contribution from all input months combined (Fig. 5a). Positive (negative) values on the heat map indicate regions contributing to above- (below-) normal forecasts. In Fig. 5d, the positive heat map values suggest that the CNN model identifies features over the Indian, Pacific, and Atlantic Oceans as crucial for successfully forecasting above-normal seasonal ERE frequency in 2007. Hence, both IG and heat map analyses highlight the importance of the simultaneous contribution of the entire tropical basin in predicting seasonal ERE frequency, indicating the role of the G-ENSO in its predictability. The IG analysis of the CNN forecast for the 2007 ERA (Figure S6) further emphasizes the tropical basin’s collective influence on ERA over CI.

Discussion

Considering the significant socio-economic losses from rapidly increasing trends of cumulative frequency and activity of daily rainfall extremes over CI, forewarning seasonal mean ERE frequency has become imperative for disaster loss minimization and adaptive strategy planning but lacking currently.

  1. I.

    To our knowledge, our study is a first attempt in this direction. In the absence of a physical basis or potential skill estimate for forecasting seasonal ERE frequency during the Indian summer monsoon season, we first propose a hypothesis for physical basis of seasonal ERE frequency prediction. As the seasonal frequency of the extreme rainfall events during June-September are closely related to the mean Indian summer monsoon climate, we argue that the slowly varying global ocean dynamics drive seasonal ERE frequency in a manner similar to that drive the ISMR24. We propose an ERA index, analogous to the accumulated cyclone energy (ACE) for tropical cyclones, to quantify the cumulative impact of the seasonal ERE frequency. Correlations of ERE and ERA with monthly mean D20 support our hypothesis.

  2. II.

    With the physical basis in place, we estimate the potential skill for forecasting seasonal ERE frequency and ERA up to 24-month leads using a linear predictor discovery algorithm and unravel that the potential skill of seasonal ERE frequency and ERA is lower at short leads (1-4 months lead) and increases at longer leads remaining almost consistent with a mean value of ~0.85 between 5 and 22 month leads. We further demonstrate the feasibility of achieving the potential skill employing both linear and non-linear prediction techniques.

  3. III.

    We realize that while the predictors from the linear algorithm is useful in estimating the potential skill using linear regression model, due to the non-linearity in the contribution of small-scale D20 anomalies to the predictor, it may be challenging to realize the potential skill with linear prediction model, indicating that a non-linear predictor discovery technique could further increase the forecast skill. We further demonstrate skillful forecast of both seasonal ERE frequency and ERA using a deep learning model guided by the physical association between G-ENSO and seasonal ERE frequency/ERA. To demonstrate the proof of the concept, we present useful and reliable forecasts of seasonal ERE frequency and ERA 1-month in advance in this study to help policy makers and stakeholders develop adaptation strategy. The usefulness of the CNN forecast is presented with multiple skill score calculations and its ability to forecast successfully majority of seasonal ERE frequency during the testing period 1992–2022. Using IG and heat map analysis, we support our argument and demonstrate that the link identified by the CNN model are physically significant and consistent with contributions of the D20 anomalies from all three ocean basins. It is notable that our physics-guided CNN model outperforms the predictions of extremes in seasonal ERE frequency and ERA at 1-month lead as compared to a linear regression model.

While the CNN model is useful, it is certainly not perfect as it misclassifies few ERE frequency and ERA events. Most CMIP6 models are of low horizontal resolution and underestimate the Indian monsoon ERE frequency and ERA. With more ensemble members from more higher resolution CMIP6 models available to train on the nuances of G-ENSO and seasonal ERE frequency/ERA relationship, there is considerable scope in improving the CNN model’s forecast.

Methods

Extreme rainfall activity calculation

To provide a measure of destructive potential of a TC, an index called ACE is defined54. The ACE is the square of maximum sustained wind speed accumulated every six hours after it reaches tropical storm category (>= 34 knots). This can be done for a single TC or a group of TCs such as during a specific season and makes sense as the destruction of TC comes mainly from strong winds. In the case of daily rainfall extremes, damage and destruction is more associated with the heavy rain rather than heavy winds. Higher the rainfall packed in a day, higher the hydrological disaster potential. Given the uneven distribution of ERE across India7,55, it’s crucial for the region chosen to study ERE to exhibit a relatively homogeneous seasonal mean climate and daily rainfall variability. The regional extent of CI (74°E-86°E, 15°N-26°N) is well-suited for defining ERE based on a fixed threshold on summer monsoon precipitation4 (Fig. 1a, b). In terms of disaster potential, the ERE are like miniature TC. Therefore, we also define an index of extreme rainfall activity (ERA) as the sum of daily rainfall associated with seasonal ERE frequency during June-September over CI normalized by the 99.5 percentile value of average rainfall over the region (R99.5)4,8. The time series of seasonal ERE frequency and ERA between 1901 and 2022 calculated from JJAS daily rainfall data from India Meteorological Department (IMD)5 gridded at 1° x 1° resolution is shown in Fig. 1a. It is notable that seasonal ERE frequency and ERA are nearly identical in terms of variability.

The ERA during the summer monsoon season of any year t over CI is calculated using the following equation,

$${{\rm{ERA}}}_{{\rm{t}}}=\left(\mathop{\sum }\limits_{{\rm{i}}=1}^{{{\rm{F}}}_{{\rm{t}}}}{{\rm{R}}}_{{\rm{i}}}\right)/{{\rm{R}}}_{99.5}$$
(1)

Where, R99.5 is the mean value of the 99.5 percentile rainfall over the period under consideration generated from JJAS daily rainfall data over CI, Ft is the aggregate number of daily rainfall events during JJAS exceeding R99.5, and Ri is the daily rainfall value during JJAS over CI exceeding R99.5 value. As an ERE is defined as daily rainfall exceeding R99.5, the seasonal ERE frequency represents the total number of ERE events during a given JJAS season. The ERA represents ERE events in terms of base units of R99.5. Again, higher this number, higher the hydrological disaster potential. As expected this number is always larger than the accumulated frequency of ERE events during the season (Fig. 1a, b). It is notable that the interannual variations of ERA and ERE frequency are nearly identical indicating that the proportion of very heavy and very small ERE remain universal. This makes the frequency itself a good measure of ERE activity, a fact not obvious to us at the beginning. The anomalies of seasonal ERE frequency and ERA are calculated by removing the non-linear trends.

Predictor discovery algorithm

Using D20 anomalies gridded at 0.25° x 0.25° over 0°-360°E, 30°S-30°N from Ocean Reanalysis System 5 (ORAS5)56,57 and seasonal frequency of ERE and ERA from the daily rainfall dataset of India Meteorological Department (IMD)5 gridded at 1° x 1° between 1958 and 2022, we generate Dp predictors up to 24-month leads (Fig. 6). Considering the seasonal ERE frequency and ERA during June-September over CI between 1960 and 2022, for each lead month, the Dp predictors are generated by projecting the D20 anomaly of the corresponding month onto the correlation pattern between that month’s D20 anomaly and the seasonal ERE frequency/ERA. For instance, the Dp for lead 1 is obtained by projecting the May D20 anomaly from 1960 to 2022 onto the correlation between the May D20 anomaly and the seasonal ERE frequency/ERA for the same period. Similarly, the Dp for lead 6 is derived by projecting the December D20 anomaly from 1959 to 2021 onto the correlation between the December D20 anomaly and the seasonal ERE frequency/ERA for 1959-2021 and 1960–2022, respectively. The correlation between Dp and seasonal ERE frequency/ERA as a function of lead month is shown in Fig. 1c.

Fig. 6
figure 6

Schematic diagram of the predictor discovery algorithm.

Full size image

The lead-lag correlation maps between the D20 and seasonal ERE frequency anomalies indicate the different phases of evolution of the G-ENSO. The canonical pattern of the G-ENSO and its evolution with time, as associated with the variability between D20 and seasonal ERE frequency is shown in Fig. 7. The similarity between the G-ENSO patterns associated with ERE frequency anomalies (Fig. 7) and those linked to ISMR24 supports our argument that the predictability and seasonal forecasting of ERE frequency during Indian summer monsoon season are primarily influenced by the slowly varying G-ENSO dynamics, similar to the mechanisms governing the predictability of the ISMR. This also justifies the use of the Sharma et al.24 predictor discovery algorithm in this study.

Fig. 7: Empirical Orthogonal Function (EOF) analysis of the correlation between D20 and seasonal ERE frequency anomalies up to 24-month leads.
figure 7

a EOF 1, (b) EOF 2, and (c) Principal components of EOF 1 and EOF 2.

Full size image

Physics-guided deep learning model

Our CNN has one input layer, six 2-dimensional convolutional layers, two 2-dimensional average pooling layers, and two dense layers. Normalized tropical D20 anomaly maps from October(-1) to May(0) are stacked together into an array of dimensions 144x48x8 (longitude x latitude x month) and fed as input into the CNN. For each month, the normalized D20 anomaly for training, validation, and testing is determined by subtracting the climatology and dividing by the standard deviation at each grid point over the training, validation, and testing period, respectively. The CNN is trained over the relationship of monthly D20 anomalies across the tropics from October(-1) to May(0) with the seasonal frequency of daily rainfall events larger than 99.5 percentile value over CI during JJAS from a large sample of such relationships simulated by the CMIP6 models between 1850 and 2014 (Table S1). There are 32 filters in the first two convolutional layers (Conv2D) and 16 in the last four. The filter dimensions for feature extraction is kept at 3 x 3 with tanh activation function in all the convolutional layers. The receptive field of the average pooling layers (AvgPooling2D) is kept at 2 x 2 with stride 2. The output from the final convolutional layer is flattened and linked to a series of 64 hidden nodes of the first dense layer (Dense) with sigmoid activation function. The final dense layer generates the forecast of the seasonal ERE frequency (ERA) using linear activation function. To prevent overfitting L2 regularization is implemented in all the convolutional and dense layers. Moreover, each convolutional layer is followed by a batch normalization layer (BN) and one dropout layer (DP) with drop rate 0.5 is used between the dense layers. The parameters of the CNN are tuned using the mean absolute error function and correlation coefficient metric optimized by the Adam optimizer with learning rate 0.0001, 300 epochs, and batch size of 400. The parameters of the CNN are finalized based on its performance on the validation dataset, which consists of D20 and seasonal ERE frequency/ERA anomalies from ORAS5 at 2.5° × 1.25° and IMD at 2° × 2° resolution, respectively, between 1958 and 1988, using an early stopping model callback algorithm. To improve the performance, robustness and generalization of the CNN, ensemble training technique is employed. A ten-member ensemble is created by selecting models with at least 95% confidence level forecast skill on the validation dataset. The ensemble mean forecast of the ten members on the testing dataset, consisting of D20 and seasonal ERE frequency/ERA anomalies from ORAS5 gridded at 2.5° × 1.25° and IMD gridded at 2° × 2°, respectively, between 1992 and 2022, is presented as the final prediction. Our CNN architecture is inspired from the CNN model proposed by Ham et al.57 for detecting climate change signals. However, we have introduced several key modifications to make the model suitable for our specific application. These modifications include the addition of average pooling layers, an extra convolutional layer, batch normalization layers, 32 extra nodes in the first dense layer, and a dropout layer between the two dense layers. These adjustments improve the model’s robustness and performance. With the training on CMIP6 model simulations and validation and testing done on observed data, the overfitting problem is avoided.

Relative Operating Characteristics (ROC)

ROC is a measure of the performance of a forecast model based on classification thresholds. The area under the ROC curve (AUC) represents the accuracy of the model’s classification skill. Larger the AUC, higher is the model’s classification skill and vice versa. The ROC curve is made using false positive rate (FPR) as the x-axis and true positive rate (TPR) as the y-axis.

$${\rm{FPR}}=\,\frac{{\rm{FP}}}{{\rm{FP}}+{\rm{TN}}}$$
(2)
$${\rm{TPR}}=\,\frac{{\rm{TP}}}{{\rm{TP}}+{\rm{FN}}}$$
(3)

where, for each defined classification threshold true positive (TP), false positive (FP), true negative (TN), and false negative (FN) are calculated from a 2×2 contingency table (Table 2). Given a threshold, the True Positive (TP) count increases when the model correctly predicts events that exceed the threshold. The False Positive (FP) count increases when the model incorrectly predicts events as exceeding the threshold. The True Negative (TN) count increases when the model correctly predicts events that do not exceed the threshold. The False Negative (FN) count increases when the model incorrectly predicts events as not exceeding the threshold.

Table 2 Contingency table for ROC calculation
Full size table