Optimal life-cycle adaptation of coastal infrastructure under climate change

Abstract

Climate change-related risk mitigation is typically addressed using cost-benefit analysis that evaluates mitigation strategies against a wide range of simulated scenarios and identifies a static policy to be implemented, without considering future observations. Due to the substantial uncertainties inherent in climate projections, this identified policy will likely be sub-optimal with respect to the actual climate trajectory that evolves in time. In this work, we thus formulate climate risk management as a dynamic decision-making problem based on Markov Decision Processes (MDPs) and Partially Observable MDPs (POMDPs), taking real-time data into account for evaluating the evolving conditions and related model uncertainties, in order to select the best possible life-cycle actions in time, with global optimality guarantees for the formulated optimization problem. The framework is developed for coastal adaptation applications, considering a wide variety of possible action types, including various forms of nature-based infrastructure. Related environmental impacts of carbon emissions and uptake are also incorporated, and social cost of carbon implications are discussed, together with several future directions and supported features.

Introduction

Climate change is impacting every region worldwide, with human activities serving as major drivers behind the observed shifts in climate patterns and extreme weather events¹. Greenhouse Gas (GHG) emissions are causing a rise in global temperatures, leading to Sea Level Rise (SLR) through land ice loss and thermal expansion from ocean warming^2,3. The acceleration of SLR can pose critical risks and consequences to coastal communities, including permanent submergence of land⁴, accelerated coastal erosion and scour of coastal infrastructure systems⁵, loss of vital coastal ecosystems, and a higher frequency and severity of extreme events like floods⁶, and storm-induced surges⁷. Furthermore, these SLR-related risks are projected to escalate substantially by the end of this century, compounded by continuing development along low-lying coastlines⁸. Consequently, there is an urgent need for effective and well-designed coastal flood protection management policies to mitigate these increasing risks.

A variety of flood protection strategies are available, which typically include the construction of physical barriers, preservation and enhancement of natural barriers, retreat or relocation of development, floodproofing of buildings, and installation of active measures such as pumps, among others⁹. Owing to the substantial uncertainties associated with the evolving climate demands and the limited resources for implementing various flood protection strategies at a given point in time, it is desirable to determine the optimal flood management policy. This can involve a synergistic implementation of different flood protections in time, minimizing the present total life-cycle utility value, usually expressed in monetary units, that can include expected flood damage costs and risks, implementation and associated maintenance costs of flood protection measures, and any other desired quantities of interest.

The identification of the optimal flood management policy is non-trivial, in general, primarily due to various levels of associated uncertainties^10,11, which can include: (i) aleatoric (irreducible) uncertainty associated with the natural variability of climate change processes and extreme events; (ii) epistemic (reducible) uncertainty associated with imperfect knowledge of the underlying climate scenario models; related also to (iii) uncertainty regarding future knowledge and updates of climate models; (iv) epistemic uncertainty associated with the evolution of future trajectories given assumed climate scenario models; and (v) uncertainty associated with available observations in time. Depending on whether and how observations and information in time are used to guide policy plans and lead to more-informed decisions, the decision-making frameworks available in the literature can be broadly categorized into static/non-adaptive (no utilization of additional information in time) and dynamic/adaptive (additional information is utilized in time).

A conventional approach for climate change planning has been primarily driven by cost-benefit analysis (CBA), particularly when federal and national agencies and authorities are concerned^12,13,14. Typical CBA solves a static optimization problem where the optimal time of investments and the optimal infrastructure-related actions are determined by maximizing the sought-after benefits while minimizing the related investment costs over the planning horizon. Most of the work in the existing literature implements such static optimization frameworks, often within settings focused solely on levee heightening. The expected flood damages within such frameworks are determined by assuming a fixed base-case scenario of the evolution of the probability of failure¹⁵ or by considering an ensemble range of future SLR and storm surge events to be representative of the range of possible future scenarios^{11,16,17,18,19,20,21}. Different solution techniques have been utilized to solve these static optimization problems, including genetic algorithms¹⁶ and non-linear programming¹⁵.

The result of the static optimization approach, as defined here, is a single policy which is deemed optimal over the average, or any chosen percentile, of potential future climate trajectories; and this policy is to be implemented without further guidance by the actually evolving climate conditions observable in time. However, the actual climate trajectory may deviate considerably from the ensemble mean (or the chosen target percentile) owing to the substantial amount of uncertainty present in climate model projections¹, as illustrated in Fig.  1A, which inevitably means that the identified policy is highly likely to be sub-optimal with respect to the actual climate trajectory that evolves in time. Thus, static optimization approaches solely rely on simulation results and do not utilize actual observations in time to update the knowledge regarding the associated risks and any of the (ii)-(v) related aforementioned uncertainties.

Fig. 1: Underlying uncertainty based on different decision-making approaches.

A Range of uncertainty associated with the climate projection model based on IPCC AR6 (SSP5-8.5)¹ from the present time step. s.d stands for standard deviation. B Related uncertainty bands associated with sequential decision-making at each decision step.

There are a few works in the literature offering adaptive planning approaches, as defined previously, which determine policy actions depending on the evolving, observed climate conditions in time. The existing literature on adaptive planning of infrastructure systems against climate change typically involves methodologies including direct policy search (DPS)^22,23, real options analysis (ROA)^24,25, and stochastic control methods^{26,27,28,29,30}. With DPS, the proposed actions are condition/state-dependent, as observed in time; however, the expressions relating the observable state variables to the triggering actions are action-specific and derived based on exploratory analyses or expert judgment, complicating somewhat the extension of DPS to several different action types and applications. DPS solutions are based on heuristic searches and thus lack optimality guarantees, while also exploring only a relatively limited subset of the policy space, which is captured by the state-action expressions. ROA also offers an adaptive planning framework, typically based on binomial decision trees with two possible actions/options at each node²⁵. The action at each node is determined by computing action values backward in time. This analysis represents a form of backward induction that can quickly become computationally intractable as the number of branches increases exponentially with the number of time steps, actions, and observations. Alternative approaches have also been suggested through adaptive pathway solutions^31,32,33, identified through explorative modeling of adaptation tipping points in time. These methods rely on forward simulations/trajectories of possible futures in time to identify the tipping points³³, sharing many of the limitations of the cost-benefit and DPS methods and resulting in the adaptive pathway solutions likely to be sub-optimal for the actual evolving conditions in time. Overall, these adaptive methods have been typically applied in levee construction/heightening applications, where a single levee exists or is constructed at one point in time and is then subsequently heightened in future time steps.

Examples of adaptive planning optimization against climate change effects using stochastic control frameworks can be found in the relatively recent literature, where the sequential decision-making process is formulated as a Markov Decision Process (MDP) or Partially Observable MDP (POMDP), with state variables typically representing the related conditions, and decisions being the possible infrastructure actions. The MDP/POMDP solution determines optimal actions to be taken at each decision time-step, conditioned on the observed, or inferred for POMDPs, system states, and thus uniquely selected for the actual realized trajectory path, successfully optimizing the total overall expected costs/rewards over the decision horizon. MDP/POMDP approaches can offer superior and more general solutions, using the same models, with respect to other static and/or adaptive planning methods, as described above, since they are based on dynamic programming and closed-loop stochastic control foundations³⁴, that offer robust guarantees of global optimality of the action paths. A few notable works based on the MDP framework, also for levee heightening problems, can be seen in^26,27,28,35. Some of these works importantly also consider climate model uncertainty utilizing POMDPs and simple hydrological scenarios^27,35. However, the main limitation of standard MDP/POMDP solution approaches is the curse of dimensionality^36,37, which often dictates an overly simplified representation of the system, and/or a crude approximate solution technique that does not offer optimality guarantees. The existing related methodologies in the literature largely follow these patterns and are quite specific to the characteristics of their particular application scenarios, with relatively few possible states at each point in time, while also employing approximate solution techniques, all in order to make a solution to the problem feasible. Utilization of their solution framework to other applications, and/or extensions is thus not straightforward, particularly also when a variety of possible actions needs to be considered.

In this work, we are presenting extensions and capabilities for adaptive planning approaches and a general and sophisticated solution framework for the optimal life-cycle adaptation of infrastructure under climate change. We are relying on the firm mathematical foundations of closed-loop stochastic control through MDPs /POMDPs, while offering general, easily adaptable, and extensible implementations of infrastructure adaptation problems under climate change, with higher-fidelity representations in comparison to currently available existing approaches. We utilize the current Intergovernmental Panel on Climate Change (IPCC) models¹ in our framework, accommodating and presenting a wide variety of possible action types, such as floodwalls, nature-based infrastructure, and seawalls, considering model uncertainty aspects, and incorporating the social cost of carbon considerations in the analyses. Simultaneously, and in contrast to currently available relevant approaches in the literature, we are formulating the entire framework in such a way so that it can be generally supported by state-of-the-art, efficient MDP and POMDP optimization solvers with global optimality guarantees, based, of course, on the modeling assumptions used in the optimization process. We, hence, enable both accurate solutions and a finer representation of the problem settings based on higher-dimensional state-spaces and time/history dependent risks and costs, among others. Our suggested framework can also, in principle, accommodate all types of (ii)-(v) uncertainties mentioned earlier. As illustrated in Fig.  1B, the integration of up-to-date annual information in relation to the current environment conditions substantially reduces the type (iv) uncertainty concerning the evolution of the actual evolving trajectory until the next observation becomes available. Although type (iii) uncertainty regarding future knowledge and updates of climate models is the only type that has not been directly addressed in this work, specific ways to incorporate these aspects in the framework are explained later in the Discussion section, together with still existing limitations and further possible future directions.

The environmental impacts of coastal flood protection strategies have so far often been inadequately factored into the decision-making process, either being completely ignored or not given the appropriate consideration. Recent research has highlighted the economic costs of coastal flooding and the long-term benefits of flood protection infrastructure^38,39,40, however, only a limited number of studies have quantified the life-cycle environmental impacts of such systems^41,42. Our work also aims to fill this gap by presenting a dynamic decision-making framework that also considers GHG emissions from a life-cycle perspective. The framework thus notably contributes to the planning of flooding risk mitigation by offering a more holistic approach that considers both economic and environmental dimensions of coastal flood protection.

To summarize, this study makes the following contributions to the planning of climate-change-related risk mitigation:

• Presents the foundational elements of a general framework for life-cycle adaptation under climate change based upon the stochastic control methodologies of MDPs and POMDPs. The framework is sufficiently general to be readily extended to consider a wide range of action types and is fully compatible with state-of-the-art MDP/POMDP solvers with global optimality guarantees.

• Formulates a general POMDP setting by considering climate model uncertainties updated in time through relevant observations.

• Considers environmental impacts, the social cost of carbon, and related carbon emissions and uptake in the decision-making process.

• Accounts for various nature-based infrastructure types as potential flood risk mitigation measures and demonstrates their effects in mitigating carbon emissions.

Results

General problem formulation of coastal flood risk mitigation

In general, a MDP represents a controlled stochastic process defined by the 5-element tuple 〈S, A, P, R, γ〉⁴³. The decision maker observes the true state of the system s ∈ S, takes an action a ∈ A from a finite set of possible actions A, receives a reward or cost R(s, a), and, as a result of taking this action, the system transitions to a new state $s^{\prime}$ according to a stochastic model with Markovian transition dynamics $P(s^{\prime} | s,a)$, as shown schematically in Fig.  2.

Fig. 2: Sequential decision-making as a Markov Decision Process.

At each time step the action A taken is determined by the current state, s. The reward, R, depends on the current state and the action taken. The next state is determined by the Markovian transition dynamics of the underlying model, M, given the current state-action pair.

In the context of optimal adaptation of coastal infrastructure systems subjected to risks posed by storm surges and sea level rise in this paper, the states of our formulated MDP include the discretized SLR and storm surge levels, with the SLR states evolving probabilistically through transition dynamics computed from climate projections taken from the recent IPCC report (AR6)^1,44. The MDP states additionally include the infrastructure related states, since the damage induced at any point in time is dependent not only on the SLR and storm surge levels but the implemented system protections as well. Unlike the existing, more simple, adaptive policy settings of levee heightening, where the level of protection at any time is fully determined by the height of the levee, in our presented general setting, the infrastructure state and protection are determined by the entire history of the previously executed adaptation actions up to the current time step, given that these actions can completely now alter the system configuration. Although this essential characteristic for this type of problems is non-Markovian, given that the entire history of previously executed actions is now relevant, this issue has also been addressed in this work by appropriately and originally augmenting the state space in such a manner so as to enforce a Markovian formulation, which is also importantly generally consistent with state-of-the-art dynamic programming solvers.

Formulating optimal flood risk management as a MDP provides strong global optimality guarantees for long-term objectives, within the optimization problem posed, through dynamic programming principles. The long-term objective, also known as the value function V, is expressed as an expected sum of γ discounted rewards (or costs) generated from a given initial state s and following a chosen policy π, expressed as:

$${V}_{\pi }\left(s\right) :={E}_{\pi }\left[{\sum }_{k=0}^{\infty }{\gamma }^{k}{R}_{t+k}| {S}_{t}=s\right]$$

(1)

The state-dependent sequence of actions defines an agent’s policy π : S → A, and the goal is to find the optimal policy that maximizes the value function. The optimal value function V* under the optimal policy π^* from a given system state s and for an infinite horizon can be shown to follow the recursive Bellman⁴³ equation:

$${V}{*}\left(s\right)={\max }_{a\in A}\left[R\left(s,a\right)+\gamma {\sum}_{s^{\prime} \in S}p\left(s^{\prime} | s,a\right){V}{*}\left(s^{\prime} \right)\right]$$

(2)

In our optimal flood management setting, related costs at each time-step consist of flooding risks, maintenance of current flood protection assets, and implementation costs of new flood protection measures associated with the action being taken. Thus, the value function in Eq. (2) can be expanded as:

$${V}{*}\left(s\right)= {\max }_{a\in A}\left[\left\{{R}_{i}\left(s,a\right)+{R}_{m}\left(s,a\right)+\gamma {\sum}_{s^{\prime} \in S}p\left(s^{\prime} | s,a\right){R}_{f}\left(s^{\prime} \right)\right\}\right.\\ \left .+\gamma {\sum}_{s^{\prime} \in S}p\left(s^{\prime} | s,a\right){V}{*}\left(s^{\prime} \right)\right]$$

(3)

where R_i, R_m and R_f are costs associated with implementation of new flood measures, maintenance of existing measures, and flood-induced damages, respectively. Carbon emissions associated with each one of these components and related implications are also considered in our framework. More details can be found in the Methods section.

In our suggested framework, the SLR and total peak water levels are fully observed in time, while in our POMDP formulation the underlying climate models, represented by the Shared Socio-economic Pathways (SSPs) provided in the most recent assessment report (AR6) of the Intergovernmental Panel on Climate Change (IPCC)¹, are hidden state variables which are only probabilistically inferred based on the SLR observations in time. A POMDP is thus denoted by the tuple 〈S, A, P, O, P_O, R, γ〉, where O represents the set of possible observations and P_O is the likelihood function related to the observations. A belief b is formed and updated over the related climate models in time, representing a probability distribution over the considered SSP models and a sufficient statistic of the history of past SLR observations regarding model uncertainty. Hence, the decision-making problem in a POMDP setting starts with an initial belief over the considered models, and at each time step an action is taken and an observation is made, with the belief over the models updated by Bayes’ rule, as:

$$b\left(s^{\prime} \right)=\frac{p\left(o| s^{\prime},a\right)}{p\left(o| {{{\bf{b}}}}_{{\bf{s}}},a\right)}{\sum}_{s\in S}p\left(s^{\prime} | s,a\right)\,b\left(s\right)$$

(4)

with $p\left(o| {{{\bf{b}}}}_{{\bf{s}}},a\right)$ being the normalizing constant:

$$p\left(o| {{{\bf{b}}}}_{{\bf{s}}},a\right)={\sum}_{s^{\prime} \in S}p\left(o| s^{\prime},a\right){\sum}_{s\in S}p\left(s^{\prime} | s,a\right)\,b\left(s\right)$$

(5)

Relevant to the MDP case, the optimal value function for infinite horizon POMDPs also follows a recursive Bellman equation⁴⁵, as:

$${V}\,{ * }\,\left({{{\bf{b}}}}_{s}\right)={\max }_{a\in A}\left\{{\sum }_{s\in S}b\left(s\right)\,R\left(s,a\right)+\gamma {\sum}_{o\in O}p\left(o| {{{\bf{b}}}}_{{\bf{s}}},a\right){V}\,{ * }\,\left({{{\bf{b}}}}_{s^{\prime} }\right)\right\}$$

(6)

Due to the continuity of the belief space, POMDP problems are in general more difficult to solve than MDPs⁴⁶. However, the optimal value function over the belief space can be shown to be piecewise-linear and convex⁴⁷, comprising a finite number of hyperplanes. This attribute reduces the problem to determining a finite set of hyperplanes, also referred to as α-vectors. Each such vector is associated with a unique optimal action and the optimal POMDP value function can then be obtained using dynamic programming and α-vector backups^45,46,48,49. In this work, the problem is generally set up so that state-of-the-art point-based, asynchronous dynamic programming solvers can be utilized, able to provide solutions with global optimality guarantees for both MDP and POMDP problems, such as the Focused Real-Time Dynamic Programming (FRTDP) algorithm⁵⁰ specifically utilized here. More details are provided in the Methods section.

Sea level rise and storm surge processes

In the coastal flood management problem, the MDP state variables are defined as the possible total peak water levels that can be realized in time. In this work, the total peak water level incorporates the combination of local sea level rise and storm surge (from hurricane events, for example).

The SLR model is derived based on regional SLR projections using Coupled Model Intercomparison Project Phase 6 (CMIP6) models from the AR6 of the IPCC¹ for the Battery tidal gauge⁵¹, representing the general NYC area, without loss of generality. The SLR projections extending to 2150, relative to the 1995-2014 reference period, are obtained from^44,52,53. These projections are based on various climate change and emission scenarios, represented by distinct SSPs. SLR projections under three such SSPs are shown in Fig.  3A. For each SSP, an ensemble of 21 Global Climate Models (GCM) is considered in this work, in an effort to consider epistemic model uncertainties within a SSP scenario. A future SLR trajectory follows here a certain projection percentile within the likely range, which captures the long-term non-stationary SLR trend resulting from multiple factors, such as thermal expansion, ice sheet contributions, and glacial melt. However, these projections do not include the short-term variability inherent in the historical observations, which are contributions of various natural fluctuations. In order to also consider these variations in the future sea level rise projections, a regression model is fit to the observed historical data provided by the National Oceanic and Atmospheric Administration (NOAA)⁵⁴, as shown in Fig.  3B, and the random noise present in the observations is utilized, assuming to follow a Gaussian distribution with a constant standard deviation of 3.18 cm, as also confirmed in the normal probability plot in Fig.  3B. Indicative resulting sea level rise process simulations derived from the IPCC climate model projections, along with the historically observed variability, are illustrated in Fig.  3C. The time-variant and uncertain sea level rise evolution is modeled by a Markov process with transition dynamics estimated based on these simulated trajectories. Without any loss of generality, any sea level rise transition model can also be similarly utilized in the presented framework, including those accounting for large-scale uncertain events like rapid ice sheet collapse, as considered in¹¹.

Fig. 3: Sea level rise and storm surge stochastic models.

A Regional sea level change in New York City region (msl denotes mean sea level). The likely range indicates the 5th to 95th percentiles. B Observed variability in historical observations, the polynomials fitted to the observations, and sample hindcast realizations with observed variability distributed according to Gaussian distribution. C SLR realizations based on IPCC SSP2-4.5 projections along with the variability present in past observations. D GEV probability density function of the storm surge model.

Storm-driven surges are annual maximum surge heights simulated here based on a generalized extreme value (GEV) distribution. The parameters of the GEV distribution are determined by maximum likelihood estimation on the observed annual maximum tidal gauge readings at the Battery tidal gauge after removing the local sea level rise trend^55,56. The estimated parameters for the Battery tide gauge are μ = 0.936m, σ = 0.206m, and ϵ = 0.232, which results in a surge model with an estimated 100-year surge level of 2.62 m above the Mean Higher High Water (MHHW) datum, and its distribution is shown in Fig.  3D. This estimated 100-year surge level is reasonably consistent with other estimates available in the literature^57,58.

The local SLR modeled as described above captures the most prominent non-stationary effects of climate change. On the other hand, possible effects of climate change on intensification of storm surges are not yet fully understood and require further analysis. As such, the storm surge model used in this study assumes stationarity with a constant distribution over the future. However, the proposed framework is sufficiently flexible and can straightforwardly incorporate non-stationary storm surge models as the science on this issue develops. For example, by incorporating non-stationary Generalized Extreme Value (GEV) parameters, dependent on SLR states or not, the framework can capture increasing 100-year surge levels over time⁵⁶.

Coastal city setting

To illustrate the adaptive risk mitigation framework in the context of flood management, we consider two application scenarios in this paper - an idealized coastal city setting with limited space near the coastline (inspired by Manhattan) that practically precludes nature-based infrastructure options, like oyster reefs and salt marshes, without substantial transformation, and a coastal community setting (inspired by Staten Island) where the construction of such nature-based infrastructure are more viable choices. The coastal adaptation problem is modeled here as an MDP, where the states (S) correspond to the total peak water levels observed every year with certainty. These states guide the selection of the actions A over time. In all the experiments in this work, the SSP 2-4.5 scenario is chosen to predict the future SLR trajectories, unless otherwise stated. In this section, the coastal city setting is described, together with the set of available flood protection actions and related results.

The idealized coastal city on a rising shoreline inspired by Manhattan (Fig.  4A) is first described. For the purpose of this study, a representative unit width of the city parallel to the coastline and multiple flood protection measures are considered. Due to the exponentially increasing state space with the number of possible strategies, as explained in detail in the Methods section, the representatively considered MDP policies presented here consist of two possible dynamic actions that can be implemented adaptively in time. Scaling up of the state and action spaces will at some point eventually lead to cases that cannot be solved with global optimality guarantees. In such cases, deep reinforcement learning-based solution methods can then be employed^59,60, which have shown superior capabilities to a vast array of learning and planning problems across a variety of disciplines, albeit at the cost of loss of global optimality guarantees.

Fig. 4: Coastal city setting.

A View of the South Street Seaport area of Manhattan, NY, that inspired the coastal city setting. Courtesy Google Maps. (Maps data ©2024 Google). B Idealized topology of a coastal city inspired by Manhattan with two floodwalls (F1 and F2) as adaptive flood protections. Zone 1 below the base of the first floodwall is unprotected. Zone 2 is protected by the first floodwall F1 and is damaged only when F1 is over-topped. Zone 3 lies above the top of F1 and below the base of F2. Zone 4 is again protected by F2 until over-topping occurs. Zone 5 continues from the top of F2 till the highest elevation point of the city.

In the discussed city setting, two distinct flood management scenarios are studied, each with its own characteristics and flood protection measures. The first scenario explores the implementation of two fixed-height floodwalls at different city elevations, in the absence of pre-existing flood protection buffers, like a seawall, as illustrated in Fig.  4B. These floodwalls serve as dynamic strategies to protect against rising flood levels. In the second scenario, which assumes the presence of an existing seawall, the option of re-purposing the land located next to the shoreline into a multi-functional green zone is considered. Such a green zone offers several advantages, including lower construction and maintenance costs, and possesses carbon uptake properties, contributing to the sequestration of carbon dioxide from the atmosphere. By examining these scenarios the analysis aims to identify effective flood management strategies in the city setting, considering the unique challenges and opportunities presented by each situation.

For the first scenario, sample realizations of computed MDP policies are shown in Fig.  5A. With the SLR process being a non-stationary evolving process, the sequential decisions to be taken in time are hence primarily driven by the SLR. As seen in Fig.  5A, the floodwall at the higher elevation is erected only when an extreme level of SLR is observed. The floodwall at the lower elevation is constructed at the beginning of the planning horizon, highlighting the urgency of flood risk mitigation actions in this setting. The immediate action of constructing the floodwall at the lower elevation can be understood by the high vulnerability of the coastal city, with the highest possible flood damage being $ 1 million per meter of shoreline, when the flood height reaches the highest elevation of the city at an assumed 8.5m. In this scenario, the absence of pre-existing flood protection assets, such as seawalls or wetlands, contributes to the vulnerability of the city to any level of storm surge. Consequently, the adaptive solution promotes early interventions to mitigate the impact of floods.

Fig. 5: Policy for the coastal city setting with two floodwall options.

A Two different policy realizations where the second floodwall is constructed only when the observed sea level rise (SLR) process progresses more rapidly (left panel). The shaded region associated with the SLR process represents the uncertainty over the next time step from the current time step. B Overall expected normalized costs of MDP-based policy compared against the static baselines: DN-no measures taken, F1- only floodwall F1 constructed, F2- only floodwall F2 constructed, and F1, F2- both floodwalls constructed at the beginning of planning horizon. C, D Decomposition of monetary and carbon costs respectively. Total (construction + maintenance + flood damage), construction, maintenance and flood damage expected costs are in $ (construction and maintenance costs are amplified 10 times for visualization purposes).

The comparison of expected costs in all considered settings has been performed based on 1000 Monte Carlo realizations of the computed policies, which are deemed sufficient based on the low standard errors of the result statistics, as shown in Table 3 in the Supplementary Materials. The overall cost achieved by MDP policies in this scenario is compared in Fig.  5B with that of static baselines that utilize various combinations of available individual flood protection measures at the beginning of the management horizon and represent predetermined strategies without any adaptation based on observed changing conditions. In comparison to the static benchmarks, the total overall (monetary and carbon) costs of the MDP policy are the lowest, as guaranteed also theoretically, due to the global optimality feature of dynamic programming solutions, resulting in a reduction of over $ 0.03 million per meter coastline compared to the closest static solution. The MDP policies dynamically adapt and optimize flood management actions over time based on the actual observed water levels.

The expected monetary costs associated with each considered policy are decomposed into three relevant cost categories: construction, maintenance, and flood damages in Fig.  5C. To show the importance of including carbon costs in the decision-making framework, the carbon cost associated with each cost category is shown in Fig.  5D. The carbon cost in each category is calculated as the sum of social carbon costs accumulated over the future horizon, from the time of carbon emission in the atmosphere. Considering the accumulation of carbon emissions and the projected increase in the social cost of carbon in the future, as indicated by⁶¹, it becomes evident that the carbon costs become comparable to the monetary costs (Fig.  5C) and hence can play a substantial role in driving adaptation policies.

In another scenario inspired by Manhattan, we consider a setting where an existing seawall of a fixed height is already in place. In this context, we explore the possibility of re-purposing the land adjacent to the shoreline into a multi-purpose green zone. Along with the green zone, a floodwall located at a higher elevation is considered another possible adaptive flood protection action. The static seawall positioned at the community’s edge provides protection against water levels rising up to 1.2 meters. This static seawall now acts as an initial line of defense against flood events which was absent in the previous setting with the two floodwall choices. In this setting, the policy realizations consistently indicate the adaptive construction of the green space in response to rising water levels. It is to be noted that the floodwall at the higher elevation typically does not need to be erected in this case. Some representative policies are shown in Fig.  6A, where the green zone solution gets optimally constructed in time according to the evolving conditions.

Fig. 6: Policy for the coastal city setting with green space and floodwall options.

A Two policy realizations with an existing seawall scenario where the green zone is constructed adaptively. B Overall expected costs of MDP-based policy compared against the static baselines: DN-no measures taken, GR- only green resistance zone constructed, F- only floodwall constructed, and GR, F- both measures constructed at the beginning of the planning horizon. C, D Decomposition of monetary and carbon costs respectively. Total (construction + maintenance + flood damage), construction, maintenance and flood damage expected monetary costs are in $. The corresponding expected carbon costs along with uptake (rewards) are also in $.

By dynamically adjusting the construction of the green space according to water level changes, the adaptive policy effectively optimizes the use of resources while maintaining adequate flood protection, as is evidenced in Fig.  6B, where the normalized total policy costs of different policies are shown. The MDP policy results here in savings over $ 0.004 million per meter of coastline compared to the closest static solution. Figure 6C and D present a breakdown of monetary and carbon costs, respectively, for this scenario. Trees and grasses in the green zone can also assimilate carbon and remove it from the atmosphere. As a result, an estimated value of $ 0.007 million per meter of coastline is attributed to the expected total carbon costs removed through this carbon uptake process over the planned horizon. Indeed, this carbon uptake capability of the green zone, coupled with the relatively low carbon emissions associated with its construction and maintenance processes, brings an additional advantage of reducing the overall carbon costs.

Coastal community setting

In this setting, a coastal low-lying area, inspired by Staten Island, NY is considered, as illustrated in Fig.  7. This coastal community setting represents a notably flatter terrain compared to the urban coastal city inspired by Manhattan. Although more susceptible to flood inundations, the associated damages are less than those of the coastal city setting, as indicated by the parameters in Tables 1 and 2 in the Supplementary Materials. Again, a representative unit width of the coastal community parallel to the coastline is considered.

Fig. 7: Coastal community setting.

A Staten Island, NY coast (credit: Shannon McGee⁹⁰, published under CC BY-SA 2.0 License). B A representative coastal community setting inspired by Staten Island with flatter topology than the city setting. The nature-based solution is primarily situated in the estuarine zone. The floodwall F protects the zone located behind it, which is damaged only when F is overtopped. C Salt marsh nature-based infrastructure (credit: US Fish and Wildlife Service⁹¹, published under CC Public Domain Work License). D Oyster reef nature-based infrastructure (credit: US Fish and Wildlife Service⁹², published under CC BY 2.0 License).

Within the context of this coastal community setting, the illustrative dynamic actions considered consist of the construction of a fixed-height floodwall, at a predefined elevation of 1.5 m, and a selection of nature-based infrastructure. The floodwall serves the same purpose as described in the coastal city setting. The nature-based infrastructure considered here are a constructed or restored salt marsh and a constructed oyster reef. Representative photographs of these mitigation options are shown in Fig.  7C and D, respectively. These types of nature-based options are able to provide attenuation of wave energy and height due to the increased drag as water moves around and through them. These green infrastructure options are assumed to be primarily in the estuarine zone beyond the community’s edge, where the salt marsh is 100 m in length (in the direction normal to the coastline into the ocean, away from the community’s edge) and the oyster reef is 5 m. Wave attenuation is specified as a function of total water level, based on field measurements in a salt marsh and constructed oyster reef⁶² (details presented in Table 8 in the Supplementary Materials).

In the scenario with salt marsh, sample policies realized are shown in Fig.  8A. In the scenario with oyster reefs, sample policies realized are shown in Fig.  9A. The policies realized under the two scenarios are similar in that the floodwall is constructed when the sea level is observed to reach a level of about 0.5 m. This observation is reasonably consistent with the positioning of the floodwall, which is placed at a height of 1.5 m. The normalized total policy costs of different policies in the two scenarios are shown in Figs.  8B and 9B, respectively. Figures 8C and 9C show the expected monetary costs decomposed into the relevant cost categories in the two scenarios, respectively. Similarly, the corresponding carbon costs for the two scenarios are shown in Figs.  8D and 9D, respectively. Comparing Figs. 8 and 9, it can be seen that implementation of the oyster reefs solution is associated with lower overall costs, due to greater effectiveness in this case in attenuating flood heights, as indicated by the values in Table 8 in the Supplementary Materials, and thus reducing flood damages. On the other hand, oyster reefs have negligible carbon uptake potential compared to coastal salt marshes, as indicated by the values in Table 9 in the Supplementary Materials. However, the carbon uptake benefits of the coastal salt marsh are not sufficient to overcome the greater flood-induced damages experienced due to lower wave attenuation capabilities.

Fig. 8: Policy for the coastal community setting with salt-marsh and floodwall options.

A Two policy realizations where the salt marsh is constructed immediately, followed by adaptive construction of the higher floodwall. B Overall expected normalized costs of MDP-based policy compared against the static baselines: DN-no measures taken, SM- only salt marsh constructed, F- only floodwall F constructed, and SM, F- both measures constructed at the beginning of the planning horizon. C, D Decomposition of monetary and carbon costs respectively. Total (construction + maintenance + flood damage), construction, maintenance and flood damages expected monetary costs are in $ (construction and maintenance costs are amplified 10 times for visualization purposes). The corresponding expected carbon costs along with uptake (rewards) are also in $ (construction and maintenance costs, and carbon uptake rewards are amplified 10 times for visualization purposes).

Fig. 9: Policy for the coastal community setting with oyster reef and floodwall options.

A Two policy realizations where the oyster reef is constructed immediately, followed by adaptive construction of the higher floodwall. B Overall expected normalized costs of MDP-based policy compared against the static baselines: DN-no measures taken, OR- only oyster reef constructed, F- only floodwall F constructed, and OR, F- both measures constructed at the beginning of planning horizon. C, D Decomposition of monetary and carbon costs respectively. Total (construction + maintenance + flood damage), construction, maintenance and flood damages expected monetary costs are in $ (construction and maintenance costs are amplified 10 times for visualization purposes). The corresponding expected carbon costs are also in $ (construction and maintenance carbon costs are amplified 10 times for visualization purposes). Expected carbon uptake rewards are negligible in this case.

It is to be noted in Figs.  8 and 9, that the overall (monetary + carbon) cost associated with MDP is optimal. The MDP-based policies result in a reduction of about $ 0.018 million per meter of coastline compared to the closest static baseline in the case of coastal salt marsh. Similarly, in the presence of oyster reefs, the MDP-based policy results in saving about $ 0.01 million per meter of coastline over the best static policy.

As shown, the MDP results have guarantees to converge to globally optimal solutions, pertaining specifically to the posed optimization formulation, even in complex cases when the identification of such solutions is challenging and may potentially be very close to some sub-optimal choices, as in some of our presented cases here.

Effect of social cost of carbon

The social cost of carbon is a metric used to quantify the economic impact of GHG emissions on society, encompassing both present and future damages resulting from climate change^63,64. The Environmental Protection Agency (EPA) has recently put forth a proposal that quantifies the estimated social cost of carbon emissions to $190 per ton^65,66. This new estimate of the social cost of carbon is nearly four times higher than its current value of $51 per ton set by the U.S. federal government⁶⁷. By substantially increasing the estimate, the EPA emphasizes the need for stronger climate change mitigation and adaptation efforts.

In this work, the effect of increasing the social cost of carbon on the adaptation policies has been studied in the coastal city setting with two floodwalls. An increase in the social cost of carbon increases the frequency in which the floodwall at the higher elevation is constructed, as reflected in Fig.  10A. Increasing this cost also encourages earlier interventions, which is evident in Fig.  10B, showing an increase in the probability of construction of the floodwall at the higher elevation earlier in the planning horizon, when this higher value of the social cost of carbon is used. The proposed increase in the social cost of carbon also results in a considerable increase in the accumulated carbon emission costs over the planning horizon, with more details provided in the Supplementary Materials in Fig.  4.

Fig. 10: Effect of the social cost of carbon and model uncertainties on the obtained policies.

A, B Effect of the social cost of carbon on the (A) frequency of construction and (B) the time of construction of the higher floodwall. No carbon cost corresponds to the case without considering the social cost of carbon; current carbon cost corresponds to the case with the current value of $51 per ton; proposed carbon cost corresponds to the case where the value is increased to the recently proposed value of $ 190 per ton⁶⁵. C Effect of the model used for computed policies. Two policies trained on different SSPs show very similar behavior under similar climate observations. D–F The belief evolution over two SSP scenarios is shown at 5 years intervals based on three different, observed SLR trajectories in time. F1 and F2 in the legends of Fig. (D–F) correspond to construction of floodwalls F1 and F2, respectively.

Effect of model

To study the effect of the underlying climate model on the computed policies, we have considered two policies trained on two different climate model scenarios - one based on SSP2-4.5 and the other on the more intense SSP5-8.5 scenario. Since both trained policies are driven by the in-time observations, it can be seen in Fig.  10C that they are quite similar given similar climate trajectories. This important observation is attributed to the fact that the MDP-based decision-making framework is mainly driven by observations and hence can essentially lead to similar adaptive actions under similar conditions, despite being trained on different models. It is noted though that in this analysis the climate models are not seen to deviate substantially within the studied horizon of 40 years, and they only start to show considerable differences approximately beyond the year 2100.

The underlying climate model uncertainty is also directly incorporated in the decision-making framework, leading now to a POMDP-based problem formulation. We again consider two possible SSP scenarios, SSP2-4.5 and SSP5-8.5, and we consider these two different models as hidden states in the framework. The belief about the two models is thus updated based on SLR observations available in time, using Bayesian principles, as shown in Eq. (4). This belief evolution over the two models at 5-year intervals, starting from a uniform initial belief, is shown for three different realized SLR trajectories in Fig. 10D–F. Figure 10D shows a SLR trajectory where, at each time step, the mean of the possible SLR states is selected as the observed SLR. As seen, for such a SLR trajectory, the two models remain largely similarly probable. On the other hand, for a possible SLR trajectory where lower percentile SLR values are observed, as in Fig.  10E, the belief over SSP2-4.5 gradually increases, as the likelihood of observing lower SLR states is higher under the SSP2-4.5 scenario than the SSP5-8.5 one. Accordingly, Fig.  10F shows a SLR trajectory following a possible higher percentile, having higher likelihood of being observed under SSP5-8.5, and therefore the probability of SSP5-8.5 gradually increases as observations in time are collected. The obtained POMDP policies also show similar behavior with the MDP policies, that is the higher floodwall is constructed when a sufficiently high SLR process is observed, as seen in Fig.  10F. Finally, we have also evaluated the effect of applying a POMDP policy trained with the SSP2-4.5 scenario on the SLR trajectories driven by the SSP5-8.5 scenario, observing the same action realizations as those obtained using a policy trained with the correct SSP5-8.5 model, similar to Fig.  10C, due to reasons mentioned above. Although in this study the modeling effects seem to not influence the derived policy in any crucial way, a more detailed analysis can be performed in the future, to further assess how the utilized climate models can affect the performance of the MDP/POMDP decision-making process, particularly when these models are having sufficiently diverse climate outputs.

Discussion

In this work, the life-cycle adaptation of infrastructure under climate change has been framed as a stochastic-control MDP/POMDP problem, which is generalized to accommodate a wide variety of action types, even system-altering ones. Within this framework, optimal actions are identified to be taken adaptively in time based on observed data and realized conditions, reducing life-cycle costs and carbon emissions and responding to the important uncertainties that characterize the problem. The efficacy of the MDP-based adaptive framework is guaranteed over other methods, for the same optimization problems posed, due to better-informed policies, which are chosen according to the actual evolving climate conditions, and global optimality guarantees, mathematically defined. As expected, the MDP-based policies thus lead to improvements over the considered static policies, albeit marginal to moderate ones for the analyzed settings in this work. This is because the identified adaptive policies here also involve actions at the beginning of the considered horizon, or else an existing seawall already in place, which are also identified by the static policies. However, the fact that the static and dynamic policies differ by a single action, since only one action was deemed optimal for the static policies in all cases, and still a non-trivial 2% − 5% life-cycle cost savings is already realized, shows the potential of sequential decision-making and timely actions based on observed conditions. It is expected that with even higher fidelity and consideration of more available actions, a greater reduction in life-cycle cost relative to the static policy will be seen. The substantial aleatoric uncertainty associated with storm surge risk plays a role in these optimal policies. However, the computed policies and the sequential actions in time seem to be primarily driven by the identified patterns of the evolving SLR process. A final note about the cost-benefit-based static policy considered in this work is that it is only evaluated here based on the initial conditions of the studied horizon, as typically done. Future work can, however, further study these practical approaches based on open-loop control formulations, i.e., recomputing the cost-benefit policy in time with updated conditions as things evolve. However, such static optimization and resulting open-loop policies come with the risk of yielding less-informed and potentially suboptimal solutions, as the identified actions are based upon an averaged forecast over the remaining time horizon without incorporating real-time feedback.

The presented framework can incorporate various levels of uncertainties in its formulation. The epistemic uncertainty associated with the evolution of future trajectories is directly addressed within the MDP framework, and the POMDP extensions further allow the incorporation of uncertain observations and climate model uncertainty consideration, by utilizing observations over time to infer the underlying climate scenario model. While this paper does not directly address the uncertainty associated with future knowledge and climate model improvements, with this additional possible future information further increasing the benefits of the sequential decision-making framework, our presented approach is capable of also accommodating this feature by similarly incorporating potential future model information as the different climate model scenarios have been considered in the POMDP framework. Of course, if the actual future model information deviates substantially from the assumed characteristics, the policies can be retrained, if needed, as this is something that all optimization methods, including static policy ones, can do in an open-loop format.

The growing recognition of the environmental and societal impacts of carbon emissions is leading to an increased emphasis on incorporating carbon considerations into decision-making processes. By considering carbon costs in our decision-making framework, we develop comprehensive and sustainable adaptation policies that is an important step toward addressing both monetary costs and environmental impacts. As the decision horizon increases, and because of the accumulating nature of carbon emissions, the associated costs become substantial and comparable to the monetary costs related to flood management and adaptation measures, thus having an important influence on the identified adaptation policies. In this work, we assume that the current value of the SCC is deterministically known, as it is typically set by relevant policy makers. However, the SCC value may change in the future, due to changing administrations, for example. This is also relevant to the previous discussion about additional information that may come in the future, and can be addressed within our framework, as explained earlier, either in closed-loop representations, by updating our knowledge of some representative potential SCC values in time, or by resolving the policy with the updated value of SCC, if substantial changes are suggested.

Our work also explores the effectiveness and suitability of different nature-based infrastructure for optimal flood risk management by taking into account their unique characteristics. The ability of nature-based infrastructure to assimilate carbon and/or facilitate carbon burial can help in sequestering carbon and contribute to mitigating costs related to carbon dioxide emissions. Considering their holistic benefits, including flood protection through wave attenuation in this study, carbon sequestration, and cost savings through inexpensive carbon footprint associated with construction, these infrastructure options offer eco-friendly and sustainable alternatives for mitigating climate change impacts. There are also often co-benefits that can come from the implementation of nature-based infrastructure that are not accounted for here; though this element of multi-functionality is something that makes nature-based infrastructure increasingly of interest⁶⁸. For example, both oyster reefs and marshes can create habitat for aquatic organisms, including fishes⁶⁸, and have the potential to enhance water quality. The multi-functional green resistance zone considered in the Manhattan scenario is intended to provide recreation opportunities and is likely to offer other cultural and social benefits. These additional ecosystem services can bring additional monetary benefits that have not been accounted for in the current study, yet can readily be included as the knowledge and science of the complete benefits of these nature-based options matures.

Overall, this work aims to present a generalizable stochastic control MDP/POMDP framework for life-long optimal infrastructure adaptation based on the observed evolving climate conditions. The developed framework is general in that it can incorporate a wide variety of possible action types, also considering nature-based infrastructure options along with grey infrastructure choices, without making any structural changes to the problem setting. This work also provides globally optimal solutions while accommodating various other features, such as model uncertainty aspects and the social cost of carbon considerations in the analysis. As with all optimization approaches, including MDP/POMDP-based frameworks, the computational complexity increases with an increase in the number of states, possible dynamic actions, and time horizon length, which at some point cannot be practically handled with MDP/POMDP solvers that provide guarantees of global optimality. This limitation and dimensionality challenges can be overcome, at the expense though of optimality guarantees, by pursuing deep reinforcement learning-based solution techniques, allowing consideration of a larger number of adaptive actions and longer time horizons. Given this higher flexibility and fidelity, the identified adaptive policies may be expected to result in substantially higher cost savings in comparison to static cost-benefit analysis solutions, something we are aiming to explore in the future. Other adaptive approximate methods, such as those based on DPS, for example, can also offer benefits in reducing the dimensionality of the decision variables of the problem and, therefore, may offer solutions and viable alternatives in situations where state-action relation mappings are available, and optimality guarantees are not required or cannot be achieved. Moreover, the various considered objectives have been aggregated in this work into a single scalarized objective function. In future work, we also aim to extend our framework to consider relevant multi-objective formulations to the problem by balancing various tradeoffs among the constituent objectives. Finally, the presented framework here can be further compared in the future with other adaptive solutions in the literature and can also inspire the general further development of even more sophisticated and scalable dynamic programming and pertinent solvers for problems of this type. Since it can provide globally optimal solutions for related optimization problems, as developed in this work, it can also serve as an additional point of reference, when applicable, for assessing the accuracy and performance of newly suggested methodologies.

Methods

MDP formulation components of coastal flood risk mitigation

In this section, we further describe the MDP formulation of the problem at hand, namely the optimal adaptation of coastal infrastructure systems subjected to risks posed by storm surges which are amplified by sea level rise (SLR). The MDP formulation boils down to specifying each element in the tuple 〈S, A, T, R, γ〉.

The state space (S) of the MDP formulation in this work consists of the total peak water level, comprising two components; a local sea level rise and a storm-driven surge level. To formulate a discrete-state MDP, both continuous state variables are suitably discretized. The SLR variable is discretized with a step size of 2 cm, with the maximum SLR value being 150 cm, resulting in a total number of 77 discrete SLR states (S_SLR). Similarly, the storm surge variable is defined by 72 discrete surge levels, with a step size of 10 cm and a maximum surge level of 700 cm. The SLR variable is discretized with a finer step-size in order to capture the slowly evolving process and to obtain more tailored decisions in time. The two components are combined to obtain a total of 5,544 states, as S_t = S_SLR × S_surge. In order to facilitate the solution of the problem in a Markovian setting, the state space needs further augmentation⁴⁶ to incorporate distinct features invoked in this decision-making problem.

Firstly, according to the AR6 of the IPCC (2021)¹, the sea level change process is shown to be non-stationary with gradually increasing values over time. Given the non-uniform progression of sea level rise, the rate of sea level rise is different every year, and thus, it becomes essential to integrate time or else the sea level rise rate into the state space for accurately representing the non-stationary nature of the SLR process within the state information. As far as the storm surge model is concerned, a stationary model has been used, without any loss of framework generality and formulation, since it is assumed that the climate change related non-stationarity is mostly contributed from mean sea level rise. Here, a 40 years horizon has been considered and, therefore, the state space consists of 77 primary SLR states (S_SLR) considered for each of the 40 different time points, resulting in a total of 3080 SLR states ($S^{{\prime} }_{SLR}$), combined with 72 surge states (S_surge), resulting in a total of 221,760 states, as ${S}_{t}=S^{{\prime} }_{SLR}\times {S}_{surge}$.

Additionally, in this problem, the actions represent various flood protection measures which, once implemented, directly affect how the environment evolves over time. In other words, the immediate flood-induced damages/costs cannot be fully determined with the information of only the current action being taken, but the information of all previously executed actions is needed, which essentially determines the existing flood protections already implemented in time. For example, a future storm of a certain height will flood different regions and incur different flood damage induced costs depending on the pre-existing flood mitigation measures that are already in effect, as shown in Fig.  1 in the Supplementary Materials. However, this necessity of using the entire history of previously executed actions to inform the immediate cost model at any point in time breaks the desirable Markovian property of the decision-making process. In this study, we address this complexity by augmenting the state space with the information of already existing flood protection measures at any point of the decision horizon, which is referred to as the "system" information. With a given set of flood protections, there can be various system configurations, each characterized by a unique combination of the available protective measures. For example, with two possible interventions, the two floodwalls F1 and F2, as shown in Fig.  1 in the Supplementary Materials, there can overall be 4 possible system configurations; configuration A where none of the measures have been implemented yet, B where only the lower floodwall F1 is constructed, C where only the higher floodwall F2 is constructed, and lastly the terminal system D where both floodwalls have been built. As a result, the state space also includes the states corresponding to each possible system configuration. Thus, with two possible interventions leading to four possible systems, the state space is augmented here to incorporate 221,760 states belonging to each system, resulting in a total of 887,040 states. Finally, the state space needs to account for the finite horizon formulation. This finite horizon formulation requires, for the utilized solver in this work, adding an additional absorbing state at year 40, which marks the end of the decision-making horizon, leading to a total of 887,041 overall discrete states for this problem.

The actions a ∈ A represent the state-dependent flood protection actions available, such as construction of barriers, floodwalls, or seawalls, as well as green infrastructure options, like repurposed land, salt-marsh, and oyster reefs. The possible set of implementable actions will naturally depend on the type of coastal setting under consideration.

The effect of taking an action in this work is reflected only in a deterministic transition from one system to another. As mentioned previously, there are four possible system configurations in this work, with two available flood protection actions. The effect of taking any action (apart from do-nothing action) is to make a transition from the current state s_t in system X, at time step t, to state $s^{{\prime} }_{t+1}$ in system $X^{\prime}$, at the subsequent time step t + 1, where $X^{\prime}$ represents the changed system, containing now the flood prevention measure associated with the action. Under do-nothing action, the environment continues in the same system configuration based on the transition probabilities derived from the climate projection models.

The transition probability matrix ${{\bf{T}}}(s^{\prime} | s,a)$ models the probability $P(s^{\prime} | s,a)$ of transitioning to state $s^{\prime} \in S$ from state s ∈ S after taking an action a ∈ A. The transition probabilities for SLR states in time under do-nothing action are derived from the probabilistic climate models driving the SLR process based on Monte Carlo simulations, a few of which are shown in Fig.  3C. The method to obtain the SLR transition probabilities for each year of the planning horizon is documented in detail in the Supplementary Materials.

The storm surge transition probabilities ${P}_{surge}(s^{\prime} | s,a)$ can be straightforwardly obtained from the distribution of the underlying Generalized Extreme Value (GEV) model. The transition probabilities for the combined states ${P}_{total}^{t\to t+1}(s^{\prime} | s,a)$ are obtained by suitably taking the combination of each surge and SLR state, and calculating their transition probabilities based on their individual transition matrices. After obtaining the 5544 by 5544 transition probability matrix ${{{\bf{T}}}}^{t\to t+1}(s^{\prime} | s,a)$ for each t of the planning horizon, the transition matrix obtained after accounting for the time effects over the 40 years planning horizon, ${{{\bf{T}}}}^{time}(s^{\prime} | s,a)$, is constructed as follows:

(7)

$${{\bf{T}}}{(i\to j)}_{(11,088\times 11,088)}=\left[\begin{array}{cc}{{{\bf{0}}}}_{(5,544\times 5,544)}^{ii}&{{{\bf{T}}}}_{(5,544\times 5,544)}^{ij}\\ {{{\bf{0}}}}_{(5,544\times 5,544)}^{ji}&{{{\bf{0}}}}_{(5,544\times 5,544)}^{jj}\end{array}\right]$$

(8)

where i and j represent arbitrary, consecutive years. The sub-matrix T(i → j) represents the fact that the only non-zero probabilities are the ones corresponding to the transition from year i to the subsequent year j.

${{{\bf{T}}}}_{(221,760\times 221,760)}^{time}$, as shown in Eq. (7), is not the complete transition matrix of the problem, as it only considers the effects of doing nothing over time and still needs to incorporate the different system configurations due to possible action effects on the system. The complete transition matrix T^Full(a) associated with an action a is constructed by stacking the ${{{\bf{T}}}}_{(221,760\times 221,760)}^{time}$ according to the deterministic transition between the systems under the effect of the action a.

Do-nothing action, a_DN: Under do-nothing action, the environment evolves following T^time in Eq. (7), while staying in the same system. Therefore, the do-nothing related transition matrix is defined as:

$${{{\bf{T}}}}^{Full}\left({a}_{DN}\right)=\left[\begin{array}{cccc}{{{\bf{T}}}}_{(A\to A)}^{time}&{{\bf{0}}}&{{\bf{0}}}&{{\bf{0}}}\\ {{\bf{0}}}&{{{\bf{T}}}}_{(B\to B)}^{time}&{{\bf{0}}}&{{\bf{0}}}\\ {{\bf{0}}}&{{\bf{0}}}&{{{\bf{T}}}}_{(C\to C)}^{time}&{{\bf{0}}}\\ {{\bf{0}}}&{{\bf{0}}}&{{\bf{0}}}&{{{\bf{T}}}}_{(D\to D)}^{time}\end{array}\right]$$

(9)

Action ${a}_{{F}_{1}}$: This action corresponds to the implementation of only one flood preventive measure, the lower floodwall F₁. Thus, this action will result in transitioning to systems B and D if the current systems are A and C, respectively. However, if the environment is already in system B or D, it naturally remains there. The ${a}_{{F}_{1}}$ transition matrix is then defined as:

$${{{\bf{T}}}}^{Full}\left({a}_{{F}_{1}}\right)=\left[\begin{array}{cccc}{{\bf{0}}}&{{{\bf{T}}}}_{(A\to B)}^{time}&{{\bf{0}}}&{{\bf{0}}}\\ {{\bf{0}}}&{{{\bf{T}}}}_{(B\to B)}^{time}&{{\bf{0}}}&{{\bf{0}}}\\ {{\bf{0}}}&{{\bf{0}}}&{{\bf{0}}}&{{{\bf{T}}}}_{(C\to D)}^{time}\\ {{\bf{0}}}&{{\bf{0}}}&{{\bf{0}}}&{{{\bf{T}}}}_{(D\to D)}^{time}\end{array}\right]$$

(10)

The transition matrices for the remaining actions are formulated using the same logic as above and are shown in the Supplementary Materials.

In flood management, the reward R(s, a) at every time-step after taking an action a ∈ A from state s ∈ S consists of the following cost categories: (i) flood damage costs (R_f or C_f) expected over the next time-step, (ii) implementation/construction costs (R_i or C_i) of measures associated with action a, and (iii) fixed annual maintenance costs (R_m or C_m) of the flood protection measures already present in the system. The GHG emissions associated with each monetary cost category are also incorporated in our analysis with the life-cycle GHG emissions translated into a net present monetary value using the concept of Social Cost of Carbon (SCC), defined as the net present value of the long-term climate change impacts caused per ton of carbon dioxide equivalent released into the atmosphere at a certain point in time⁶⁴. Thus, each cost category mentioned above includes two cost components, the capital/monetary costs and the carbon costs. The rewards model under each action possible is shown below.

Do-nothing action a_DN rewards: For the do-nothing action with no protective measure being implemented, the reward R(s, a_DN) corresponds to flood damage costs ${C}_{f}(s^{\prime} )$ that can be expected from transitioning to any state $s^{\prime} \in S$ from the current state s ∈ S, along with maintenance costs C_m of already imlemented flood measures. In Eq. (11), the flooding costs C_f(s,) are defined as a function of the initial state s ∈ S for the do-nothing action, and, thus, the probability $P(s^{\prime} | s,{a}_{DN})$ of transitioning to state $s^{\prime} \in S$ from s ∈ S is implicitly considered. Since the environment continues in the same system configuration for the do-nothing action, the flood risk is then calculated under the current system containing the initial state s:

$$R({s}_{X},{a}_{DN})=\gamma{\sum}_{s^{\prime} \in S}\left[P(s^{{\prime} }_{X}| {s}_{X},{a}_{DN}){C}_{f}(s^{{\prime} }_{X})\right]+{C}_{m}({s}_{X})$$

(11)

where $s^{{\prime} }_{X}$ and s_X represent the water level states $s^{\prime}$ and s respectively, both in system X ∈ {A, B, C, D}, and γ stands for the discount factor.

Actions ${a}_{{F}_{1}}$, ${a}_{{F}_{2}}$, ${a}_{{F}_{1},{F}_{2}}$ rewards: For any other action (a_i) apart from do-nothing, the cost includes the expected flood damage costs and maintenance costs along with the implementation cost (C_i) of the flood mitigation measures associated with the action. Such actions result in a transition from one system configuration to another, and the flood risk is calculated according to the new system as:

$$R({s}_{X},{a}_{i})=\gamma{\sum}_{s^{\prime} \in S}\left[P(s^{{\prime} }_{X^{\prime} }| {s}_{X},{a}_{i}){C}_{f}(s^{{\prime} }_{X^{\prime} })\right]+{C}_{i}({a}_{i})+{C}_{m}({s}_{X})$$

(12)

where $s^{{\prime} }_{X}$ and s_X represent the water level states $s^{\prime}$ and s in systems X and $X^{\prime} \in \{A,B,C,D\}$ respectively, and γ is again the discount factor.

Certain nature-based infrastructure options also possess carbon uptake properties, thus, construction of such solutions also results in carbon uptake benefits (positive rewards) every year post construction.

POMDP formulation components

To directly accommodate climate model uncertainties, the decision-making problem is formulated as a POMDP. The previously defined MDP state space is, therefore, further expanded to include all relevant states within each of the climate scenario models under consideration. Hence, the previously defined state transition matrix ${{\bf{T}}}^{time}({s}^{{\prime} }| s,a)$ is now dependent on the underlying climate model, and the transition matrix ${{\bf{T}}}_{M}^{Full}({s}^{{\prime} }| s,a)$ needs to be obtained for each of the k climate models M_n under consideration, with  n = 1, . . . , k. The transition matrix for the entire POMDP state space under any action a is thus obtained by stacking the individual climate model transition matrices ${{\bf{T}}}_{M}^{Full}({s}^{{\prime} }| s,a)$, as:

$${{{\bf{T}}}}^{{{\rm{POMDP}}}}(a)=\left[\begin{array}{cccc}{{{\bf{T}}}}_{{M}_{1}}^{\,\textit{Full}\,}(a)&{{\bf{0}}}&\cdots \,&{\bf{0}}\\ {{\bf{0}}}&{{{\bf{T}}}}_{{M}_{2}}^{\,\textit{Full}\,}(a)&\cdots \,&{\bf{0}}\\ \vdots &\vdots &\ddots &\vdots \\ {\bf{0}}&{\bf{0}}&\cdots \,&{{{\bf{T}}}}_{{M}_{k}}^{\,\textit{Full}\,}(a)\end{array}\right]$$

(13)

The underlying climate model M ∈ {M₁, . . . . , M_k} is not known with certainty in this framework and is inferred based on observations in time. We consider two possible climate model scenarios in this work, SSP2-4.5 and SSP5-8.5, without loss of generality, and the belief over these models is updated using SLR states, which are fully observable in time. For any given SLR state, there exists a probability of observing that specific SLR state under each of the considered climate models. These probabilities are represented in the observation likelihood matrix, as:

$${{{\bf{P}}}}_{{{\bf{O}}}}=\left[\begin{array}{cccccccc}{p}_{11}^{i}&0&0&\ldots \,&{q}_{11}^{i}&0&0&\ldots \\ 0&{p}_{22}^{i}&0&\ldots \,&0&{q}_{22}^{i}&0&\ldots \\ 0&0&{p}_{33}^{i}&\ldots \,&0&0&{q}_{33}^{i}&\ldots \\ \vdots &\vdots &\vdots &\ddots &\vdots &\vdots &\vdots &\ddots \end{array}\right]$$

(14)

where ${p}_{jj}^{i}$ represents the probability of being in SLR state j and observing the same SLR state under model M₁, i.e., SSP2-4.5. Similarly, ${q}_{jj}^{i}$ represents the probability of being in SLR state j and observing the SLR state j under model M₂, i.e., SSP5-8.5. The superscript i corresponds to the year, as the observation probabilities for different SLR states under each model are non-stationary. The model belief updating is shown in Fig. 10 for some representative cases, and further details are provided in the Supplementary Materials and Supplementary Fig. 3 there.

MDP/POMDP Solver

In this work, an asynchronous value iteration algorithm has been used to solve the resulting MDP/POMDP problems, called Focused Real-Time Dynamic Programming (FRTDP)^50,69. FRTDP proceeds in a series of repeated trials by forwardly expanding from the initial state and selects actions and outcomes based on certain criteria to focus on more promising states in each trial. The algorithm terminates each trial based on an adaptive depth termination criterion and performs Bellman backups on all the states visited along each trial⁵⁰. A schematic of this process is displayed in Fig. 7 in the Supplementary Materials. FRTDP keeps track of both upper (V^U) and lower value function bounds (V^L), such that V^L≤V^*≤V^U, which are used in the final convergence criterion of the algorithm and also help guiding the prioritized search over the vast state space domain⁶⁹.

Life-cycle environmental impacts and social cost of carbon

Life-Cycle Assessment (LCA) is a standardized method to quantify the environmental impacts associated with the life-cycle stages of a product system, from the extraction of raw materials until disposal⁷⁰. LCA provides decision-makers with quantitative assessments, enabling informed decisions on sustainable strategies, policies, and designs that minimize negative impacts on the environment. There are different approaches to conducting an LCA, including process-based LCA, hybrid LCA, and Economic Input-Output LCA (EIO-LCA)^71,72. EIO-LCA quantifies the environmental impacts of an economic system by combining such impacts with economic input-output analysis^73,74.

In this study, we utilize the U.S. 2007 Benchmark Producer Price Model from the EIO-LCA to quantify the GHG emissions associated with each flood protection action considered within our decision-making framework. The model combines 2011 economic input-output tables from the Bureau of Economic Analysis (BEA) with 2013 GHG data. The GHG data are based on a cradle-to-gate system boundary, which accounts for emissions from resource extraction to the final assembly of products, as illustrated in Fig.  8 in the Supplementary Materials. Using this model, we estimated the GHG emissions for flood protection actions based on the economic cost⁷⁵.

The life-cycle GHG emissions associated with each action have then been translated into a net present monetary value using the Social Cost of Carbon (SCC). This approach accounts for the long-term impacts of GHG emissions cumulatively, as well as the economic costs associated with each scenario. The SCC values used in the model are the latest values released by the New York State Department of Environmental Conservation (NYSDEC)⁶¹. The values are obtained based on a 3% average discount rate for 2021-2050 and are extrapolated to 2050-2060 based on observed trends from earlier years.

Costs and damage relations: capital costs

Flood damage costs

Different damage estimation methods with varying degrees of accuracy can be employed to calculate the damage costs C_f under flooding^76,77,78. Here, damage incurred in each zone (C_f(s)) due to a flood level (s) has been calculated based on a simple yet effective damage function, as used in¹⁶, which depends on the vulnerability of the zone to flooding (f_damage), value of the zone (Val_Z), volume of the zone (Vol_Z), and the volume flooded (Vol_F):

$${C}_{f}(s)={f}_{damage}Va{l}_{Z}\frac{Vo{l}_{F}(s)}{Vo{l}_{Z}}$$

(15)

where Vol_F depends on flood level (s) and the established flood protection measures under the implemented policy. The city value is assumed in this work to be uniformly and continuously distributed from the coastline to the highest city elevation. The city model parameters inspired by Manhattan are based on various assumptions and are listed in Table 1 in the Supplementary Materials. These parameters can be suitably adjusted to represent any particular city.

Floodwall costs

The existing literature shows a wide range of construction and maintenance costs associated with floodwalls and dikes^79,80,81, and for this work the relevant costs are based on simple assumptions with the aim of representing the efficacy of the framework under a representative coastal city setting, which can also be easily modified. The construction cost has been established as: 

$${C}_{cfw}={H}_{fw}\times {C}_{fw}$$

(16)

with C_cfw being the cost in US dollars per meter of floodwall length; H_fw designates the height of the floodwall; and C_fw is the unit cost per meter of height and length. The unit costs of floodwall construction and maintenance in NYC are sourced from the literature⁸², adjusted for inflation using the Consumer Price Index (CPI) index, to account for price changes over time⁸³, and are presented in Table 4 in the Supplementary Materials.

Green resistance costs

The construction and maintenance costs of nature-based infrastructure reported in the literature are limited and vary highly based on the location and type of nature-based infrastructure implemented. In the coastal city setting, this study considered a multi-functional re-purposed green space that would be implemented behind the sea wall, adjacent to the shore. This 150m deep zone into the city comprises turfgrass, meadows, trees, and a walkway. This zone is assumed to have negligible wave attenuation properties and some capacity to assimilate carbon from the atmosphere. The monetary costs are determined based on a review of existing studies on the previously mentioned land cover categories. Based on the construction and maintenance costs reported by⁸⁴, calculated capital and maintenance costs for the green resistance are set as $25/m² and $2.7/m² per year, respectively, as shown in Table 6 in the Supplementary Materials. In the coastal community setting, this study considered two types of nature-based infrastructure, namely coastal salt marsh and oyster reefs. The cost of these considered nature-based infrastructure choices varies widely as indicated in Table 6 in the Supplementary Materials. Construction costs of the salt marsh and oyster reef are primarily assumed here to be $100,000/ ha and $50,000/ ha, respectively. Their maintenance costs are assumed to be negligible.

Costs and damage relations: carbon emission costs

Each category for which the capital costs are computed, as outlined in the previous subsection, is accompanied by corresponding carbon costs. The carbon emissions attributable to flood damages arise from post-damage recovery efforts. Naturally, construction and maintenance of flood protection assets also contribute to carbon emissions. Carbon emissions are subsequently quantified in monetary terms through the SCC metric.

Once carbon emissions are released into the atmosphere remain there unless they are removed. Consequently, the costs associated with carbon emissions continue to accumulate from the time of emission, albeit discounted over the future planning horizon. To account for the cumulative nature of carbon costs, without necessitating the explicit tracking of current atmospheric carbon levels, the total carbon cost related to emissions at any given point in time is computed as the sum of discounted carbon costs over the remaining planning horizon from the moment of emission. The carbon emission costs associated with each category are described in detail below.

Flood damage carbon emission costs

We have estimated the life-cycle GHG emissions of flood damage by considering the environmental impact resulting from post-flood construction activities of damaged buildings. For this purpose, we have used the land-use distribution of buildings destroyed by Hurricane Sandy, as released by the Department of the City Planning, City of New York⁸⁵. We then matched this distribution to the representative EIO-LCA construction sector, as shown in Table 5 in the Supplementary Materials. For example, the portion of damaged buildings designated as “One and Two Family” buildings in the land use plan is paralleled with the “Single Family Homes” construction sector in the EIO-LCA⁸⁶. We calculated a single Flood Damage GHG emission Intensity (GHGI_f) in tons of CO2-equivalent per dollar, by weight-averaging the GHG intensities for different land uses based on their percentage of the total. As an illustration, knowing that 88% of the damaged buildings pertain to the category of “One and Two Family Homes”, 88% of the flood damage emission factor is then allocated to the construction sector of “Single Family Homes” in the EIO-LCA model. This enables the estimation of Flood Damage GHG (GHG_f) in tons CO2-equivalent, based on the damage cost (C_f) in Eq. (15), corresponding to the flood depth (s), as described below:

$$GH{G}_{f}(s)=GHG{I}_{f}\times {C}_{f}(s)$$

(17)

Flood Damage GHG emissions (GHG_f) are then converted into monetary flooding carbon cost, ${C}_{f}^{c}$, using the SCC as a function of the emission time, t_e. The SCC is applied in a cumulative manner, accounting for the economic damage caused by one ton of CO2-equivalent from the time it is emitted (t_e) until the end of the examined period, t_N. The discount factor (γ) is also applied to discount the carbon cost, reflecting the time value of money:

$${C}_{f}^{c}(s,{t}_{e})=GH{G}_{f}(s)\times {\sum }_{t={t}_{e}}^{{t}_{N}}SCC(t){\gamma }^{(t-{t}_{e})}$$

(18)

Floodwall carbon emission costs

To determine the GHG intensity associated with floodwall construction (GHGI_cfw) and annual maintenance (GHGI_mfw), we use ECO-LCA and select the categories of Construction of Infrastructure and Repair and Maintenance of non-residential buildings sectors^75,87, respectively. The GHGI values are reported in tons of CO2-equivalent per dollar. To estimate the total CO2-equivalent associated with floodwall construction, we use:

$$GH{G}_{cfw}=GHG{I}_{cfw}\times {C}_{cfw}$$

(19)

which incorporates the construction cost (C_cfw). Similarly, the carbon cost associated with the construction of the floodwall has been determined using the SCC and the discount factor (γ), as:

$${C}_{cfw}^{c}(s,{t}_{c})=GH{G}_{cfw}\times {\sum }_{t={t}_{c}}^{{t}_{N}}SCC(t){\gamma }^{(t-{t}_{c})}$$

(20)

where t_c is the time of construction. The carbon cost associated with annual maintenance is then calculated following the same methodology. The GHG intensity values associated with flood damage, construction and maintenance of floodwalls are listed in Table 4 in the Supplementary Materials.

Green resistance carbon emission costs

Carbon emissions associated with construction and annual maintenance and biological carbon uptake are calculated based on the area-weighted summation of carbon emission or uptake rates. Carbon emissions associated with construction and annual maintenance are sourced from existing life-cycle analyses for components of the green resistance zone and are estimated to be 1.6 kg C m² and 0.09 kg C per m² per year, respectively, as listed in detail in Table 7 in the Supplementary Materials. Biological carbon uptake for the green resistance zone is estimated at 0.17 kg C per m² per year, as shown in Table 9 in the Supplementary Materials. Construction carbon emissions for restored or constructed salt marsh and oyster reefs are not well documented in existing studies. For this study, we are assuming the cost of installing salt marsh to be slightly lower than turf grass installation, as shown in Table 6 in the Supplementary Materials. Carbon uptake via biological assimilation or burial can occur in salt marshes (0.44 kg C per m² per year) and oyster reefs, as documented in field investigations^88,89; however rates are negligible in oyster reefs ( ~ 3^-10 kg C per m² per year) due to lack of vegetation, as shown in Table 9 in the Supplementary Materials.

Flood protection options and their roles

This subsection explains the different flood protection actions or options considered in this work and their flood protection roles and modeling in the MDP/POMDP framework. Flood protection options affect the flood damage relationship in Eq. (15) in distinct ways, depending on their unique characteristics.

Seawall: In the coastal city scenario, we consider the option of an existing seawall of fixed height along the city’s edge. The city suffers no damage if flood levels are below the seawall height. When the flood level exceeds the seawall height, the excess water height over the seawall results in flood-induced damages.

Floodwalls: In both the city and community settings, the floodwalls have fixed heights and positions and are implemented adaptively over time. The floodwalls protect zones located behind them until they are overtopped. If a floodwall is overtopped, this is considered a failure event, resulting in flooding throughout the entire zone protected by the floodwall, consistent with the relevant flood elevation.

Green zone: The multi-functional green zone in the coastal city scenario offers decreased flood damages due to the reduced vulnerability of the green zone (${f}_{damage}^{GZ}$). In this work, a 20% reduction in vulnerability is assumed over the green zone. The green zone can also potentially have some wave attenuation properties, as indicated in Table 8 in the Supplementary Materials, which can also reduce flood heights and thus the volume being flooded (Vol_F) in Eq. (15). In this work, this attenuation  has been considered negligible and is, therefore, excluded from the final calculations.

Nature-based infrastructure: The nature-based infrastructure options considered in the coastal community scenario include salt marshes and oyster reefs. Both solutions possess considerable wave attenuation properties, as listed in Table 8 in the Supplementary Materials, which result in a reduced flooded volume (Vol_F) in Eq. (15). The water level responsible for flooding (h_f) is obtained by reducing the total incoming water level (h) by the attenuated water height (h_a). This attenuated height is obtained in turn by applying the relevant wave attenuation function (f_wa), reported in Table 8 in the Supplementary Materials, associated with the nature-based solution and the incoming total water level (h), with a maximum limit being the surge height h_surge, as:

$${h}_{f}= h-{h}_{a}\\ {h}_{a}= min({f}_{wa}(h),{h}_{surge})$$

(21)