Analysing the use of emissions scenarios in practice

Abstract

Emissions scenarios generated by Integrated Assessment Models (IAMs) are an important element in the IPCC’s relevance for climate action, but little attention has been given to how scenarios are used in practice. This article draws on insights from Science and Technology Studies (STS) to develop a framework for analysing scenario use and provides an exploratory case study of scenario users in government and energy-related companies in Norway. We find that scenarios are used for contrasting purposes but are often actively interpreted and processed by informed users who are aware of different scenarios’ strengths, limitations and uncertainties. Institutional ties, trust, and alignment in problem definition play important roles when organisations choose which scenarios to work with, and the political implications of scenario choice are frequently acknowledged. We conclude that more attention should be paid to the actual use of scenarios in discussions about the IPCC’s policy relevance.

Introduction

Scenarios generated by Integrated Assessment Models (IAMs) play a central role in the work of the Intergovernmental Panel on Climate Change (IPCC)^1,2,3. IAMs are numerical models that combine different knowledge disciplines—typically economics, environmental sciences and engineering—to model the interactions between climate, land use, economic and technological systems. They are used to produce scenarios that represent consistent pathways of emissions, energy use and economic variables towards different levels of future warming. Although IAM generated scenarios are not strictly products of the IPCC, the IPCC has played an important role in organising what is often referred to as the “IAM community”, and is therefore closely tied to the production and circulation of these scenarios⁴. Emissions scenarios have also been among the most discussed and referenced parts of IPCC assessment reports and are therefore seen as key both to the IPCC’s policy relevance and to the wider role it plays in informing and structuring climate change discourse^2,5.

The growth of IAMs for emissions scenario development has been accompanied by continuing discussions about their usefulness and credibility, including discussions about the needs of scenario users^6,7. Already in 2008, prominent IAM modelers O’Neill and Nakicenovic⁸ called for future work on emissions scenarios to include a “much greater emphasis on understanding how scenarios are actually used”. Following this call, a large literature has discussed the usability and relevance of scenarios. As documented in a recent review of scientific critiques of emissions scenarios⁶, these discussions have focused in particular on the inclusion of stakeholders and decisionmakers in scenario development, and how scenarios are communicated to policymakers. In response, actors within the modelling community have called for more systematic involvement of scenario users and more attention to the needs of decision-makers in the production of IAM scenarios⁹. The IPCC has also taken a role in facilitating discussions about how scenarios can be made more relevant to decisionmakers¹⁰.

A striking feature of these discussions, however, is that they to a very limited extent are based on empirical knowledge about how scenarios are used in practice. With some notable exceptions^11,12, few studies have so far investigated how scenarios are perceived, understood, and used by the “scenario users” among decisionmakers in government or industry in specific contexts. The result is that we know little about the choices and considerations that lie behind how scenarios specifically from IAMs are used in decisions that affect climate action. This is puzzling, given that one of the primary purposes of these scenarios is to inform climate policy development and decision-making. Improving the understanding of scenario use is particularly important in the current situation, where the IPCC is chartering its direction for its upcoming cycle: more knowledge about how outputs from IPCC processes affect decisions and outcomes, and how IPCC products stand out from other information, is indeed important when discussing the relevance of IPCC knowledge to climate action^1,13.

Instead of solid empirical knowledge about scenario use, discussions about useability and relevance in the scenario literature have tended to be based on what Geden¹⁴ has criticised as a “functionalist” understanding of scientific policy advice: An approach that assumes a means-end rationality in which scientific knowledge can be “applied” directly to decisions that in turn determine outcomes. Consequently, the focus is placed on “improving communication” of scenarios or “building capacity” for understanding them¹¹, instead of examining more openly how they are actually used and with what effects – including effects that may be counterintuitive and surprising. Where the users of emissions scenarios are described or discussed, they are often categorised through rather simple typologies. For example, Kriegler et al.⁹ discuss the needs of two different kinds of users: “end users”, which includes decisionmakers who use scenario outputs in various decision processes, and “intermediate users”, which include researchers who use scenarios as inputs into their work. As Geden points out, however, scientific policy advice is received and used by a very heterogenous group, and their use of scientific advice needs to be understood within a broader political landscape¹⁴. This calls for empirical studies of scenario use as a socially and politically embedded practice, as a basis for more nuanced and realistic discussions of future scenario development.

Hence, there is a need for studies that document and analyse how emissions scenarios are perceived, used, and translated in processes of climate and energy decision-making. The aim of this paper is to contribute towards such a research agenda in two ways: First, we provide an empirical exploration of the use and perception of scenarios within a group of government decision-makers and industry experts, through a case study from one specific country context, i.e. Norway. As a country with stated leadership ambitions in climate policy, in combination with a large oil and gas sector, Norway is an interesting case for exploring how scenarios of future energy use and emissions are taken up in policymaking and industrial strategy. This is done through in-depth interviews with civil servants and industry experts, supplemented by document analysis.

Secondly, we synthesise relevant theoretical insights from Science and Technology Studies (STS) with the aim to inform ongoing debates on the use and relevance of emissions scenarios. STS has a long-standing tradition of studying the role that modelling, scenarios, and quantified information may play in decision-making, with a particular focus on the practices by which these forms of future-oriented knowledge are put to use. However, the STS literature is only sparingly utilised and referenced in the community of IAM modellers. Hence, a synthesis of key insights from this field can be a useful contribution towards debates among modellers and IPCC contributors on the relevance of emissions scenarios and their future role in the work of the IPCC.

The paper is guided by two research questions: (1) How are emissions scenarios perceived and used in processes of climate and energy analysis and decision-making? (2) How can perspectives from the STS literature provide a lens for analysing the use of emissions scenarios in practice? To answer these questions, we draw on the above-mentioned case study of public and private decision-makers in Norway. The case study is exploratory in nature: Rather than seeking to draw generalisable conclusions, our aim is to understand some of the mechanisms through which scenarios are taken up by users, and to illustrate how STS insights can be used to analyse such mechanisms. In this way, the exploratory study presented here may also provide a basis for typology-construction or hypothesis-building in later studies.

The paper proceeds as follows: The methods section first provides a synthesis of key insights from the STS literature, setting out a tentative theoretical framework for the empirical analysis. It also provides an overview of the material and context of the case study that follows. The results of this study are reported in the following section, while the concluding section discusses the results in light of our theoretical framework and sums up how such analyses may inform ongoing broader debates about the future of emissions scenarios and the IPCC.

Methods and analytical framework

Insights from STS: A theoretical framework

The field of Science and Technology Studies (STS) studies scientific knowledge and political agency not as separate domains but as co-constituted and closely intertwined (cf. the introduction to this Special Collection)¹³. Reflecting the prominent role that IAMs have gained in the IPCC, a growing literature in STS has analysed how IAMs are produced and how they should be used to inform policy outcomes. Several studies have mapped the rise of IAMs as a new epistemic field with a particular prominence in the work of the IPCC^4,5,15. More critical work has highlighted the power of IAM scenarios to shape climate policy discourse, and argued that scenarios unduly structure climate action by constraining the range of what is seen as possible or desirable futures^16,17,18. Building on the insight that models can have “performative” effects by shaping the reality they purport to describe¹⁹, it has been argued that IAM scenarios hold a form of “world-making power”¹⁶. However, much as in the discussions within the modelling community, the actual users tend to remain hypothetical entities also in this literature: As far as we are aware, there are no in-depth empirical studies of the practices in which emissions scenarios are put to use that show how potentially “performative” effects may arise.

Despite this empirical lacuna, the field of STS offers a range of insights and concepts that may serve as a framework for such studies. In the following, we highlight three such insights regarding (1) the non-linear relationship between scientific advice and policy “uptake”; (2) the different roles scientific knowledge can play in broadening or narrowing down societal deliberation and political disagreement; and (3) the variable ways in which the relationship between science and policy can be organised and institutionally structured. We then set out a framework for analysing scenario use drawing explicitly on the two latter insights, while the former provides a more general lens on the use of emissions scenarios in decision-making that challenges some of the functionalist assumptions described above.

A general lesson emerging from STS work on scientific policy advice is that we should not expect scientific knowledge to lead to decisions in a linear fashion^20,21. Rather, knowledge may be used in various and often contradictory ways by actors pursuing different ends. Ideals of objectivity and scientific independence are often mobilised to shore up claims to epistemic authority. Highly quantified knowledge can be a form of maintaining such authority, as “trust in numbers”²² provides a sense of objectivity that may serve as an important rhetorical resource and allows knowledge to travel easily across domains perceived as separate. However, as highly precise quantified numbers are translated across different domains, their use and perceived content may change substantially, leading to over-confidence in their truth-claims or loss of crucial contextual information about, for example, assumptions and uncertainties.

The modelling of complex systems can be a particularly attractive way of turning uncertainties about the future into quantified knowledge that can underpin action in the present^23,24. In particular, numbers resulting from complex modelling may function as “black boxes”²⁵ that can circulate easily and are taken as facts on the basis of the independence and authority of its producers, precisely because the underlying process, assumptions and uncertainties are hidden from view²⁶. This could have the effect of “closing down” or avoiding debate on controversial topics, on the basis that modelled numbers are invested with credibility as incontestable facts that are not opened up to scrutiny or critical debate. Stirling²⁷ contrasts this with the effect of “opening up”, which is achieved when scientific advice is used to broaden the range of possible outcomes and questioning assumptions, increasing the scope for deliberation and/or dissent.

Another important topic in the STS literature is how different institutions organise the interface between science and policy, and the effects of different organisational choices. In some cases, scientific credibility is invoked by ensuring separation between policy processes and knowledge production, and by relying on the established scientific authority of bodies such as the IPCC²⁸. This can be seen as a way of establishing trust in knowledge-claims through appeals to ideals of objectivity and epistemic authority. In other cases, it is rather close proximity between scientists and policymakers that produces trust and strengthens the credibility of knowledge as a basis for action²⁹: Organisational integration and institutional ties may lead to mutual adjustments that stabilises knowledge-claims over time^30,31. Here, too, quantification and precise numbers may aid the stabilisation of knowledge in a way that allows different groups of actors to act on the basis of shared facts despite different ways of using or interpreting them²⁶. Such stabilised facts have been described as “boundary objects”³², a term that has often been used to analyse the role of quantified scientific knowledge that circulates between climate science and policy³³. To operationalise key insights from the above-mentioned literature, we suggest mapping the use of emissions scenarios along two dimensions. The first draws on Stirling’s²⁷ conceptual dichotomy between two different purposes toward which the use of scenarios may be directed: Either the aim of “opening up”, i.e., widening the field of possible solutions and fostering critical deliberation; or the aim of “closing down”, i.e., cutting off debate or stabilising certain outcomes by appeal to scientific authority. Of particular relevance for understanding how scenarios are used in this regard is the use of quantified information and the understanding of preciseness relative to uncertainty in scenario data.

In the second dimension, we map how the relationship between scenario users and the production of scenario data is organised, as this is expected to be particularly salient for the trust that users place in scenarios. Building on Sundqvist et al.^28,29, we draw on a distinction between separation and integration: On the one hand, users’ trust in scenarios may rely on a strong separation between decision-making and scientific advice, while on the other hand, trust may be enhanced by direct involvement in the process of knowledge production or institutional alignment between scenario producers and users.

In summary, the framework introduced here allows for the practical uses of emissions scenarios to be understood along two dimensions that indicate (1) the aims toward which actors employ scenarios (“opening up” versus “closing down”) and (2) how the relationship between users and producers is organised and represented (“separation” versus “integration”). This framework is proposed as a starting point for understanding scenario use from the actors’ point of view. The ambition is not to put forward strong claims regarding generalisability or causal mechanisms; rather, we intend to explore and explain variation and lay the groundwork for more conceptual studies. Moreover, the framework should not be seen as reductive, limiting the analytical view of how scenarios are used in practice. Starting from the insight that a linear transmission of knowledge into policy is the exception rather than the rule, the analytical aim should instead be to capture the empirical richness of scenario use, and to keep an open mind regarding the assumptions about scenario users that tends to dominate discussions about scenario relevance.

Explorative case study

While many societal actors may be seen as potential users of emissions scenarios, the following case study focuses on national government officials and experts from the energy industry in a particular country, i.e. Norway. On the governmental side, national-level decision-makers are key to developing and implementing climate policy, as the current international regime relies on parties to the Paris Agreement to contribute to global mitigation efforts through “Nationally Determined Contributions”. Norway is a particularly interesting context for exploring the use of emissions scenarios for several reasons: First, the government has explicitly stated ambitions to be a leader in climate policy and has demonstrated active involvement in multilateral settings such as the IPCC and the UN Framework Convention on Climate Change (UNFCCC) over many years³⁴. Second, the economy is closely linked to the future trajectory of energy markets, as it is dominated by oil and gas production, and energy-related companies with a stake in the future of energy markets thus play a particularly important role in the economy. The country’s leadership ambitions in climate policy and heavy reliance on fossil fuel production has become increasingly difficult to reconcile in recent years and is a source of frequent political controversy³⁵. This makes climate and energy related scenarios potentially both useful and contested as inputs to policymaking processes. And third, the country has a long-standing tradition for future-oriented economic planning, with economists and economic models having a particularly strong role in many government bodies³⁶.

The case material consists of semi-structured interviews with civil servants and industry experts, supplemented with an in-depth analysis of documents that were identified based on the interviews. The interviews were conducted in person or online, each lasting for 30–60 min, and were transcribed and analysed manually. The documents identified for analysis comprise a total of 615 pages, which were scanned for references to emissions scenarios before relevant sections were qualitatively analysed through close reading. In the analysis, the documents are presented in separate boxes as examples to illustrate the findings from interviews.

Seven government officials representing five government bodies were interviewed. They represent all the main government bodies involved in policymaking related to energy and climate change in Norway: The ministries for Petroleum and Energy, Climate and Environment, and Finance, as well as several relevant government agencies: The Environment Agency and the Petroleum Directorate. From the private sector, four experts from four different companies particularly relevant to the energy industry, were interviewed. The experts all hold senior or leadership positions in leading companies within the oil and gas, power, and energy sectors: Statkraft, Equinor, Rystad Energy, and DNV. All the companies are based in Norway but operate globally. In the discussion below, the interviewees have been anonymised to avoid identification of any person or affiliation associated with particular statements.

The STS literature gives reason to expect significant differences in how scientific knowledge is validated, embedded and used in policymaking processes across countries^37,38,39. This also applies to the use of modelling and scenarios^11,40, which means that the particularities of how IAM scenarios are used in policymaking are likely to differ between national contexts. Similarly, it is to be expected that different types of industry actors relate differently to scenarios in their work. The empirical focus in our study, a relatively small group of actors from a single country, does not enable a systematic mapping of differences across countries or across industries. Our aim is rather to explore variation and to illustrate the use of STS insights to analyse scenario use. An in-depth study of a limited number of actors is thus well suited because it allows us to show in more detail how individual actors perceive, use, and translate scenarios in practice. A relatively low number of interviews has been shown to be sufficient for providing data saturation in purposive sampling⁴¹. Even though particularities of scenario use may differ across countries and industries, many of our findings are also likely to hold across these, not least because our informants from the private sector are from companies operating on a global basis. An in-depth study of a limited sample like this may therefore also provide a starting point for developing typologies or building hypotheses that can be further examined in subsequent studies.

Results

Government officials

The government officials interviewed were all involved with developing and implementing energy and climate related policies. They were all familiar with emissions scenarios and they indicated (with one exception) that they typically related to them when developing analysis that is meant to inform or justify policy. This included, for example, as input into the preparation of white papers or budget documents, as a basis for decisions on policies; or for use in the text of such documents to provide justification for new or existing policies. Scenarios were also sometimes used as part of information provided by the government in response to Parliamentary inquiries.

The main way in which emissions scenarios were used was to assess the consequences of or pathways towards specific policy targets – particularly the targets of the Paris Agreement. Typically, scenarios were used to provide different types of information on characteristics of emissions pathways or endpoints related to such targets, including carbon prices, energy prices, energy mix, and future emissions levels or overall carbon budgets.

Regarding the specific sources used, several were mentioned but some clearly dominated. The scenarios of the IEA were most prominently mentioned, followed by IPCC scenarios, either in the form of illustrative scenarios mentioned in IPCC reports or the full IPCC scenarios database. Other multilateral sources such as the IMF, OECD and IRENA were also highlighted. In addition, scenarios from some private providers were mentioned, including BP, BloombergNEF, DNV and Equinor.

Use of scenario data

When asked how they worked with scenarios, interviewees described using them primarily for attaining specific quantified variables, such as a carbon price, energy prices or emissions levels at a specific point in time. Several interviewees were particularly concerned with oil prices, a natural concern in a petroleum-exporting country like Norway (for an example, see Box 1). The numbers of interest were used as a basis for further internal analysis, as input to their own calculations or even as input to their own modelling, for example by modelling developments in the Norwegian economy under different assumptions about future oil prices. They could also be communicated externally through government reports or policy documents to justify particular policies.

Because of the need for specific numbers as input into further analysis or as a basis for justifying policies, some interviewees preferred scenarios that provide clear point estimates over the wide ranges provided by the IPCC’s scenario database:

We do have the IPCC’s compilation of scenarios, but there’s a wide spread in that. A very wide spread. And it’s a bit difficult to get an overview over them, what assumptions they are based on. In the 1.5 [special] report (…) the [carbon price] interval is between 135 and 5500 dollars per tonne. And that’s… it’s not really very operational.

On the other hand, most interviewees explained that they often do not rely on a single scenario or estimate but rather compare different sources to provide a broad picture. This could for example be by trying to identify ‘consensus’ estimates, i.e. comparing numbers from different scenario providers to identify a central range around which most scenarios seem to converge, and to disregard perceived outliers. It could also be by comparing change over time, for example how key variables change between annual reports as a way of gauging the direction of change.

This suggests that interviewees were aware of the limitations inherent in any single scenario analysis. Indeed, such an awareness was often explicitly stated: The interviewed officials frequently acknowledged the uncertainties associated with modelling the future, and that “these models are by definition going to get it wrong”. Remarks such as “nobody has the answer”, and “garbage in, garbage out” were frequent among all informants, and some were specific about assumptions they thought made the models less realistic. At the same time, their work required quantified estimates even though they were recognised to be highly uncertain:

If you look at many of the big [providers] (…) there is at least some consensus on the carbon price. But then of course the assumption is that everything is done in a cost-efficient manner on the global scale (…). And that’s not particularly realistic. But it is a reference point. We see it as important for Norwegian climate policy to have a reference point.

In other words, the officials on the one hand need specific numbers rather than wide ranges or uncertainty estimates, while on the other hand they are acutely aware that no single number would have a claim to being correct:

It makes it all more concrete, right? That’s the thing with numbers, they make it easier to imagine it, and also to do further calculations. But that’s also probably the danger, because they are scenarios, so we shouldn’t be too confident in the numbers either.

In this way, the interviews generally paint a picture of ‘informed use’: Model outputs are not used uncritically but assessed in relation to other results as well as in-house expertise on modelling, energy markets and so on. Furthermore, several interviewees expressed an awareness that scenarios are primarily story-telling tools rather than forecasts. This is also how they explain using them in some concrete cases: For telling a story or making a point, often in relation to existing policy. In summary, uncertainty is acknowledged, but eventually specific numbers are still required.

Box 1. Example: Report to parliament on long-term economic perspectives (2021)

The white paper on “long-term economic perspectives”⁴⁶ illustrates how government officials use emissions scenarios in their work. The document was published by the Ministry of Finance, with input from other ministries, and presented to Parliament in 2021. It was mentioned in all interviews as a prominent example of a document where emissions scenarios are used, probably because it was under development at the time of the interviews. Emissions scenarios are referenced extensively in two sections – one named “The climate challenge” and one titled “Outlook for the petroleum industry”. The former discusses climate policy measures and carbon prices compatible with Paris, while the latter discusses developments in the oil and gas markets in light of the Paris agreement.

The chapter on “the climate challenge” uses emissions scenario data from IPCC reports to document overall challenges of reaching global climate targets and the necessary level of emission reductions⁴⁶. In particular, median values for emission reduction levels and carbon prices compatible with the 2°C target are reported and compared to carbon price estimates from the IEA and the IMF⁴⁶. The chapter discusses several uncertainties, especially the level of negative emissions implied in most scenarios.

In discussions of future energy markets, 2 °C compatible scenarios from the IEA and IMF are used to “calibrate” the ministry’s estimates of future oil prices: Because the ministry’s estimates fall between the higher-end IEA and lower-end IMF estimates, they are “assumed to be compatible with the world reaching the goals of the Paris agreement”⁴⁶. In this way, the scenarios lend credibility to the ministry’s own estimates, allowing the ministry to argue that its assumptions about the future oil market is already in line with global climate goals and therefore no adjustments are necessary.

In sum, the document draws on scenarios from several sources, but only from intergovernmental organisations. The IPCC dominates as a source where the focus is on global emissions pathways, while the IEA and to some extent IMF dominate when the focus is on energy markets and economic variables. The ability of scenarios to provide precise numbers is important throughout, but numbers are not used uncritically: Rather, they are calibrated against each other, and uncertainties are acknowledged and discussed. Rhetorically, the scenarios are employed partly to signal urgency and describe the magnitude of the climate challenge, and partly to justify existing assumptions and policies, including existing plans for future oil and gas extraction.

Choice of scenarios

Our informants mentioned a broad range of sources, but they also indicated that not all sources are equally ‘citeable’ in official documents. Officially recognised institutions such as multilateral organisations were preferred over private actors. At the same time, the interviewees highlighted that different institutions have different strengths and weaknesses, and that this informs which scenarios to use. For example, this is how one interviewee described the IEA:

The IEA stands out as a source because they are often seen as the most recognised and well known. They are considered transparent, clearly stating their assumptions. And they have (…) a good grasp of energy markets and how they relate to climate policies. And then they provide updates every year, so you know you will have regular access to updates from the same source, and you will have experience with it.

Among these reasons for preferring IEA scenarios, the expertise in energy markets was highlighted in several interviews. Trust in their analysis was not unconditional, however:

But it’s not like you just copy and paste it. Especially on oil price, for example (…) they have been among the most optimistic over time, way above what has been the consensus in the market or from other sources.

Other multilateral institutions were also mentioned as particularly trustworthy and citable in official documents. The OECD was seen as having particular insights into financial markets and the economic outlook, while the IPCC was mentioned as a source with high scientific credentials.

Furthermore, the interviews clearly indicated that institutional ties play an important role when choices are made about which scenario providers to reference in analysis and official documents. Interviewees from institutions overseeing oil and gas policy mentioned this explicitly as a reason for preferring IEA analysis:

We are part of the IEA system. We are a member of the IEA, and we participate actively in different working groups. (…) We do not live in a bubble, only looking at the IEA. But we lean more towards them.

At the same time, interviewees with responsibilities in economic policy pointed to analysis from the OECD and IMF for similar reasons, while institutions overseeing climate policy saw it as their role to champion scenarios from the IPCC, and to some extent IRENA. One interviewee explained that their colleagues had been part of the IPCC process, and therefore

they know the information very well. You probably feel more confident when you have been in those meetings, you know why the nuances in the summary [for policymakers] turned out the way they did.

This shows that direct relations to the organisation providing scenarios is important for the level of expertise ascribed to the organisation and the trust invested in it. It is also clear that different governmental institutions prefer to use analysis from organisations with a problem definition similar to their own: Oil policy institutions prefer IEA analysis, which often foregrounds issues like energy security and energy market dynamics. Officials working with economic policy prefer analysis by institutions such as the IMF, whereas climate policy institutions see it as their role to foreground the IPCC reports and the scenario information they provide.

Energy industry experts

The industry experts interviewed were all highly skilled scenario producers as well as users of scenarios produced by other providers. They work in companies related to the energy industry, such as energy production and consultancy, that produce their own scenarios or forecasts of energy markets and emissions (for an example, see Box 2). These scenarios are produced for various purposes: For strategic internal planning as well as concrete decision making, for promotion of the company, customer advice, and communication to shareholders.

When it comes to the use of scenarios provided by others, the IEA was again the most widely used provider. From the IPCC, climate scenarios (i.e., from Working Group I) were used more frequently (to assess climate aspects) than emissions scenarios (from Working Group III). For information pertaining to emissions scenarios, informants in this group relied more on their own scenarios. Other scenario providers mentioned by informants in this group include public institutions such as the U.S. EIA, IRENA, IAEA, and the EU, as well as private companies like BloombergNEF, BP, Shell, McKinsey, Exxon Mobil, DNV, Equinor, and Statkraft.

Box 2. Example: DNV GL energy transition outlook 2020

The four companies that our informants come from all produce their own reports presenting their own scenarios. While reports from DNV, Statkraft and Equinor are publicly available, reports from Rystad Energy are not. To complement the interviews, we look at how scenarios are referred to and used in the DNV GL Energy Transition Outlook 2020⁴⁷, which is the year that the interviews were conducted.

The DNV ETO 2020 is presented as “a global and regional forecast to 2050”. It provides an extensive presentation of their single scenario, which is their “best estimate” forecast of the energy future. The motivation behind this forecast is “to shine a light on a transition that represents the greatest source of risk, and opportunity, in our business environment”⁴⁷. The report recognises how some of the predictions in it may be “startling”, which is an implicit acknowledgment that they deviate from the views or projections from other providers. But they also highlight that their “predictions are rooted in real-world experience with energy customers across the world spanning the full energy mix”.

The DNV ETO 2020 refers to the IPCC 24 times, more often in relation to emissions and climate implications (for example, the remaining carbon budget or temperature impacts)than in relation to emissions scenarios and their technological and economic implications. They also cite IEA eight times, but mostly in relation to data used as inputs to their own modelling. Confirming the interview data, the report shows that DNV combines inputs from a number of different sources: “The equations and parameters in the ETOM [ETO Model] are based on academic papers, external databases, commercial reports, and expert judgement from both within and outside DNV GL. Examples of external databases used include IEA World Energy Balances, IRENA Capacity & Generation Database, Platts World Electric Power Plants Database, GlobalData Power and O&G Databases, Rystad Upstream Database, UN Comtrade Database, and Clarksons Shipping Intelligence Network”⁴⁷.

The report also provides examples of how DNV uses other scenario providers as a point of reference and comparison and for consistency checking. For example, “in our Outlook, we use the Physical Energy Content Method. This approach is in line with organizations such as Eurostat, IEA, and OECD, and allows for easy comparison with most other reference forecasts”⁴⁷. And it shows how, in some cases, DNV does not shy away from making choices that make their scenarios different from other scenario providers. For example, “contrary to other modelling frameworks, such as the IEA’s TIMES and the EU’s PRIMES, our approach does not ensure the global optimality of solutions”⁴⁷.

Use of scenario data

The informants described using scenario data from other providers, such as the IEA or the IPCC, mainly as a point of reference and comparison, for benchmarking, and “consistency checking” for instance of their own assumptions, but also for “inspiration”. Some informants spent a lot of time keeping up to date with scenarios from other providers and mentioned contact and collaborations with other providers. One informant mentioned that it is important for them to know what other commonly used scenario providers say, precisely because they are used by so many others.

The assessment and evaluation of scenario data from other providers was not based on formal processes but done mostly ad hoc and informally. There were no standardised approaches to the assessment and evaluation of external scenario data. But relationships with other scenario providers, including some degree of collaborations, “good dialogue” with specific institutions, and paid subscriptions – which sometimes involve more than just receiving the data (e.g. discussions or meetings) – were mentioned when asked what other scenario data were used. One informant mentioned that there was an explicit desire within the company to not formally collaborate with other scenario providers because they want to avoid being accused of not being independent.

Scenario data from other providers was actively assessed using expert judgment. One informant stated that “we first and foremost trust our own assessments, which we believe are as comprehensive as possible”. Several informants expressed that they don’t believe the world will evolve the way other scenarios imply, referring to specific aspects such as the rollout of electric cars or total emissions, where one informant believed the emissions reductions in other scenarios were too optimistic, and another informant believed the same emissions reductions were too pessimistic. In general, the scenario users in this group do not shy away from applying their expert judgment to criticise aspects of scenarios from respected sources. They do not passively accept external scenarios, nor the assumptions behind them. Even for key scenario inputs such as population and GDP growth, some of the informants chose to use assumptions that differ from those used by other scenario providers, even highly reputable ones, such as population estimates from the UN. Other informants stated that they don’t want to veer too far from the consensus view on areas where they have less expertise.

Choice of scenarios

Several of the informants expressed distrust and scepticism towards some scenarios or aspects of these. Sometimes this also acts as a motivation for developing own scenarios. For example, informants stated that some scenarios underestimate technology implementation, overestimate oil demand (oil companies and the IEA and EIA) or underestimate renewable energy (IEA). Sometimes, the reason to distrust scenario data from other providers were based on beliefs that these providers have reasons to give a pessimistic or optimistic picture of certain aspects of the future. For instance, one informant stated that they think scenarios from oil companies are too optimistic with regards to future demand, and that they are not able to provide an independent view. Another informant stated that the IEA’s view of renewable energy is “like drunk seamen” and that they simply ignore the IEA’s perspective on these aspects. Instead, they pick and choose from external providers based on what they “think different [scenario providers] are good at”.

Similar sentiments were expressed about scenarios that provide pathways toward climate targets. One informant said that “many of the scenarios that approach climate targets are extremely political in the sense that they accept what politicians say and take this as a given and ignore how difficult it is to deliver”. This informant stated explicitly that the explanation of how world energy demand in 2050 can be met with large reductions in fossil fuels is the biggest weakness of all these scenarios. The same informant stated that none of these scenarios are “very robust”, but that some scenarios “are based on what I believe are more robust assumptions regarding how difficult these things are to change politically”. They questioned whether the scenarios are political statements or statements based on a good understanding of technology and of how the market can be satisfied with different solutions and said that “very few emphasise scenarios that stray from what is politically correct, or politically desirable”. Contrary to this informant, a different informant expressed that scenarios from most providers are not “extreme enough” in the sense that emissions might go down much more quickly than we think.

The perceived lack of independence and biases of some scenario providers was stated by one informant as one of the reasons why they chose to develop their own scenarios. Independence or neutrality was also mentioned by other informants as an important aspect when choosing scenarios and they show an acute awareness of the political role of scenarios provided by different actors, in particular commercial actors (e.g. IRENA is “an institution set up to promote renewable energy”, the IEA “is a representative of rich industrial nations”, BloombergNEF “live off of giving advice on new energy finance”, scenarios from Greenpeace “are self-evidently not modelling but policy recommendations”). Given this, one informant states, it is important to cover the breadth of scenarios.

Finally, informants, sometimes the same informant, expressed varied and at times conflicting views regarding the purpose of scenarios. Perceived purpose ranged from realism, best guess projections, or even forecasting, to spanning the full range of possible outcomes. But forecasting, i.e. predicting the most likely of all possible futures, stands in conflict with depicting the full range of all possible futures. Related to this, informants express varied opinions about the use of probabilities in relation to scenarios.

One informant stated that a general strength of scenarios is their ability to span the entire possibility space, but also that scenario providers should be able to say something about what they think is more and less likely. A key motivation for the development of scenarios by this informant’s company is their use as inputs to strategic decision making and advice to customers, and for this they want to know what will most likely happen and what they can expect, rather than optimistic or pessimistic views on the future. This gives this company a clear interest in gauging the realism and accuracy of scenarios.

The use of own scenarios for decision making was important for several of the informants in this group. A second informant stated the importance of being explicit about the assumptions that decisions are based on, and that long-term investment decisions necessitate beliefs about the future; “it’s just a question about whether these [beliefs] are explicit or implicit”. This informant also said that, from an investment perspective, “for us, the most important thing is to find out what will actually happen”. And they expressed that for the purpose of providing meaningful decision support, a set of scenarios is less useful because, if a set of scenarios is provided, everyone just ends up using the middle scenario while ignoring the others anyway.

A third informant, on this topic, said that much of the groundwork of their scenario development is a necessary part of their operations regardless, because their operations require a long-term perspective. And for investment purposes, the most important period for decision making is the next decade or two: the outcome in 2050 is less important, because of discounting.

This informant also said that a strength of using scenarios with a long time horizon is that they can form the basis of reasoned and good discussions about possible pathways that are very different, but that this requires creativity and bravery in order to span the entire possibility space. The downside, they said, is that wrong use of scenarios can lock in decisions based on a wish or a belief. In the candid words of this informant, it is “garbage in garbage out no matter what model you have in the middle, and none of us know the outcome in 2050 anyway. So, in the end, it’s faith, hope and love”.

Discussion

In the following, we first discuss some main findings across the two groups of informants, before relating them to the theoretical framework introduced in section 2. Our informants were generally “informed users”, who recognise the uncertainties inherent in the modelling behind emissions scenarios and who are aware of the political stakes involved in preferring one scenario provider over another. Many of the scenario users were actively processing scenario data through their own calculations, modelling and scenario building. At the same time, the same users may in some instances also use scenarios more as ready-made stories to make a specific point or to justify existing policies. This mixed use of scenario data suggests that the distinction Kriegler et al.⁹ makes between “end users” and “intermediate users” of scenarios is not always useful: In practice, scenario data is processed, re-used and translated in ways that makes such distinctions less clear-cut.

In general, users are interested in emissions scenarios as illustrations of plausible ways to reach specific targets (such as the 1.5 or 2 °C targets), as providing a range of “likely” outcomes, or as “what if” scenarios that can be used to explore a clear storyline or a specific policy choice. Carbon prices were often of particular interest to policymakers, and both government officials and industry actors placed much emphasis on energy prices and technology costs. This may to some extent be specific to the case of Norway, where petroleum prices are of particular importance. Nevertheless, it is an important finding because it suggests that energy prices such as oil price may in many cases be used as an output from emissions scenarios. Because not all IAMs model energy prices endogenously, this may lead to a certain circularity between model inputs and outputs, as has been a long-standing criticism of global energy modelling²³.

Several interviewees called for assumptions and uncertainty in scenarios and IAM modelling to be more clearly communicated. In some interviews, scenario providers that communicated their assumptions more clearly were seen as more trustworthy and ‘citeable’. At the same time, too much variation in scenarios also reduces their relevance: Eventually, scenario users are often looking for specific numbers, either as a basis for further calculations, or for providing rhetorical legitimacy to decisions or policies. This points to a tension in how scenario data becomes “relevant” for users: On the one hand, emphasising uncertainty by providing ranges rather than point estimates is seen as increasing the credibility of scenario providers. On the other hand, providing uncertainty ranges and wide spreads (such as the ones contained in the IPCC scenario database) may reduce the immediate “useability” of scenarios in the eyes of users.

This tension can be analysed by placing different practices of scenario use along one of the dimensions introduced in our analytical framework, i.e. the distinction between using scientific data either to “open up” an issue for debate and interrogation, or to “close down” controversy to arrive at a certain conclusion or provide justification for a specific course of action. The different purposes to which scenarios are put by our interviewees may have both of these effects, as summarised in Table 1. However, some qualities in scenarios – such as an emphasis on communicating uncertainty or exploring a range of ‘what if’ possibilities—may be more aligned with uses that aim to “open up”, while other qualities—specific point estimates or unambiguous assertions of ‘likely’ outcomes—may be more aligned with uses that aim to “close down”. This illustrates how understanding scenario use along this dimension is a fruitful way of increasing awareness of the various uses of scenarios and their implications for deliberation and decision-making.

Table 1 Purposes of scenario use

Another main theme that cuts across both groups of informants is their awareness and explicit recognition of the political aspects of choosing one scenario provider over another. Generally, we see that actors tend to prefer and trust institutions that have a similar approach, problem-definition and value commitments as themselves—e.g., the IEA is favoured by energy actors, while the IPCC and IRENA are favoured by climate policy officials. Industry actors relate more actively to private scenario providers, but all interviewees also emphasise proven expertise and consistency over time as important determinants of trust in scenario providers.

Among government officials, we see a clear tendency that actors prefer scenario providers that in some way back up their own views vis-à-vis other government bodies: The Ministry of Climate, for example, favours the “more ambitious” renewable energy scenarios of IRENA over the more conservative ones from the IEA. Somewhat surprisingly, these choices—and the justification for making them—do not appear as clandestine political positioning. To the contrary: Actors are generally open about the choices they make, and why. They have explicit reasons for trusting one institution over another and for preferring one type of scenarios over others.

Having worked with or having institutional ties to a scenario provider is among the factors that several interviewees highlight as important for choosing specific scenarios. Another is providers’ perceived impartiality and independence. Here lies another notable tension: On the one hand, some of the factors that underpin trust in scenarios from a particular institution are related to its independence, for example by being free of commercial ties or specific political biases. On the other hand, trust is also enhanced by the closeness an institution may have to the subject-matter, for example by being close to the energy markets and therefore having particular insights into them, or by having direct connections through established institutional ties (such as government membership in the IEA or representation in the IPCC decision-making bodies) or, in the case of industry, commercial cooperation.

The different sources of trust in scenarios can be understood using the distinction between separation and integration set out in our analytical framework. The tension between organisational independence and close ties seen in our case study has been described as a general paradox in organising “policy-relevant” knowledge production on climate change^28,29. The IPCC, in particular, has been conducting a constant balancing-act between asserting scientific independence and ensuring a close connection to governments through its formalised decision-making procedures⁴².

A particularly interesting finding is that official bodies such as multilateral institutions generally seem to have higher trust and be more acceptable sources, especially for government officials and in official policy documents. The official status of organisations such as the IEA and the OECD may be taken to be both a source of independence from certain biases and as a form of integration with government processes. Generally, the case study illustrates how the distinction between integration and separation set out in the theoretical framework can offer a useful way of understanding the different factors that affect choice of scenarios and trust in scenario providers (see Table 2). Beyond this categorisation, however, it also clearly indicates that scenario choice can in many cases be seen as a political act, which is tied to the actors’ aims, interests and problem formulation. Furthermore, the fact that scenario information is often reprocessed and used as input for further analysis also adds an element of integration even in settings of formal separation: Different scenario providers are informed by each other’s work, which they use both for benchmarking, “sense-checking” and as input to their own analysis.

Table 2 Factors affecting trust in scenarios

Towards our conclusions, we summarize our findings in relation to the two research questions set out in the introductory section. In answer to the first question, we highlight three key findings from our explorative case study. First, scenarios play widely diverging roles and perform different functions for different scenario users. Even the purpose of emissions scenarios is contested, where the expectation that scenarios should span the possibility space conflicts with the expectation that scenarios should be realistic and depict more likely futures. This variation highlights that scenario use should not be seen as a linear “application” of scientific advice into policymaking, but rather as a diverse set of practices that is closely connected to the diverse aims and roles of those using the scenarios.

Second, our study indicates that many scenario users are aware of uncertainties and limitations of scenario data, as well as of the political aspects of both scenario production and scenario use. In some cases, scenario data is used despite uncertainties and other limitations because “numbers are needed”, which fits with expectations from the STS literature²². The explicitly political needs of decision-makers to justify policies may sometimes play an important role in the choice of scenarios, which, in combination with the large range of available scenarios, generates a persistent risk of “cherry-picking” scenario data. Even when specific numbers appear in a context of clearly stated uncertainties or debatable assumptions, they may in practical use become translated and presented in ways that make them appear as definitive facts. This is seen, for example, in the way median oil price values from the IPCC database were used to justify existing forecasts and policies for future oil production (see Box 1). Awareness about these dynamics provides a caution to modellers that even if simpler communication is in demand from policymakers, there are reasons to “keep it complex”⁴³ in order to highlight the choices and trade-offs that policymakers should be grappling with. Tailoring the production and communication of scenarios to the kind of use that “opens up” space for deliberation and political debate (as opposed to “closing down”) may provide a strategy more resistant to an over-reliance on scenario data as quantified truth.

Third, aspects other than the scenarios or the models behind the scenarios, such as trust in scenario providers and institutional ties, have an important impact on what scenarios are used by different scenario users. Informants note that providers have to be “citable”, which may favour providers that are generally seen as authoritative, such as the IEA. The informants are also cognisant of differences in perceived expertise between institutions, and whether institutions are seen to favour certain aspects (for example, renewables or fossil fuels). Overall, the credibility of the scenario providers and their perceived strengths and weaknesses appear to be key. Credibility can however, be achieved both through “integration” and “separation”—two different ways of organising the relationship between the process of scenario production and scenario users which can be in tension with each other. This suggests that while an increased orientation towards ensuring relevance is probably welcome, modelers should not assume that participation by itself will increase relevance or uptake. A key question is also what kind of institutional ties are established, and to what extent the providers of scenarios are seen as trustworthy and reputable.

In response to our second research question, we have suggested a framework for how the diversity in scenario use that we observe can be analysed. The framework draws on two conceptual dimensions – between the aims of “opening up” and “closing down”, and between closeness and separation in the organisation of scientific policy advice—as well as the general lesson from STS regarding how science and policy tends to interact in a non-linear fashion. Applying this framework demonstrates the value of an STS perspective on the use of emissions scenarios in practice, and provides reason to caution against simplistic assumptions about scenario users as a homogenous group passively receiving or ‘implementing’ scenario data. The emerging picture from our study is that many scenario users actively work to make sense of scenarios and navigate within a broad landscape of scenarios and scenario-producing institutions based on their specific needs and expertise. In this landscape, numbers are given special authority, and institutional ties and trust – as well as the aims toward which the use of scenarios is directed—play significant roles when actors choose what numbers to use.

It should be noted that the scenario users in this study were generally highly educated and knowledgeable regarding scenarios. Other scenario users that were not included here, for example, higher level decision makers, negotiators, or users in industries that do not themselves produce scenarios, are likely less aware of uncertainties and limitations of scenario data and may therefore use scenario data less critically. Furthermore, other studies indicate large differences between Northern and Southern countries in their capacity to work with scenarios¹¹. This points to the need for further explorative studies as well as more systematic mappings of scenario use across national contexts.

Our case study also holds important insights for discussions about how the IPCC can maintain its relevance in providing knowledge for climate action. Among our informants, the IPCC is not the main or generally preferred provider of emissions scenarios. Other international institutions, in particular the IEA, as well as a range of private providers, are in many cases seen as more relevant, usable or “citable”—for several reasons. The tensions our analysis has shown in the use and re-packaging of scenario information—between integration and separation when it comes to trust in scenario providers, and between using scenarios for “closing down” or “opening up” discussions—are inherently difficult for the IPCC to handle, as they go to the core of the IPCC’s own balancing act as being “policy relevant but not policy prescriptive”^42,44.

Due to the high diversity in scenario use and re-use across different actors, keeping track of and being responsive to scenario use may be a challenging task for the IPCC system. Our analysis further demonstrates that the lines between users and producers of scenarios are not always clear-cut. Against this backdrop, it could be argued that relevance of IPCC assessed scenarios on a general level could be improved by paying closer attention to how and why IPCC scenarios are used vis-à-vis other scenarios and scenario-providers, and draw on those insights when scoping, writing, reviewing and approving future reports. This would resemble elements in the reform agenda “Orchestrating broad knowledge generation” outlined in the introduction to this special collection¹³.

On a more general level, being open to the messy and performative effects of scenarios implies that the roles of scientific expert bodies such as the IPCC or various IAM modelling groups is not simply as “facilitators” of climate action⁴⁵. Rather, the production and circulation of scenarios must be seen as a form of climate action in itself. This means that we need to study how scenario users act with scenarios, rather than simply “applying” them as “end users”. Similarly, we need to understand the IPCC and the IAM modelling community as acting with scenarios, rather than simply “communicating” them. This, we suggest, provides the most fruitful entry point for reflexively discussing the political implications of scenarios as a form of climate action.