Affective integration in experience, judgment, and decision-making

Abstract

The role of affect in value-based judgment and decision-making has attracted increasing interest in recent decades. Most previous approaches neglect the temporal dependence of mental states leading to mapping a relatively well-defined, but largely static, feeling state to a behavioral tendency. In contrast, we posit that expected and experienced consequences of actions are integrated over time into a unified overall affective experience reflecting current resources under current demands. This affective integration is shaped by context and continually modulates judgments and decisions. Changes in affective states modulate evaluation of new information (affect-as-information), signal changes in the environment (affect-as-a-spotlight) and influence behavioral tendencies in relation to goals (affect-as-motivation). We advocate for an approach that integrates affective dynamics into decision-making paradigms. This dynamical account identifies the key variables explaining how changes in affect influence information processing may provide us with new insights into the role of affect in value-based judgment and decision-making.

Introduction

Affective states influence judgments and behavior^1,2,3,4,5. Nevertheless, within the rich research literature on affect and decision-making, many findings are contradictory and there is a lack of consensual mechanistic explanations for how affect influences judgment and decision-making^6,7. Hence, we need new approaches to reach a satisfactory understanding of how affect influences and better predicts judgments and decisions. We propose that one of the under investigated aspects for the study of the link between affect and decision-making lies in the temporal dimension. The experience of affect and its influence on judgment and decision-making are time- and context dependent. To better understand the continuous and changing involvement of affective processes in decision-making, a sufficiently good representation for the construction of affective experience over time should be integrated into behavioral paradigms and models.

Signals that are important for decision-making (e.g., expectations, reward, and loss) prompt changes in the affective state of the individual^1,8,9, which are integrated over time into a unified overall affective experience^10,11,12, a process we call affective integration. This dynamic integration is shaped by context and goals, which makes affect is a constructive process best described as a hedonic summary of recent events and expected consequences of actions. Hence, there is great potential in measuring and modeling affective integration over time together with carefully defined formal decision models to provide mechanistic explanations for the role of affect in judgment and decision-making. The recent advances in continuous measurement techniques^13,14 (see BOX.1) and computational modeling of subjective feeling states^{12,15,16,17,18,19} (see BOX.2) make it possible to study changes in affect over time and its continuous involvement in decisions.

Box 1 Measuring affective experience

Affect and emotions have experiential, physiological, and behavioral components. Hence, affective science has typically relied on three broad methods to measure and quantify affective processes¹¹⁵. Subjective self-report measures usually adopt Likert scales and are often seen as gold standard to assess how individuals feel. These commonly involve participants rating their current (or anticipated) feelings based on several adjectives (e.g., calm, happy, anxious, pleasant). This method is also widely used for repeated sampling of individuals’ affective experiences in their natural daily environments (called ecological momentary assessment; EMA). EMA studies are commonly used for understanding the individual variation in affective dynamics⁶¹. However, these studies operate on a longer temporal perspective (e.g., hours or even days) in comparison to the current article and typically are not integrated in a decision context.

Furthermore, since affective processes often involve changes in autonomic nervous system activity it is common to use physiological measures (e.g., skin conductance, cardiovascular measures, or pupil dilation). Although physiological reactions when combined with self-reports could help build a more detailed assessment, changes in many of these signals can be sluggish (e.g., skin conductance) and span over several trials in a typical experiment, which may make them difficult to integrate in the dynamic approach we present in the main text. Moreover, behavioral responses have been also utilized as measures of affect and emotions, which generally involve documenting facial expressions and, to a lesser extent, vocal and bodily expressions¹¹⁵. However, recent findings documenting wide variability and weak reliability^116,117 suggest that facial expressions may be less informative than previously thought.

For understanding the continuous impact of affective experience on judgment and decision-making, measures should ideally be continuous and able to capture rapid changes before and after a decision. A continuous measurement method with a high resolution could enable a fine-grained assessment of affect. To be able to capture rapid changes the measurement scales should be intuitive and applicable across tasks and contexts. For instance, it was recently suggested that a revised version of a two-dimensional (valence and arousal) scale, called Affect Grid¹¹⁸, could be integrated in decision tasks to understand the role of affective dynamics in judgment and decision-making¹³. To assess temporal fluctuations in affective experience such a scale could easily be used several times within a trial and/or task.

Previous research suggests that a small number of dimensions could explain much of the variation in momentary subjective affect¹¹⁵, the main dimension being valence. In the main article, we advocate for the two-dimensional core affect perspective to represent momentary affect. However, there may be other dimensions that are also important in conceptualizing affective states (e.g., control or novelty). Importantly, a subjective affective experience model based on other dimensions than valence and arousal (e.g., positive affect / negative affect, or valence/arousal/control) could easily be adopted in the dynamic approach we advocate, provided that it models how affective experience is constructed over time and updated in the face of the ongoing information flow.

Box 2 Modeling affective experience

Humans navigate complex environments and process a constant stream of information flow. Affective states fluctuate in a moment-to-moment fashion during this ongoing stream, reflecting the continuous relationship between the individual and their environment. However, there are open questions about how a stream of input is dynamically represented in affective experience. Recent advances make it possible to use formal computational models to interpret and analyze affective fluctuations as a function of events, stimuli, and actions^15,16,19. Computational models are flexible tools that can be used as data analysis methods to investigate complex patterns and validate hypotheses¹¹⁹. They can be formulated based on theories and used as tools for theory testing and comparison^120,121. A specific formulation of a model ensures that the relationships between constructs are defined explicitly and mathematically. Hence, the interpretation is also based on explicit mathematical properties of the model parameters, limiting researcher degrees-of-freedom in analyzing the data and interpreting the outcomes. For our purposes, computational models of affect, with their precise mathematical formulations, help us theorize and understand how recent events induce affective changes, how long these affective changes endure, and how goals and individual differences influence this process.

There have been recently several attempts to model momentary fluctuations in affective state as a function of ongoing information processing. One such influential model assumes that a subjective affective state is generated based on a leaky integration of affective consequences of recent reward expectations and prediction errors resulting from these expectations¹². This model has been adapted to study affective state in various contexts such as risky choice, reinforcement learning, motor challenge task, and voluntary attention task^{12,78,79,96,97,99,122,123}. Furthermore, it has been suggested that affective states represent a moving average of reward prediction errors and bias perception of subsequent rewards¹¹, which can accelerate learning of optimal actions¹⁸. In these models, parameters define how outcomes, prediction errors, and subjective values of the current and future states accumulate to influence affective experience, which in turn influence subsequent actions through biasing perception of rewards^17,18,80,103. Another recent perspective argues that a momentary mood state represents expected values of all possible actions enabled by the context that biases the decision to seek a particular reward¹²⁴. The simulations show that a flexible, in comparison to an unchanging and neutral, affective state leads to more adaptive choices in response to changing environmental circumstances.

Affect terminology

In affective science, there are longstanding debates about the definition, causation, nature, and consequences of affective states. Here, we will briefly clarify how we use different terms to avoid confusion and demarcate the focus and scope of the current article, which is not on specific emotions but rather on low-dimensional, continuous, and valenced affective states.

The main task of the brain is to manage resources in physiological systems to ensure survival by producing physiological adaptations to meet anticipated demands²⁰. It also continually represents the bodily consequences of these physiological adaptations²¹. Affect is linked to these ongoing sensory changes that result from physiological systems such as the autonomic nervous system, the immune system, and the neuroendocrine system^22,23. Thus, affect is a fundamental aspect of human experience that could be defined as the constant stream of fluctuations in one’s neurophysiological state²⁴. This suggests that every waking moment is infused with an affective tone²⁵. Studies have reliably shown that this continuous affective state is subjectively experienced as a feeling ranging from pleasure to displeasure (i.e., valence), accompanied by a certain degree of activation^23,24,26,27.

Emotions are also valenced affective states. We consider emotion as an intense, short-term and an object-focused state. The notion of valence is also at the heart of this definition, but emotions are multidimensional and complex affective states. Psychological constructionist theories of emotions posit that the interplay between ongoing activity in the physiological systems, contextual information, cognitive appraisals, and conceptual knowledge can lead to a range of emotional experiences that are tailored to a given situation^22,24,28. The exact constellation and timing of various components and mechanisms through which emotions are constructed vary in different theories, but arbitrating between these different theories of emotional experience is not the goal of the current article. While we acknowledge that more complex, higher-order emotions are important for judgment and decision-making, we limit our primary focus on how low-dimensional and continuous affective state (i.e., a valenced state representing the continuous neurophysiological activity) fluctuates in the face of ongoing information flow from our surroundings and how these fluctuations continuously influence judgment and decision-making.

Affect and decision-making

Traditionally, research on value-based decision-making has largely relied on rational choice models, in which decision-makers are assumed to maximize utility^29,30. The magnitude of the outcome and the subjective probability or the belief of the occurrence of that outcome are the main determinants of the overall utility for each alternative (i.e., subjective expected utility theory - SEU)³¹. In the SEU framework, decisions can be aggregated as a numerical utility function. Normative choice theories such as SEU theory have for a long-time dominated the research on value-based decision-making³². Descriptive theories were developed as alternatives allowing for more realistic assumptions about how people actually make decisions. Prospect theory³³ is arguably the most important descriptive theoretical framework ever developed in the field of judgment and decision-making. The heart of the theory is the value function, proposing that the carriers of value are positive or negative changes from a reference point. The function is nonlinear, reflecting diminishing sensitivity to magnitude (i.e., decreasing marginal utility) and allowing for asymmetric values for gains and losses, where the psychological value of a loss is magnitudes larger than a comparable win (i.e., loss aversion). Kahneman³⁴ observed that “if prospect theory had a flag, the value function would be drawn on it” (p. 282).

However, these theories treat decision making as a process isolated in time: decision makers make the decision based on a summary description (decisions-from-description) of the likelihood and magnitude of an outcome (e.g., a choice between receiving100 USD with certainty or a 50/50 chance of receiving either 200 USD or 0 USD). The main problem here is that even though these theories may allow for preferences to be influenced by changes in decision-maker’s internal state and context, they assume no dependence between the two states of the decision-maker. Recently, theories suggesting that this is far from how people make decisions in everyday life instead model human decision making as a dynamic process – we learn about choices from case-by-case observations (decisions-from-experience)^32,35,36. For example, models focused on exhaustive sampling of information reject the notion of utilities³². Instead, decision makers are assumed to make binary comparisons between the present option and alternatives drawn from memory and the current environment³⁷. So rather than to rely on explicit, aggregated representations of “utilities”, decision makers accumulate evidence about the state of the world through a localized experience process akin to basic perceptual processing³². Thus, rather than departing from economic assumptions (i.e., homo economicus choosing to maximize utility), these models rely on a few simple assumptions about the cognitive nature of humans (e.g., people rely on small samples; recent information is weighed more heavily; people keep a rough count of the frequency of experiences) in explaining how decisions are made^36,37. Remarkably, even though such models start from making no assumptions about “utilities”, they can recreate basic predictions of normative or descriptive utility frameworks. For instance, decision-by-sampling can produce concave utility functions, exponential discounting for decisions with delayed outcomes, and overestimation of small probabilities and underestimation of large probabilities³⁷.

But both the decision-from-description and the decision-from-experience classes of theories are relatively silent about the role of affect in decision-making. This is surprising as affect and emotion nowadays is seen as central for behavior³⁸. In a 2009 Annual Review of Psychology article on judgment and decision-making, Weber and Johnson concluded that the field had experienced an “emotion revolution” where affective processes are seen as an efficient heuristic that can produce and motivate adaptive behaviors, rather than a bias leading to irrational behavior³⁰. Similarly, in a recent call to “rise of affectivism”, it was suggested that the inclusion of affective processes in models of judgment and decision-making not only explains affective phenomena but also cognition and behavior more broadly³⁹.

Functions of affect in judgment and decision-making

Affect can serve various functions in motivating decision behavior. Strong affective reactions can directly elicit approach or avoidance behaviors⁴⁰ or by changing the way information is processed and how information is weighted⁴¹, and low-intensity, subtle affect appears to have a pervasive influence on our thoughts, behaviors, and judgments¹. In many situations, rather than relying on analysis and deliberation, people rely on their affect as input into the decision process (affect heuristic¹). In such cases, the experienced feelings are used as information to guide judgment and decision making⁴². The affect heuristic work assumes that people consult or refer to an “affect pool” containing positive and negative tags consciously or unconsciously associated with the representation of the decision problem. Thus, affect assigns value to the object of judgment that seemingly is the cause of affect (affect-as-information). Importantly, both integral affect (affect that is experienced while considering the object of judgment) and incidental affect (affect that is independent of the object of judgment but can be misattributed to it) are used in judgments and decisions in this way. However, it is important to note that integral affect is more likely to lead to adaptive use of affect in judgments and decisions as it is normatively related to and representative of the decision at hand. Incidental affect, from a normative viewpoint, can be seen as a “bias” as it is incorrectly attributed as being related to the decision⁴³.

Affect can also motivate decisions through goal-directed behavior (affect-as-motivation). In early work on the role of affect in decision making, Isen⁴⁴ showed that positive affect often leads to risk aversion when the decision task is realistic, which can be explained as mood maintenance. Participants that are in a positive mood risk “losing” their good feelings if the outcome is negative. One interpretation of this result is that positive mood participants have more to lose than neutral or negative-mood participants. Moreover, affect can act as a control system signaling how good we are doing in goal-attainment⁴⁵. We experience negative affect if we are doing worse than expected (i.e., negative prediction error) and positive affect if we are doing better than expected (i.e., positive prediction error). Hence, affect experienced in relation to our expectations signaling how fast we are approaching a goal may alter motivation for subsequent behavior. Furthermore, affective experience is modulated by how events, in a given situation, are evaluated or appraised in terms of their novelty, goal-relevance, and significance²⁸. A wealth of empirical research and theories suggest a link between these kinds of appraisals and action tendencies as well as motivations to approach or avoid⁴⁶.

Finally, affect can act as a spotlight shifting attention and weighting of information in decisions (affect-as-a-spotlight). For example, negative mood participants sometimes engage in more careful and deliberate information processing (cognitive tuning⁴⁷). Rottenstreich & Hsee⁴⁸ showed that affect systematically influenced weighting of information, especially probability. When faced with a small probability of a strong affective event (e.g., an electric shock) decision makers weight the possibility rather than the probability – creating an insensitivity to probability information (i.e., a willingness to pay almost the same amount to insure against a 1% risk and a 99% risk of receiving an electric shock). On the other hand, a weak affective event (e.g., loosing 20 USD) grows more linearly with its probability (i.e., participants pay low amounts to insure against a 1% chance of losing 20 USD, and a significantly large amount to insure against a 99% probability). Taken together, it seems as these three broad mechanisms (affect-as-information, affect-as-motivation, and affect-as-a-spotlight) can appear to capture many of the observed effects of currently experienced affect on judgments and decisions³⁰.

Still, it is difficult to answer the question “what is the role of affect in judgment and decision making?” as there are many contradictory observations explained through a large array of theoretical mediators. For instance, positive affect is associated with increased risk-taking in some studies (a mood-congruent, informational effect^{e.g.,49,50,51}) and decreased risk-taking in others (a mood-incongruent effect^{e.g., 52,53,54}). Affect-as-information account may predict that positive affect increases the perceived value of winning and thus drive risk taking for a potential gain. Affect-as-motivation, on the other hand, may predict that positive affect attributed to better-than-expected progress could cause individuals to be more risk-averse to protect their mood state. The prediction of affect-as-a-spotlight view would depend on how the probability information is weighted. As a result, an intense positive affect may increase risk taking for a small probability gain but not so much for a high probability gain. Thus, whether the valence of an incidental mood increases or decreases risk-taking depends upon which mechanism dominates. However, the research paradigms used in linking affect and decision-making are often unable to arbitrate between these alternative mechanisms. One reason for this is that many studies tend to focus on connecting a feeling state to an increased or a decreased tendency for a certain response. Recent advances in psychological science underlines the shortcomings of this type of essentialist feeling-to-behavior mapping that overlooks contextual factors^55,56,57,58. Second, affect is often operationalized as feeling states that is implicitly (due to study design and methods) or explicitly (to simplify the problem) assumed to be static and limited in time to only here and now, a typical manifestation of the snapshot approach, which do not account for the fact that mental states and behavior are temporally dependent. Third, the traditional scientific paradigm in psychological research regards context effects as nuisance or moderators of underlying processes that can be removed by study design and analysis⁵⁶, e.g., by randomizing trials and analyzing summary scores. Collectively, these approaches are representative of a problem that is prevalent in the vast bulk of the paradigms and data on this topic – we effectively ignore the temporal nature of mental states and its dynamic impact on judgments and decisions. We argue that by ignoring the continuous nature of affect, the field can make little progress beyond simple models, and a better and systematic understanding of the mechanism through which affect influence judgments and decisions will be hard to achieve. To reconcile this issue, we suggest that there is a need to adopt an approach that attempts to understand how affect is constructed from the ongoing information flow and how this evolving state modulates information processing feeding into judgments and decisions continuously (for a similar approach, see⁵⁹).

The temporal nature of affect and behavior

The brain produces physiological adaptations to meet anticipated demands due to biological and environmental circumstances²⁰ and continually represents the sensory consequences of these physiological changes^21,60. Affective states are linked to these ongoing sensory changes within the body’s physiological systems (e.g., autonomic nervous system, immune system, neuroendocrine system)^22,55. Therefore, affective states result from the natural bodily fluctuations and changes that are prompted by sensory information from the surrounding world and can be defined as a neurophysiological representation of an individual’s ongoing relationship with their environment²⁴. Hence, affect is a dynamic process acting as a hedonic summary of recent events and varying expectations, and the weighted integration of events into this dynamic summary is shaped by the current context. In fact, accumulating empirical evidence as well as constructionist theories of emotions suggest that contextual factors including subjective evaluation of them are central in forming affective experience and behavior^22,24,28,56.

Since affect is continuous and dynamic, its impact on behavior must also be continuous and dynamic: affect modulates ongoing information processing underlying our choices (affect-as-a-spotlight) and temporarily guides behavior via altering subjective value associated with outcomes (affect-as-information) and motivational states (affect-as-motivation) in a context-dependent manner. However, in many studies, researchers use study designs and statistical analyses with the following underlying assumptions: (1) the observed response and behavior depend solely on the currently presented information and stimuli, and (2) the variation that is not explained by the current trial structure is noise. Note that these assumptions may be implicit, meaning that researchers may not explicitly introduce these boundaries. However, the ways the studies are designed, and the data is analyzed effectively introduce these assumptions, which are strictly at odds with temporally dependent nature of mental states and behavior^{56,61,62,63,64,65}. Even in scenarios that researchers use to exemplify or communicate how affective states are induced, a single stimulus such as coming across a bear in the forest evokes a cascade of reactions resulting in a mental event and a subjective experience in a (mostly passive) perceiver with no previous physiological or affective state. Thus, the dominant scientific paradigm is effectively an investigation of decision-making and mental states within discrete time steps that are independent from one another (i.e., the snapshot approach). But we know that behavior and mental events are rarely discrete or static. We receive and accumulate information across a temporal dimension^66,67,68. The brain processes information in a temporally dependent fashion and neural activity follows multiple time courses⁶². During perceptual processing, neural response to sensory stimuli depends on previous activity^{69,70,71,72,73}. Moreover, mental events like affect and emotion^10,45,74 as well as social perception⁷⁵ depend on integration of information over time. Importantly, this information integration is not a stimulus-to-response type process, and it involves both internal and external stimuli as well as contextual factors. For instance, a recent investigation reported that induced heart-rate changes in mice enhanced anxiety-like behavior only in a risky context and not in a safe context, which provides direct evidence that the brain integrates signals from the body with external sensory signals and context information to construct affective states that are tailored to the given situation⁷⁶. Thus, theorizing, modeling, and experimental paradigms must allow for the fact that affect and behaviors are continuous processes.

Integrating affect and behavior along the temporal dimension

Combining the properties of momentary affective experience (i.e., a hedonic summary of recent events and varying expectations integrated over time) with the temporal dependency of mental states and behavior generates an approach to studying the involvement of affect in judgment and decision-making. An individual’s continuous affective state can be represented as a dynamic integration of recent prediction errors and expected consequences of actions in the face of changing environmental demands. Therefore, affective state as an internal signal carries information about the availability of rewards and punishments perceived by the individual as well as current resources to act on the environment. The current affective state and its rate of change influence, via attentional deployment, detection of currently available actions and weighting of decision attributes (affect-as-a-spotlight) and, consequently, the decision process, via biasing the value assigned to decision options (affect-as-information) and modulating the likelihood of some forms of actions over others in relation to current goals (affect-as-motivation).

The first key aspect of this approach is an affect model that defines the key parameters for the construction of affect over time. To be relevant for judgment and decision-making research, this model should be able to parametrize how recent events and outcomes together with varying expectations and perceived uncertainty are integrated over time, and how this integration is shaped by goals and context. Obviously, the exact constitution of input variables will ultimately depend on the current task demands (e.g., whether learning from previous outcomes is adaptive, whether the affective fluctuations are prompted by task events or incidental cues, etc.). There have been recent advances in computational modeling of subjective affective states and how they represent information from the surroundings (BOX.2). The second key aspect is then quantifying the decision process using a formal computational choice model depending on the decision context (see Table1), with a parameter space defining how different actions are evaluated and compared, and whether there are biases. Finally, the temporal variation of the decision parameters due to current affective state and its immediate rate of change can be introduced into the model (see blue arrows marked as ‘Affective modulation of decision process’ and ‘Affective modulation of outcome perception’ in Fig.1). In practical terms, the affective state fluctuations predicted by the affect model are allowed to temporarily modulate the likelihood of model deployment, the decision parameters, and the subjective evaluation of the outcome. Critically, at this step, it is possible to formally test specific hypotheses and effects, such as the mood-congruent evaluation of decision alternatives and/or outcomes (affect-as-information), changes in affective state modulating subjective value computations and/or evidence accumulation (affect-as-a-spotlight), and affective influences on goal-directed behavior (affect-as-motivation). Importantly, these different mechanisms would be mathematically formulated in the models. Thus, this approach enables researchers to investigate various functions of affect simultaneously without having to average out potentially meaningful contextual variation as noise. This would lead to an improved ability to arbitrate between different mechanisms through which affect influences behavior.

Table 1 Brief descriptions and definitions of the two general decision-making contexts, commonly utilized formal choice models, and the decision processes that the models can explain. The models could be used in both decision contexts and in some cases even in combination

Fig. 1: Conceptual framework for investigation of affective integration in experience and choice.

The affective state of the individual, represented by the two-dimensional pleasantness and arousal features, fluctuates over time representing a continuous integration of expected consequences of actions (i.e., predictions) as well as experienced outcomes and prediction errors (orange arrows at the top of the figure). The current affective state (sAt) and its rate of change (ΔsAt) influence the attentional deployment and detection of the current action options. Depending on the context, this can also be characterized as affective state fluctuations influencing the deployment likelihood of decision models (Mi). The decision process then is represented using a formal choice model depending on the current task demands with a parameter space defined in θ_C (see Table1), which defines how different options are evaluated and compared to reach a final choice. The framework also includes an affect model, defining the key dynamic parameters for the construction of the affective state over time. This model should ideally parametrize (θ_A) how recent events together with varying expectations and uncertainty are integrated over time, and how this integration is shaped by goals. But of course, the exact constitution of input variables may ultimately depend on the current task demands. The pleasantness and arousal features of the affective state predicted by the affect model is then allowed to temporarily modulate the decision parameters (quantified by the parameter space, β) and the subjective evaluation of the outcome (quantified by the parameter space, δ). Critically, here it is possible to formally test specific hypotheses and theories, such as the mood-congruent evaluation of a reward (i.e., pleasantness linearly modulating the outcome evaluation) or arousal influencing subjective value computations and/or evidence accumulation (arousal modulating decision parameters θ_C).

Affective integration in experience

Recent studies have reported that momentary affective experience can be represented as temporal integration of affective consequences of events^10,77. Typically, in these studies, research participants view stimuli, perform tasks, and report how they feel at various time points. The investigation of how affective impact of multiple evocative stimuli is integrated into an overall affective experience shows that (1) the currently experienced affect is shaped by a temporal integration of the affective consequences of recent events with prior affective state and that (2) this affective integration occurs according to a recency-weighted averaging model^10,77,78. The temporal covariation in peripheral physiology (i.e., skin conductance, heart rate, respiration, and facial muscle activity) and subjective affective experience further supports this model¹⁰. Moreover, the integration weights are influenced by predictability of the affective context⁷⁷ and goal-relevance of the stimuli⁷⁸. Specifically, when affective context is manipulated by controlling the occurrence probabilities of positive and negative stimuli, individuals’ expectations about the upcoming events were altered, which modulated the temporal representation of affective experience. An unpredictable affective context (i.e., randomly occurring pleasant and unpleasant stimuli), critically, increased the impact of the most recent stimuli, which can be seen as the affective system narrowing its temporal focus to the most representative information for the current context⁷⁷. Moreover, context uncertainty, independent of context pleasantness, was associated with an overall increased negative affect. A separate investigation reported that task-relevant stimuli, independent of their normative propensity to induce affect, had a larger impact on momentary affect compared to task-irrelevant stimuli⁷⁸. Taken together, these findings collectively indicate that affect reflects a dynamic summary of recent events whose affective consequences are integrated over time. This integration is shaped by context, uncertainty, and goal-relevance, in a way that recently occurred events and events informative about the current context and goals have a relatively larger impact.

Furthermore, it has been shown that momentary affective experience during decision-making tasks reflects a temporal integration of reward expectations and prediction errors resulting from these expectations^{11,12,79,80,81}. The formulation that the valence feature of affect is a function of recent prediction error history effectively means that experienced pleasantness signals whether an environment is getting better or worse than expected in terms of available rewards, called environmental momentum⁸⁰. Hence, fluctuations in valence become a useful signal for learning and adaptive behavior¹⁶. The influence of prediction error history on temporal representation of affective experience has also been shown in more naturalistic settings^82,83. Moreover, expected consequences of actions induce affective changes. Anticipation of rewards and losses induce positive and negative states with a level of arousal proportional to the uncertainty of the expected outcome⁸⁴. People also generate anticipatory autonomic arousal responses when considering a risky option⁸⁵. Thus, the construction of affect incorporate elements such as “anticipatory affect”^84,85 and experienced utility from anticipation⁸⁶ through input from the current environment as well as possible future outcomes.

These findings also provide support for several theoretical frameworks focusing on the temporal unfolding of affective experiences. In general, appraisal theories of emotion posit that an event is evaluated based on several appraisal criteria (e.g., novelty, goal relevance, pleasantness^28,46). In other words, affective experience depends on novelty, goal-relevance, and pleasantness of events. Novelty aspect can be directly linked to prediction errors since novel or unexpected events generate larger prediction errors, and the sign of the prediction error likely depends on whether the event is evaluated as pleasant or unpleasant. Furthermore, goal-relevance of an experienced outcome modulate how it will eventually be integrated into the ongoing affective state. These appraisal criteria can be a useful starting point in investigating contextual and individual variation in affective integration. However, note that most appraisal theories are interested in the temporal dynamics of emotional experience within a window that starts with an event and ends in experience, which is a different scope than the current perspective. Another theoretical model of affective dynamics to note is the iterative reprocessing model, a dynamical-systems account of human information processing⁷⁴. The model underlines the importance of temporal dependence of mental states and posits that affective experience at a given point in time (t) is determined by an integration of affect experienced and anticipated at a previous time point (t-1) and the currently processed information. The empirical evidence reviewed in this section is clearly in line with this prediction.

Taken together, empirical findings and theoretical frameworks reviewed here suggest that affective consequences of experienced and anticipated events are integrated over time, and this integration is not universal but instead is shaped by context, uncertainty, and goals. Hence, affective experience is a contextual hedonic summary of recent events and varying expectations about the consequences of actions. Importantly, the research efforts that attempt to understand affect as a temporally dependent and continuous state may enable researchers to gain insights into mental health and well-being. A recent review argued that measuring and modeling momentary subjective feeling states during decision-making paradigms may provide insights into psychiatric disorders¹⁹. Understanding how affective dynamics change during well-controlled decision tasks¹⁹ as well as everyday settings⁶¹ in clinical populations may contribute to a better understanding of the underlying psychiatric disorders.

Affective integration in judgment and decision-making

Judgment and decision-making depend on internal beliefs about the hidden states of the world³⁰. Information processing about rewards, punishments, prediction errors, and counterfactuals influence these beliefs through updating expectations and confidence. As we have discussed above, affective consequences of expected and experienced events are integrated over time into a unified affective state. Thus, it is very likely that this changing affective state continuously modulates decision-making. In this section, we highlight and give examples of several contexts where considering affective dynamics can better help explain the influence of affect on judgments and decisions. Specifically, we review recent studies that go beyond the traditional randomized trial structures, attempt to incorporate the fact that mental states and behavior are temporally dependent, and/or do not average out potentially meaningful variation as noise. We argue that these are necessary to understand the construction of affective experience over time and its continuous influence on judgment and decision-making.

Risky choice

Risky choice paradigms, wherein a decision-maker typically has explicit information about the payoffs and probabilities (i.e., decision-from-description), is widely used. Typically, risk-taking behavior is summarized and analyzed across test sessions since risky choice has been taught of as reflecting stable preferences^4,6. This approach, in light of the current discussion, presents challenges and shortcomings when it comes to studying the role of affect in decisions. First, a plethora of investigations show that preferences are in fact both contextually and temporally dependent (e.g., decoy effects⁸⁷, framing⁸⁸, evaluation mode⁸⁹), even in controlled laboratory tasks specifically designed to counter such effects⁹⁰. Second, affect has been shown to be linked with reward anticipation, experienced outcomes, prediction errors, counterfactual information, and risk-taking^{91,92,93,94,95}, all of which randomly change from one trial to the next, thanks to experimenter-forced randomized trial structure. Considering these together with the dynamic nature of affect, it is safe to assume that within-task affective state fluctuations caused by this complex information flow may temporarily influence decision-making, which may not be easily captured by summary statistics. We believe that risky choice is an appropriate domain for the current approach to study affective dynamics and behavior. In fact, there are a few promising recent studies corroborating this claim. Vinckier and colleagues⁹⁶ report that fluctuations in subjective affective state, evoked incidentally and modeled as a leaky accumulation of recently encountered events, were expressed in baseline neural activity, which in turn modulated how gain and loss outcomes were weighted in subsequent choice (a form of affect-as spotlight)^96,97. Critically, the ongoing neural activity reflecting these affective state fluctuations were entered into the formal choice model to modulate decision parameters. The findings showed that increases in positive and negative affect led to an increased weights assigned to gain and loss prospects, which resulted in higher and lower risk-taking^96,97. Adopting a snapshot approach and averaging choices across trials could have led one to interpret this finding as ‘positive mood increases risk-taking.’ However, modeling of trial-by-trial variation of choice parameters due to affect dynamics clearly shows that affect alters the evaluation of positive and negative cues, which gives us more insight into the function of affect (affect-as-a-spotlight) in the observed behavior (increased or decreased risk-taking).

Another study investigating trial-to-trial affective changes on risk-taking behavior showed that within-task events determine both affective fluctuations and subsequent choices⁹⁸. The findings show that experienced valence is related to recent prediction error history and larger gain outcomes, while increased risk-aversion is associated with high arousal states. Finally, another study⁹⁹ showed that momentary affective experience reflects a temporal integration of varying expectations, uncertainty, and prediction errors and that experienced arousal impacts subsequent choices via temporarily modulating subjective value computations. Specifically, a momentary increase in arousal was associated with increased loss-aversion, larger diminishing marginal returns, and a greater choice consistency for the subsequent decision. Taken together, these studies show that affective experience continuously encodes varying expectations and uncertainty that underlie choices, and it keeps track of recent prediction error history. Critically, this dynamic representation of affective state continuously modulates behavior via changing the weights assigned to experienced and expected outcomes.

Learning in uncertain environments

In experiential paradigms, unlike traditional risky-choice paradigms, the decision variables are learned through exploration and trial-by-trial feedback. In contrast to risky choice, the role of affective fluctuations in behavior during learning paradigms is less well-studied. But affect is a central component of predictions and evaluations required for learning through experience. Currently, affective states are not a part of many models of value-guided learning and choice behavior¹⁰⁰. In line with the main theme of this paper, we argue that affect, based on the temporal integration of positive and negative outcomes and actions, modulates subsequent value computations, and therefore influences learning from recent events.

In one of the first studies on how affective processes influence decision-making, Bechara and colleagues¹⁰¹ showed that as people learn advantageous and disadvantageous options, they also start to generate anticipatory physiological arousal responses, indexed by skin conductance, when they consider a risky option. These anticipatory arousal responses seem to guide subsequent behavior. Furthermore, a recent investigation based on the same task reported that greater skin conductance responses to feedback from disadvantageous relative to advantageous options was associated with greater loss aversion and a lower learning rate, both predicting better performance¹⁰². These findings suggest that the anticipatory arousal responses seem to guide behavior and feedback-related arousal responses are associated with individual differences in subsequent learning processes.

Moreover, recently, there have been several promising theoretical and computational frameworks attempting to integrate affective state changes in the reinforcement learning formalism to model and predict when and how affect influences learning from experienced outcomes and predicts subsequent behavior^11,17,18,103. Specifically, it has been suggested that momentary affect, reflecting recent prediction error history, may influence perception of subsequent rewards⁸⁰. Recently, it was suggested that valence feature of affect modeled as the integrated advantage of an agent’s recent actions in a given context could facilitate rapid learning of optimal actions¹⁸. The authors show that this formulation of valence in a reinforcement learning context could provide parsimonious explanations for effects such as the impacts of surprise and counterfactual information on affective state changes and subsequent decision processes. Similarly, another recent computational framework proposes that specific emotional states arise through the interactions of recent prediction errors, effectiveness of actions in obtaining rewards, and probability of future rewards¹⁰³. At the core of these accounts lie the formulation that changes in affective states modulate evaluation of new information (affect-as-information) and at the same time is a valuable signal to anticipate changes in the environment (affect-as-a-spotlight). Additionally, one may also model whether affective dynamics introduce variations in the trade-off between exploration and exploitation –a central theme in reinforcement learning formalism–, an effect that can be seen as affect influencing behavioral tendencies in relation to goals (affect-as-motivation). Thus, integrating affect and choice in the same computational framework makes it possible to simultaneously investigate several mechanisms through which changes in affective state could influence decision-making.

Judgment formation through information sampling and integration

A domain in which modeling affective integration is likely to contribute greatly is judgment formation. Judgments are formed through a continued accumulation of information over time^66,67, which is not an isolated process. Goals and beliefs guide attention, which may naturally result in a skewed sampling of information, favoring evidence that supports prior views. In fact, humans exhibit information avoidance, confirmatory sampling of evidence, and misinterpretation of information^104,105. Some theories underline the role of affective processes in biased judgment formation ^{e.g., 106,107}. Facing disconfirming evidence about an existing judgment may induce a negative affective state characterized by tension, discomfort, and arousal¹⁰⁸. The individual may be motivated to reduce negative affect through selective sampling, avoidance, and misinterpretation of information. It was also suggested that the decision maker accumulates evidence to make the chosen alternative as strong as possible with motivations to reduce pre-decision conflict and post-decision regret¹⁰⁶. Nevertheless, the empirical evidence for the direct impact of affective processes on skewed information sampling and biased judgments is limited. Considering affective state fluctuations during evidence accumulation may be necessary to understand the specific role of affect in biased and unbiased judgment formation.

There are several candidate mechanisms through which affect can temporarily influence information processing during evidence accumulation. First, affect can modulate attention in an affect-congruent manner, such that individuals are more likely to attend to stimuli that are congruent with their affective state^109,110. In connection to this, it was shown that evidence accumulation to reach an undesirable conclusion is faster when under perceived threat in comparison to a neutral state¹¹¹. Second, recent studies in perceptual decision-making show that phasic changes in arousal may reduce the weight of prior expectations during evidence accumulation^112,113. It has also been suggested that an increase in arousal can upregulate the efficiency of evidence accumulation¹¹⁴. Third, observed consequences of actions (including counterfactual information) may generate prediction error signals. As we have discussed before, prediction errors induce changes in affect^11,12, which in turn may influence sampling and processing of new information^96,97,99. Taken together, a systematic investigation that combines computational models of evidence accumulation and affect dynamics may be fruitful to test the specific involvement of affect in biased and unbiased judgment formation (for an approach to test specific hypotheses, see BOX 3).

In sum, the collective evidence reviewed in this section on judgment and decision-making from several choice contexts underlines the potential of investigating temporal dependency of affective states and decision processes to better understand the role of affect in judgment and decision-making. Fluctuations in affective state may influence weighting of outcomes, integration of payoffs and probabilities to guide expectations, and behavioral and choice biases. Critically, we suggest that several mechanisms through which affect may influence behavior should be mathematically formulated and simultaneously represented in a computational framework to be able to arbitrate between various functions of affect influencing behavior.

Box 3 Affective state fluctuations and confirmation bias

Humans exhibit information avoidance, confirmation bias, and misinterpretation of information^{125,126,127,128}. Even though some theories attempt to account for the role of affect^106,107,108, the current evidence for the direct impact of affect on these behaviors is limited. Arguably, considering how affective state fluctuates over time during evidence accumulation may be necessary to understand the role of affect in biased and unbiased judgment formation. We argue that carefully defined decision models, in which temporary affective modulations of decision parameters are introduced, will likely contribute to our current understanding.

Suppose a context wherein individuals perform a sequential information sampling task to make a decision that would result in better economic and/or environmental outcomes (e.g., building a power plant that generates energy from one of two materials: A or B). To aid decision, the individual may choose to collect as many scientific samples as they wished, wherein each sample is somewhat inaccurate due to measurement noise (e.g., sampled from a normal distribution). We can characterize this process with a simplified evidence accumulation model:

$$X=x_{0}+\sum\limits_{i}(x_{i}+v), \quad \left\{\begin{array}{cc} when\, X=a, & choose\, A\\ when\, X=-a, & choose\,B \end{array}\right\}$$

Here, x_i are individual samples that are integrated, X is the accumulated evidence, and a is the relative decision threshold. x₀ is the initial belief or bias, and v is the drift. If the individual has a prior belief that investing in A would generate better economic and/or environmental returns, they may set a lower evidence threshold to reach that conclusion (i.e., x₀ > 0) and/or have a positive drift (i.e., v > 0) which effectively increases the subjective weight of confirming evidence while decreasing the weight of disconfirming evidence. To this simplified model, one can introduce trial-by-trial modulation of decision parameters by affective fluctuations, assessed via continuous self-reported ratings and/or physiological responses (e.g., skin conductance). This would enable researchers to account for and test various mechanisms through which affect can temporarily bias evidence accumulation. Two candidates are: (1) affect-congruent modulation of attention, such that individuals are more likely to attend to stimuli that are congruent with their affective state^109,110; and (2) phasic changes in arousal upregulating the efficiency of evidence accumulation and reducing the weight of prior expectations on the final judgment^112,113. To confirm the former, it is required that an increase in positive affect causes a temporary increase in v, while higher arousal causing both v and x₀ to move closer to zero would support the latter hypothesis. Obviously, it is possible to formally model and test other psychological mechanisms through which affect may influence judgment formation. But the most critical point we would like to underline is that with this approach candidate mechanisms would be mathematically formulated and simultaneously represented in the models, which makes it possible to distinguish between different functions of affect influencing behavior.

Future perspectives

The external and internal signals that are important for judgments and decisions (i.e., rewards, punishments, prediction errors, counterfactuals, expectations, beliefs, confidence, and goals) induce changes in one’s affective state.. Here, we argue for the potential benefits of an approach for the investigation of how affective experience is constructed over time and how this changing mental and physiological state continuously influences behavior. Specifically, this approach, by identifying the key dynamic variables explaining how changes in affective states influence information processing and introduce behavioral biases may provide new insights into the continuous involvement of affect in behavior (e.g., see BOX 3; and for several outstanding questions in the field that can be answered with the current approach, see BOX 4). While there are a few recent promising attempts in this direction, future studies using innovative novel task designs, continuous measurement techniques, and computational modeling may overcome some of the current shortcomings of the traditional approaches to affect in judgment and decision-making research.

Box 4 Outstanding questions for future research

Affect dynamics and representation of internal beliefs about the state of the world: Affective state reflects varying expectations about the consequences of actions and prediction errors resulting from these expectations, which also alter the internal beliefs about the hidden states of the world. Is there a connection between affect dynamics and the changes in the expectation and precision (i.e., confidence) of internal beliefs? Could we formulate and test computational models to infer changes in the precision of the internal beliefs from fluctuations of affect?

Decision strategies: What could be the consequence of adopting the affective integration models of decision-making for the research on individual differences in decision styles and strategies (e.g., exploration-exploitation; intuition-deliberation; speed-accuracy trade-off)? Further research should elucidate connections between within-task changes in affective states and individuals’ propensity to adopt various choice strategies.

Psychological factors determining affective integration: What are the cognitive factors influencing the affective integration process beyond affective context, uncertainty, and goal-relevance? Important questions remain regarding, for example, the specific role involuntary attention and predictions in a certain context based on prior experience play.

Implications for the role of incidental and integral affect on decision-making: What are the consequences of modeling temporal dependency of affect and behavior for the current functional and categorical divide between the role of incidental and integral affect on decision-making? Affective experience at a given time is likely to be a result of a combination of incidental and integral influences. Also, considering the function of affect (i.e., signaling about current resources under current demands), there may not even be an incidental-integral distinction from the perspective of the brain. Can computational models help us parse the relative influence of incidental and integral cues for the overall affective experience and decision processes? One possibility is that when integral affect is strong the effect of incidental cues may be limited. There are some studies suggesting that cues that are irrelevant to the current task has less influence on affective experience in comparison to task-relevant information ^{e.g., 78,122}.