Extended Data Fig. 1: Behavior and model fit for an example participant.

Each plot represents a shell. Points at the bottom and top of each plot indicate the participant’s trial-by-trial responses: black points at the bottom represent NoGo responses; colored points at the top represent Go responses, with the color representing the reinforcement received (blue, reward; yellow, neutral; red, punishment). a, Go probability, P(Go), predicted by the winning model (S–S–R–8; solid black line) and its corresponding S–R model (S–R–8; dashed gray line). Horizontal color-coded bars indicate the shell’s season (blue, rewarding; yellow, neutral; red, punishing). Model S–S–R–8 tracked the participant’s behavior better, especially when seasons recurred (last two seasons for each shell). b, Belief trajectory for model S–S–R–8. Horizontal color-coded bars on top indicate the season inferred by the model; horizontal color-coded bars immediately below indicate the true season. The lines depict the posterior probability distribution of the beliefs about the shell season (state), P(State), on each trial: the beliefs that the shell is in rewarding, neutral and punishing states are represented in blue, yellow and red, respectively. In the model, each state corresponds to an outcome distribution; states are not labeled a priori as rewarding, neutral or punishing. To represent the inferred seasons in the top horizontal bars and the beliefs in the plotted lines, we assigned rewarding, neutral and punishing labels to states whose final distribution consisted mostly of reward, neutral or punishment outcomes, respectively. Given that the task included three season types, the model assumed that there could be up to three hidden states (see ‘S–S–R models’ in ‘Computational models’ in the Supplementary Methods). In all plots except the top-left plot, the model only inferred up to two states; a state that was never inferred does not have outcomes associated, so we represent it in black. Some lines overlap and therefore are not visible. The line markers indicate the state that the model believes applies after each trial; usually, this corresponds to the state with the largest posterior probability, but sometimes it does not because the model has a (parameter-dependent) tendency to stick with the prior belief (see ‘S–S–R models’ in ‘Computational models’ in the Supplementary Methods).