Extended Data Fig. 7: RPE encoding of TEs requires mixed state-space representations of sensorimotor variables.

a, Initial running direction bias introduced into the tdRPE model to mimic mouse behaviour (Fig. 3j). Bars represent average trial numbers for bias and unbiased directions across 10 model simulations. b, Average reward rates (left) and times to reward (right) for cues associated with the pre-cue bias and non-bias initial running directions (compare to mouse behaviour data in Fig. 3k–l). c, Average simulated RPEs aligned to cue onset (t = 0) for trials presented with the cue associated with the bias (left) or non-bias (right) directions, for congruent (green) and incongruent (purple) trials split into thirds by simulated initial angular velocity magnitude. d, Average trial numbers for each cue for a model with no initial direction bias, presented as in a. e, Same as c for an un-biased model, split by cues associated with left and right directions. f, Average simulated RPEs aligned to cue onset on trials presented with the cue associated with the right (red) or left (blue) direction in an un-biased model. g-i, Results from a tdRL model (with the directional bias in a) in which the state only includes the cue identity variable with no mixing. g, Same as c. h, Same as f. i, Same as f with trials split by whether previous trial was rewarded (red) or unrewarded (blue). RPEs are larger for the cue associated with the initial running bias and following previously rewarded trials, consistent with the mouse DA data and cue value RPE encoding (Fig. 3d–i). j-l, Results from a model (with the directional bias in a) in which the state includes all variables (Fig. 3b) but with no mixing. RPEs are plotted as in g-i. Shaded regions and error bars in all plots are 95% confidence intervals.