Fig. 1
From: Uncertainty and reward histories have distinct effects on decisions after wins and losses
Sex dependent asymmetrical learning behaviors and computations. (a) Reward choice under changing reward probabilities within the DynaPRL task in which rats initiated trials via magazine entry, following which two levers (left and right) are inserted, and rats made their decision. Outcomes were cued with a clicker noise if rewarded or white noise if unrewarded. The reward was ~ 33μL of a 10% sucrose solution. (b) There were three distinct block types based on their reward probability contrasts. The high contrast block had a p(reward|A1) = 0.8 and p(reward|A2) = 0. 1. The low contrast block had a p(reward|A1) = 0.6 and p(reward|A2) = 0.3. The block with no contrast had a 0.45 probability of reward for both actions. (c) the reversal curve plotted based on p(perseveration) (i.e., likelihood of choosing the previous lever) score for each trial, across all block types. Males and females both reverse by the 6th trial, with females showing a lower probability of repeating the previous best choice (p(perseveration) score) from 5th trial onwards compared to males (green dots are post-hoc tests showing a significant difference between males and females). (d) Overall win-stay and lose-shift probabilities, across all block types. Both sex groups win-stay more and lose-shift less in our task, however males had a higher p(WS) and a lower p(LS) compared to females. Each dot represents an individual rat, and error bars are standard error of the mean. (e) Asymmetrical value updating model where chosen and unchosen wins and losses had distinct learning and value decay rates as free parameters (f–i) Posterior densities from the hyperparameter of sex-group differences for the four value updating parameters from the reinforcement learning model in e. Rightward shifts (above 0) indicate higher parameter values for males compared to females, and the opposite for leftward (below 0) values. Horizontal lines below represent 80–95% highest density interval. Males overall had a higher learning rate from positive (alpha +) and negative (alpha-) outcomes, compared to females. Males also had a higher value decay rate for unchosen action when the chosen action’s outcome was negative (1-gamma-), but similar value decay rates for the unchosen action when the chosen action’s outcome was positive (1-gamma +). Abbreviations: dBF = directed Bayes Factor; p(R|A1) = probability of reward given action 1 was taken; p(R|A2) = probability of reward given action 2 was taken. * p < 0.05.
