Fig. 7: POMDP model explains seemingly higher influence of choice-congruent evidence on confidence ratings.

Discrepancy in the observations used by a decision maker and those used by an experimenter studying the decision maker’s behavior could lead to biased interpretation of experimental results. a We simulated a POMDP model that uses a fraction of observations available in a trial unbeknownst to the experimenter. The observations supporting opposing choices equally inform the model’s behavior. However, an experimenter who uses a classifier to predict choices and confidence based on all observations in the trial finds an apparently larger influence of choice-congruent observations on confidence. b Forcing the classifier to have balanced weights for all observations causes lower prediction accuracy of confidence ratings, especially when the proportion of used evidence is low. In such cases, even a model that totally ignores choice-incongruent observations performs better than the balanced model. However, the better performance of models with imbalanced weights does not reflect the decision making process. It stems merely from the experimenter’s lack of knowledge about the observations used by the simulated model. c, d Same as (a, b) but observations accessible to the experimenter are noisy estimates of observations available to the decision maker. Such noise reduces the prediction accuracy of the experimenter’s classifier, but more importantly, it also causes imbalanced weights in the optimal classifier (c) and lower performance of the balanced classifier (d). That is true even when both the decision maker and experimenter use all the available observations (see inset box in (c)). The noise in these simulations comes from a zero-mean Gaussian distribution with a variance 25% larger than \({w}_{z}^{2}\).