Figure 5

The fidelity between the participant predictions and the AI classifications is higher when the AI is correct than when the AI is wrong. (A,B) are based on the entire data set, comparing all [map] conditions to all [no map] conditions (631 participants; 94,582 observations). (C,D) exclude the [no examples] trials and contrast all [helpful] trials with all [random] trials (419 participants; 62,820 observations). (A) The saliency maps improve fidelity for trials when the AI classifier is wrong but reduce fidelity when the AI classifier is correct. (B) The saliency maps make people less likely to predict that the AI classification of the target image matches the ground truth. Together, (A,B) imply that the saliency maps help people to consider that the AI classifier might make mistakes. (C) In trials with examples, helpful examples tend to help people accurately model the AI classifier in cases when the AI classifier is correct, but have a limited impact when the AI classifier is wrong. (D) Consequently, helpful examples make participants more likely to pick the ground truth option when the AI classifier is correct, but do not really impact the probability of selecting the ground truth option when the AI classifier is wrong. Collectively, these results suggest that helpful examples and saliency maps improve human understanding of the AI classifier in distinct and complementary ways: saliency maps improve error detection, whereas helpful examples enable participants to accurately determine when the AI classifier is correct. Errorbars represent 95% bootstrapped confidence intervals. All point estimates have confidence intervals, though some are too narrow to see clearly. This figure was created using the ggplot2 package (v. 3.3.2)⁴⁶ in R (v. 4.0.3)⁴⁷.