Fig. 3: Experimental confirmation of insights from saliency maps and CycleGANs via radiograph modification.
From: AI for radiographic COVID-19 detection selects shortcuts over signal
a, Left: text markers on radiographs are highlighted by saliency maps as important for COVID-19 prediction. The exchange of laterality markers between a pair of COVID-19-positive and COVID-19-negative images significantly shifts the output when compared to swapping random patches of the same size: Δ positive image (log odds) = −5.63 (empirical P = 9.99 × 10−4 based on Monte Carlo substitution of random image patches, n = 1,000); Δ negative image (log odds) = 13.85 (P = 5.00 × 10−3, n = 1,000) (‘Experimental validation of feature attributions’ and ‘Statistics’ sections). Grey dots in the distribution plots (right) correspond to the change in model output after swapping random image patches, which were used as a negative control. Red dots correspond to the change in model output for the radiographs with swapped laterality markers. b, The positioning of patient shoulders may impact COVID-19 prediction. Saliency maps highlight the shoulder region as important predictors of COVID-19 positivity after (but not before) this region is moved to the top of the image (left). This patch increased model output significantly more than random patches of the same size moved to the same corners (Δ model output = 6.57, empirical P = 5.00 × 10−3, n = 1,000). Grey dots in the distribution plot (right) correspond to radiographs with randomly selected patches. The red dot corresponds to the radiograph with the shoulder regions moved.
