Fig. 5: Uncertainty and surprise levels computed by GPT-2 correlate with pre-onset and post-onset neural activity respectively. | Nature Neuroscience

Fig. 5: Uncertainty and surprise levels computed by GPT-2 correlate with pre-onset and post-onset neural activity respectively.

From: Shared computational principles for language processing in humans and deep language models

Fig. 5: Uncertainty and surprise levels computed by GPT-2 correlate with pre-onset and post-onset neural activity respectively.

a, Trigger-averaged activity (and s.e.) across 160 electrodes with significant GloVe encoding for words (Fig. 3c). Averaging was performed separately for words correctly predicted and incorrectly predicted. Note the increase in signal activity for predictable words before onset and for unpredictable words 400 ms after word onset. b, Partial correlations between uncertainty (entropy) and signal power controlling for cross-entropy (green line). Partial correlations between surprise (cross-entropy) and neural signals controlling for correlation with entropy (red and purple lines). Asterisks indicate correlation significance (FDR corrected, q < 0.01).

Back to article page