Fig. 6: Reconstruction of experiential mental image scenario structure was commensurate with GPT-2. | Nature Communications

Fig. 6: Reconstruction of experiential mental image scenario structure was commensurate with GPT-2.

From: Neural decoding of autobiographical mental image features with a general semantic model

Fig. 6: Reconstruction of experiential mental image scenario structure was commensurate with GPT-2.

A To explore the possibility that the twenty experiential features were too few to comprehensively model the fMRI data, a comparative analysis was run using 1024 feature GPT-2 layer-16 embeddings derived from participants’ autobiographical mental image descriptions and the 240 sentence stimuli. GPT-2 features were reconstructed from the autobiographical imagery data using an fMRI-to-GPT-2 mapping fit on the sentence comprehension data (equivalent to Fig. 1C). Because there was no clear way to interpret comparisons in individual feature reconstruction accuracies between GPT-2 vs the experiential model, representational similarity analysis (RSA) was used to compare the relative ability to reconstruct scenario similarity structure. B RSA correlation coefficients are illustrated for individual participants as dots in the bar plot (dark gray: young, light gray: elderly). Bars are mean accuracy, and error bars represent SEM. Both GPT-2 and the experiential model yielded RSA correlation coefficients that were significantly greater than zero in the Core default mode network (Core-DMN), fronto-temporal DMN (FT-DMN) and medial temporal DMN (MT-DMN), according to one-sample signed ranks tests against zero (fifty participants, one-tailed, corrected according to false discovery rate (FDR)35). Significance is illustrated above bars using star notation: ***FDR(p) < 0.001. Critically, there were no significant differences in RSA coefficient magnitude between GPT-2 and the experiential model in any DMN subsystem, according to two-sample signed ranks tests (fifty participants, two-tailed). Source data are provided as a Source Data file.

Back to article page