Extended Data Fig. 7: Figure S7. GloVe’s space embedding attributes. | Nature Neuroscience

Extended Data Fig. 7: Figure S7. GloVe’s space embedding attributes.

From: Shared computational principles for language processing in humans and deep language models

Extended Data Fig. 7: Figure S7. GloVe’s space embedding attributes.

It can be argued that GloVe based encoding outperforms arbitrary-based encoding due to a general property of the space that GloVe embeddings induce (for example, they are closer / further away from each other). To control for this possible confound, we consistently mismatched the labels of the embeddings of GloVe and used the mismatched version for encoding. This means that each unique word was consistently matched with a specific vector that is actually an embedding of a different label (for example, matching each instance of the word ‘David’ with the embedding of the word ‘court’). This manipulation uses the same embedding space that GloVe uses and also induces a consistent mapping of words to embeddings (as in the arbitrary-based encoding). The matched GloVe (blue) outperformed the mismatched GloVe (black), supporting the claim that GloVe embedding carries information about word statistics that is useful for predicting the brain signal.. The error bars indicate the standard error of the encoding models across electrodes.

Back to article page