Fig. 1: Language generation with brain recordings (BrainLLM).
From: Generative language reconstruction from brain recordings
a The generation process has four main stages. S1: Brain recordings in response to the perceived continuation are collected. S2: A brain adapter extracts features from brain recordings and transforms them into hidden vectors that match the shape of text embeddings in a standard LLM. S3: Brain embeddings and text prompt embeddings are concatenated as a prompt input. S4: The prompt input is fed into the LLM for language generation. BrainLLM generates content that is an exact match ("the cutting edge of'') with, or semantically similar/gist match content ("not for everyone'') to the perceived continuation. b Examples of language generation with BrainLLM and its controls (PerBrainLLM). Text in blue and bold indicates that the generated content and the ground truth (perceived continuation) are manually annotated as semantically similar and an exact match, respectively.
