Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Advertisement

Nature Precedings
  • View all journals
  • Search
  • My Account Login
  • Content Explore content
  • About the journal
  • RSS feed
  1. nature
  2. nature precedings
  3. articles
  4. article
Generating descriptive text from functional brain images
Download PDF
Download PDF
  • Manuscript
  • Open access
  • Published: 11 February 2011

Generating descriptive text from functional brain images

  • Francisco Pereira1,
  • Greg Detre1 &
  • Matthew Botvinick1 

Nature Precedings (2011)Cite this article

  • 797 Accesses

  • 2 Citations

  • 1 Altmetric

  • Metrics details

Abstract

Recent work has shown that it is possible to take brain images of a subject acquired while they saw a scene and reconstruct an approximation of that scene from the images. Here we show that it is also possible to generate text from brain images. We began with images collected as participants read names of objects (e.g., ``Apartment'). Without accessing information about the object viewed for an individual image, we were able to generate from it a collection of semantically pertinent words (e.g., "door," "window"). Across images, the sets of words generated overlapped consistently with those contained in articles about the relevant concepts from the online encyclopedia Wikipedia. The technique described, if developed further, could offer an important new tool in building human computer interfaces for use in clinical settings.

Similar content being viewed by others

Improved image reconstruction from brain activity through automatic image captioning

Article Open access 10 February 2025

Human brain responses are modulated when exposed to optimized natural images or synthetically generated images

Article Open access 23 October 2023

Reverse-engineering the cortical architecture for controlled semantic cognition

Article 18 January 2021

Article PDF

Author information

Authors and Affiliations

  1. Princeton University, Psychology https://www.nature.com/nature

    Francisco Pereira, Greg Detre & Matthew Botvinick

Authors
  1. Francisco Pereira
    View author publications

    Search author on:PubMed Google Scholar

  2. Greg Detre
    View author publications

    Search author on:PubMed Google Scholar

  3. Matthew Botvinick
    View author publications

    Search author on:PubMed Google Scholar

Rights and permissions

Creative Commons Attribution 3.0 License.

Reprints and permissions

About this article

Cite this article

Pereira, F., Detre, G. & Botvinick, M. Generating descriptive text from functional brain images. Nat Prec (2011). https://doi.org/10.1038/npre.2011.5666.1

Download citation

  • Received: 11 February 2011

  • Accepted: 11 February 2011

  • Published: 11 February 2011

  • DOI: https://doi.org/10.1038/npre.2011.5666.1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Keywords

  • fMRI
  • topic models
  • basis images
  • semantic categories
  • wikipedia
Download PDF

Advertisement

Explore content

  • Research articles
  • News & Comment
  • Sign up for alerts
  • RSS feed

About the journal

  • Journal Information

Search

Advanced search

Quick links

  • Explore articles by subject
  • Find a job
  • Guide to authors
  • Editorial policies

Nature Precedings (Nat Preced)

nature.com sitemap

About Nature Portfolio

  • About us
  • Press releases
  • Press office
  • Contact us

Discover content

  • Journals A-Z
  • Articles by subject
  • protocols.io
  • Nature Index

Publishing policies

  • Nature portfolio policies
  • Open access

Author & Researcher services

  • Reprints & permissions
  • Research data
  • Language editing
  • Scientific editing
  • Nature Masterclasses
  • Research Solutions

Libraries & institutions

  • Librarian service & tools
  • Librarian portal
  • Open research
  • Recommend to library

Advertising & partnerships

  • Advertising
  • Partnerships & Services
  • Media kits
  • Branded content

Professional development

  • Nature Awards
  • Nature Careers
  • Nature Conferences

Regional websites

  • Nature Africa
  • Nature China
  • Nature India
  • Nature Japan
  • Nature Middle East
  • Privacy Policy
  • Use of cookies
  • Legal notice
  • Accessibility statement
  • Terms & Conditions
  • Your US state privacy rights
Springer Nature

© 2025 Springer Nature Limited

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing