Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • WORLD VIEW

Why we need mandatory safeguards for emotionally responsive AI

Virtual chatbots that simulate conversations with famous actors or sci-fi characters can have real-world consequences.
By
  1. Ziv Ben-Zion

    1. Ziv Ben-Zion is a clinical neuroscientist at the Yale University School of Medicine in New Haven, Connecticut.

Breakthroughs in large language models (LLMs) have made conversations with chatbots feel more natural and human-like. Applications such as Replika and Character.ai, which allow users to chat with artificial intelligence (AI) versions of famous actors and science-fiction characters, are becoming popular, especially among young users. As a neuroscientist studying stress, vulnerability and resilience, I’ve seen how easily people react to even the smallest emotional cues — even when they know that those cues are artificial. For example, study participants exhibit measurable physiological responses when shown videos of computer-generated avatars expressing sadness or fear.

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Nature 643, 9 (2025)

doi: https://doi.org/10.1038/d41586-025-02031-w

Editor’s note: When contacted by Nature, a Character.ai spokesperson pointed out that disclaimers and technical protections are being built into its models to reduce the potential for harm; Replika did not respond.

Competing Interests

The author declares no competing interests.

Subjects

Latest on:

Nature Careers

Jobs

  • Recruitment Announcement for Team Leaders and High-Level Talents at DIMST, CAS

    I) Leaders for twelve research divisions II) Academic Leading Talents III) Outstanding Young Talents

    No. 333 Pingdong Road, Songshan Lake Science City, Dongguan, Guangdong

    Dongguan Institute of Materials Science and Technology, Chinese Academy of Sciences (DIMST, CAS)

  • Seeking Global Talents at All Levels - SIAT

    We find Distinguished PIs, Senior PIs, Junior PIs, Senior Engineer, Junior Engineer, Post-doctoral fellow, Assistant research fellow.

    1068 Xueyuan Avenue, Shenzhen 518055 P. R. China

    Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences（CAS)

  • Assistant or Associate Professor

    The Dept. of Computational and Systems Biology at Pitt is hiring two positions, a full-time Assistant Professor and a full-time Associate Professor.

    Pittsburgh, Pennsylvania

    University of Pittsburgh Department of Computational and Systems Biology

  • Postdoctoral Fellowship (2 years): Determinants and Consequences of the Oral Microbiome

    Join us for the project “Determinants and consequences of the oral microbiome across the life course: Implications for health and disease"

    Umeå, Sweden

    Umeå University

  • FAFU Global Faculty Recruitment

    FAFU welcomes applicants of all nationalities to apply for faculty positions at any rank (full professor, associate professor, assistant professor).

    Fuzhou, Fujian (CN)

    Fujian Agriculture and Forestry University

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing

Search

Advanced search

Quick links