Skip to main content
Artificial Intelligence

A clinically validated framework for auditing AI chatbot behavior in mental health interactions.

| Source: Nature medicine

Millions of users turn to consumer artificial intelligence chatbots to discuss emotional, behavioral and mental-health concerns, creating an urgent need for rigorous and scalable safety evaluations. Here we introduce simulated (SIM) vulnerability-amplifying interaction loops (VAILs) (SIM-VAIL), a clinically validated framework for auditing chatbot behavior in mental-health contexts. SIM-VAIL simulates users with specific psychiatric vulnerabilities and conversational intents, engages them in mul

Millions of users turn to consumer artificial intelligence chatbots to discuss emotional, behavioral and mental-health concerns, creating an urgent need for rigorous and scalable safety evaluations. Here we introduce simulated (SIM) vulnerability-amplifying interaction loops (VAILs) (SIM-VAIL), a clinically validated framework for auditing chatbot behavior in mental-health contexts. SIM-VAIL simulates users with specific psychiatric vulnerabilities and conversational intents, engages them in multi-turn conversations with frontier artificial intelligence chatbots (including Claude, ChatGPT, Gemini, Grok and Llama models) and scores each exchange across 13 clinically grounded risk dimensions. Across 810 conversations, spanning 9 target chatbots and 30 simulated user profiles, concerning behavior in target chatbots was widespread, albeit reduced in newer models. Concerning behavior varied by user vulnerability and conversational intent, accumulated over turns, and could be reduced by interventions at early escalation points. Risk was highest when otherwise supportive chatbot behaviors reinforced the psychological mechanisms underlying the simulated user's vulnerability, a pattern we term a VAIL. SIM-VAIL provides a scalable framework for mapping mental-health risk across users, chatbots and conversational trajectories, offering a foundation for targeted safety improvements.

Read the original source →

Related Stories

Artificial Intelligence

Integrating cooperative breeding with reciprocity and culture extends Hamilton's rule to humans.

We study how kinship and reciprocity shape giving and child care support in societies with limited access to markets. Extending Hamilton's rule to incorporate reciprocity, we test it using behavioral data from dictator games and surveys in remote matrilocal and patrilocal villages of northeast India (N = 406 parents of young children). We replicate with data from urban Manila neighborhoods (N = 301) and the Solomon Islands (N = 624). Consistent with predictions, transfers to recipients increase

Continue reading
Artificial Intelligence

Multi-omics atlas of chronic exercise.

Exercise reduces all-cause mortality, yet its molecular effects across organs, sexes, and circadian phases remain incompletely understood. We subjected mice to 8 weeks of voluntary wheel running and collected 22 tissues and plasma during active and inactive phases for bulk RNA sequencing, metabolomics, proteomics, and single-nucleus RNA and methylation sequencing. Exercise induced canonical fiber-type switching and widespread, tissue-specific remodeling, with the largest transcriptomic changes i

Continue reading
Artificial Intelligence

A microexon regulatory code reveals autism-linked genetic variation.

Neuronal microexons are a highly conserved class of alternative splicing events enriched in genes with critical functions in nervous system biology 1,2 . Microexon splicing is often disrupted in the brains of autistic individuals, and mice with microexon deficiencies display autism-like phenotypes 1,3,4 . However, the full landscape of microexons is not known, and the impact of genetic variation on these exons has not been systematically investigated. Here, through extensive transcriptome profil

Continue reading
Artificial Intelligence

Deep learning of fossil pollen morphology reveals 25,000 y of ecological change in eastern African grasslands.

Grass (Poaceae) pollen is largely overlooked in investigations of grassland evolution because the pollen of most species cannot be differentiated using traditional optical microscopy. However, the combination of superresolution microscopy and deep learning enables the capture and quantification of distinct morphological variation across the pollen of grass species. Using a semisupervised deep-learning strategy, we trained convolutional neural networks (CNNs) on superresolution images of known mo

Continue reading
Artificial Intelligence

Tyrosine phosphorylation unfolds nucleophosmin and disrupts its integration into the nucleolus.

Nucleophosmin (NPM1) is a multifunctional nucleolar protein essential for ribosome biogenesis, genome stability, and stress responses. Its integration into the nucleolus depends on its oligomerization and multivalent interactions that enable liquid-liquid phase separation (LLPS). Here, we investigate how tyrosine phosphorylation at Tyr17, Tyr29, and Tyr67 located within the interface between monomers at the N-terminal oligomerization domain regulates NPM1 structure and function. Replacing tyrosi

Continue reading