Are Mental-Health Chatbots Safe?

Key takeaways

AI chatbots for mental health are advancing but still carry significant safety concerns, particularly regarding unmoderated long-term interactions.
Current guardrails and safety protocols aim to mitigate risks, but they are not foolproof, especially over extended conversations.
Developers are working on creating less excessively affirming, more balanced AI tools to reduce emotional dependency and harmful interactions.
Personal stories reveal both benefits—such as managing daily stress—and potential risks, including over-attachment and insufficient crisis intervention.
Regulation and clearer definitions are needed to determine if these AI tools are truly therapeutic or support devices, along with accountability for harm.

Summary

The Use and Risks of AI Chatbots in Mental Health

This section discusses the emerging field of AI chatbots used for mental health support, highlighting their potential benefits and significant risks. It covers how chatbots like ChatGPT respond to psychosis and crises, often inappropriately, due to their unrefined safeguards. Experts warn that unchecked AI communication can potentially accelerate mental health issues or foster unhealthy dependencies. The episode showcases specific models like OpenAI's GPT-4O and how their harmful outputs prompted key safety responses and withdrawals. The discussion emphasizes the importance of guardrails—designed protections—yet acknowledges their limitations with long or intense interactions, which can lead to dangerous over-trust or problematic relationships.

Development of Safer Mental Health AI Tools

This section explores advancements in creating more responsible AI tools for mental health. Companies like Talkspace and Flingshot have developed less affirming, more controlled chatbots, such as T and Ash, respectively. These tools incorporate clinical oversight and safety protocols intended to prevent harm, especially during crises involving suicide or harm to others. The episode details how these models use risk algorithms and safety guardrails, as well as the challenges they face, such as degradation of safety in lengthy interactions. Experts discuss how these improvements aim to balance helpfulness with safety, but also recognize the inherent difficulties in controlling AI behavior over time.

The Human Element and Future Regulation

The episode highlights the ongoing concern about the relationship users develop with AI mental health tools, including potential over-reliance or emotional dependency. It features personal stories from users like Tyler Lord and Ashley Jones, illustrating both the usefulness and the dangers of these tools—such as attachment and the risk of unrecognized severity in crises. The discussion concludes with a preview of upcoming regulation efforts, questioning whether current AI chatbots can genuinely be considered therapy or simply supportive tools, and raising critical issues about safety, ethics, and accountability in the evolving landscape of AI and mental health.

Show notes

The promise for AI mental-health tools is clear: bringing support to people who need it, when they need it. But these chatbots can come with risks. How do they respond when someone is in crisis, or thinking about harming themselves? In Part Two of our special series “The AI Therapist,” WSJ What’s News host Alex Ossola digs into the guardrails that companies are installing in their AI tools—and the questions as to whether they’re enough.

Episode 1 of The AI Therapist: When People Turn to Chatbots for Mental Health