From Novelty Bots to Everyday “Pocket Therapists”
AI mental health chatbots are software systems that simulate conversational support for wellbeing, often marketed as companions or assistants, and they increasingly blur the line between casual self-help, emotional support, and services that look and feel like real therapy sessions. AI-enabled mental health tools are moving quickly from novelty to mainstream use, and regulators are starting to draw sharper lines around what those tools can and cannot claim to do. New data released this week shows 54 per cent of children and teens are using artificial intelligence tools for personal and social reasons. Survey results showed 78 per cent of children and teens had used an AI assistant, such as ChatGPT, Gemini or Claude. AI assistants are primarily used as a practical tool to help with tasks such as writing, editing and sourcing information. But those same tools are becoming quiet fixtures of emotional life, turning into what many users treat as on-demand “pocket therapists” without ever booking an appointment.
Teens, Intimacy and the Drift From Homework to Help
The most uncomfortable truth underpinning mental health chatbot regulation is who is using these tools and how fast that use is changing. AI is a rapidly expanding part of young people’s education, information, entertainment and creative landscapes. Of the 54 per cent who reported using AI for social and personal reasons, 33 per cent asked AI for advice about physical health, 33 per cent asked for advice about what to do in a particular situation, and 20 per cent asked AI for advice about their mental health and wellbeing. Around 10 per cent of teens, a meaningful minority, also used AI companions that simulate sustained role play, including romantic ones. This “homework first, feelings later” pattern shows why regulators cannot treat AI therapy tools as niche gadgets. What begins as a spelling fix easily becomes a confidant for family conflict, self-worth, or sexuality, especially for disabled and LGBTQIA+ young people who already show higher engagement with AI companions.
Lawsuits and State Laws: Drawing the First Guardrails
Regulators are no longer content with gentle warnings; they are moving into enforcement. Recent lawsuits against Character Technologies Inc., the company behind Character.ai, allege that the platform hosted bots that mimicked licensed therapists, including one persona that allegedly claimed fictional professional credentials and engaged in tens of thousands of patient interactions. This is not a cosmetic branding problem—it is an allegation that a system crossed into the territory of unlicensed clinical practice. In the absence of a comprehensive federal framework, states are filling the gap. Colorado, Maine, Rhode Island, Tennessee, and Vermont have advanced AI therapy restrictions, joining Illinois, Nevada, and Utah. These laws vary in scope, but the emerging themes are clear: restrictions on advertising AI as a licensed mental health professional, limits on direct patient engagement by AI tools in clinical settings, and greater scrutiny of chatbots used by, or marketed to, children.
Supportive Tool or Illegal Therapist?
The hardest regulatory question is not whether AI mental health chatbots should exist, but what they are allowed to pretend to be. Regulators and clinicians are distinguishing between general-purpose chatbots that users may treat as “pocket therapists” and more purpose-built digital therapeutics designed with clinical guardrails and therapeutic datasets. Disclaimers alone may not be enough if a bot’s persona, outputs, or user experience suggests professional diagnosis, treatment, or therapy. Meanwhile, AI tools do not understand if their responses are factual or not, only if they sound plausible, which can lead to bullying language or harmful advice being mirrored back to vulnerable users. The litigation comes amid growing AI chatbot use for mental health advice, particularly among adolescents and young adults, and follows reports of serious safety concerns involving minors and crisis-related conversations. In practice, that means the line between “motivational companion” and “unlicensed therapist” stays contested until law and design both change.
What Comes Next: Compliance, Safety and the End of Magical Thinking
The next phase of mental health chatbot regulation will be less about headlines and more about plumbing: disclosures, escalation paths, and data discipline. The Children’s Only Privacy Code is due to be enacted this December and will likely provide certain privacy expectations that extend to AI tools. AI can also expose children and teens to harmful content; 20 per cent reported potentially harmful or inappropriate interactions with AI tools, including mean or hateful responses and attempts to control users. For companies developing or deploying AI tools in health, wellness, youth engagement, or consumer support, this is a moment to reassess product design, marketing claims, age-gating, crisis escalation, disclaimers, professional oversight, and state-by-state compliance obligations. The Federal Food and Drug Administration has not yet authorized generative AI tools for mental health treatment, and professional groups are urging Congress and federal agencies to set clearer standards. Until that happens, companies should expect continued litigation risk, a growing patchwork of state laws, and closer attention from regulators where AI systems appear to provide mental health advice without appropriate clinical controls.
