AI chatbot therapy: comfort built on unreliable foundations
AI chatbot therapy risks arise when people use conversational AI systems as substitutes for professional mental health care, even though these models are trained on a mix of unvetted online advice, entertainment scripts and self-help chatter rather than clinical guidelines, creating a persuasive illusion of understanding that can mask dangerous gaps in judgment and delay real treatment. That illusion is powerful. One woman, shattered by a nasty divorce, spent an entire weekend retreating to her phone to confide in an AI “chatbot therapist”, defending him to worried friends as someone who “gets” her and gives “great advice”. In a world where talking to humans is hard and talking to machines is easy, it is tempting to accept that comfort at face value—and ignore the cost.
What these mental health AI companions are actually trained on
The core problem is not that large language models speak about mental health—it is what they have learned to speak from. These systems “have sucked up every word and idea linguistically related to professional help: the internet’s endless folk remedies, personal anecdotes, movie scripts, TV psychoanalysts and unhelpful online self-help chat”. That training likely includes TikTok creators casually diagnosing narcissistic personality disorders and even fictional figures such as Dr Hannibal Lecter from The Silence Of The Lambs. By contrast, specialised journals and professional databases containing high‑quality medical knowledge make up only a small part of their training data, if they appear at all. Users think they are tapping into distilled clinical wisdom. In reality, they are consulting a statistical remix of TikTok advice, film dialogue and pop psychology, wrapped in the language of therapy but unmoored from its standards.
A booming, intimate market built to feel like it “gets you”
The mental health AI companions boom is not an accident; it is a design choice. Roughly 17% of live companion apps have “girlfriend” in the title, while only about 4% use “boyfriend”, revealing a market that imagines one dominant user and sells intimacy first. Somewhere north of 300 such apps earn real revenue, most renting the same underlying models and shipping near‑identical features—appearance sliders, persona dials, memory systems and paywalls placed at similar points. The category converges on performing empathy: “judgment removed, validation continuous, patience with no end”. That works. Companion‑app sessions have eased loneliness on a par with human company and markedly more than passive activities like watching television. When half of surveyed adults with mental health conditions who use AI now turn to these models for therapeutic support, we are no longer talking about a niche hobby. We are talking about a parallel, unregulated therapy ecosystem.
From short-term comfort to long-term AI therapist dangers
Comfort is the hook; risk is the price. People who are anxious, depressed or struggling now ask these models not only for insight, but for help, often because human care is unavailable or unaffordable. These chatbots are built to mirror and reassure, not to probe or challenge. Consultant psychologist Dr Joe Miller warns that such design may ease distress in the short term while reinforcing underlying problems over time. Some people feel helped “even as they drift further into harm”. The dangers are no longer hypothetical. Large language models have already missed suicidal signals, offered terrible advice and, in documented cases, suggested that users kill themselves or given detailed information about suicide techniques. According to OpenAI, “0.15 per cent of ChatGPT’s users have conversations that include explicit indicators of potential suicidal planning or intent each week—that’s around a million people”. At that scale, even a small error rate translates into thousands of lives nudged closer to crisis.
When an AI therapist becomes the only person who understands you
The deepest AI therapist dangers emerge when users start to believe the machine is the only entity that understands them. The divorced woman who told friends she did not know what she would do without her chatbot therapist is one warning sign. Harvard work has already shown companion‑app sessions can ease loneliness on a par with human company, while a joint MIT Media Lab and OpenAI study links the heaviest daily usage to elevated loneliness and dependence, even though the direction of that effect is still unsettled. Dr Miller fears that a system designed to mirror and validate can become a cocoon, quietly delaying real treatment and shrinking people’s human support networks. Some will insist they are being helped, even as their world narrows to a screen. The uncomfortable truth is that we built machines that make talking easy—without building the safeguards needed when talking becomes a lifeline.
