Can a Chatbot Be Your Therapist? A Study Found 'Amazing Potential' With the Right Guardrails

Yahoo

03-04-2025

Your future therapist might be a chatbot, and you might see positive results, but don't start telling ChatGPT your feelings just yet.
A new study by researchers at Dartmouth found a generative AI tool designed to act as a therapist led to substantial improvements for patients with depression, anxiety and eating disorders -- but the tool still needs to be closely watched by human experts.
The study was published in March in the journal NEJM AI. Researchers conducted a trial with 106 people who used Therabot, a smartphone app developed at Dartmouth over the past several years.
It's a small sample, but the researchers said it's the first clinical trial of an AI therapy chatbot. The results show significant advantages, mainly because the bot is available 24 hours a day, which bridges the immediacy gap patients face with traditional therapy. However researchers warn that generative AI-assisted therapy can be perilous if not done right.
"I think there's a lot yet for this space to evolve," said Nick Jacobson, the study's senior author and an associate professor of biomedical data science and psychiatry at Dartmouth. "It's really amazing the potential for personalized, scalable impact."
Read more: Apple's AI Doctor May See You in 2026
The 210 participants were sorted into two groups -- one group of 106 was allowed to use the chatbot, while the control group was left on a "waiting list." The participants were evaluated for their anxiety, depression or eating disorder symptoms using standardized assessments before and after the test period. For the first four weeks, the app prompted its users to engage with it daily. For the second four weeks, the prompts stopped, but people could still engage on their own.
Study participants actually used the app, and the researchers said they were surprised by how much and how closely people communicated with the bot. Surveyed afterward, participants reported a degree of "therapeutic alliance" -- trust and collaboration between patient and therapist -- similar to that for in-person therapists.
The timing of interactions was also notable, with interactions spiking in the middle of the night and at other times when patients often experience concerns. Those are the hours when reaching a human therapist is particularly difficult.
"With Therabot, folks will access and did access it throughout the course of the trial in their daily life, in moments where they need it the most," Jacobson said. That included times when someone has difficulty getting to sleep at 2 a.m. because of anxiety or in the immediate wake of a difficult moment.
Patients' assessments afterward showed a 51% drop in symptoms for major depressive disorder, a 31% drop in symptoms for generalized anxiety disorder and a 19% drop in symptoms for eating disorders among patients at risk for those specific conditions.
"The people who were enrolled in the trial weren't just mild," Jacobson said. "The folks in the group were moderate to severe in depression, for example, as they started. But on average experienced a 50% reduction in their symptoms, which would go from severe to mild or moderate to nearly absent."
The research team didn't just choose 100-plus people who needed support, give them access to a large language model like OpenAI's ChatGPT and see what happened. Therabot was custom-built -- fine-tuned -- to follow specific therapy procedures. It was built to watch out for serious concerns, like indications of potential self-harm, and report them so a human professional could intervene when needed. Humans also tracked the bot's communications to reach out when the bot said something it shouldn't have.
Jacobson said during the first four weeks of the study, because of the uncertainty of how the bot would behave, he read every message it sent as soon as possible. "I did not get a whole lot of sleep in the first part of the trial," he said.
Human interventions were rare, Jacobson said. Testing of earlier models two years ago showed more than 90% of responses were consistent with best practices. When the researchers did intervene, it was often when the bot offered advice outside of a therapist's scope -- as when it tried to provide more general medical advice like how to treat a sexually transmitted disease instead of referring the patient to a medical provider. "Its actual advice was all reasonable, but that's outside the realm of care we would provide."
Therabot isn't your typical large language model; it was essentially trained by hand. Jacobson said a team of more than 100 people created a dataset using best practices on how a therapist should respond to actual human experiences. "Only the highest quality data ends up being part of it," he said. A general model like Google's Gemini or Anthropic's Claude, for example, is trained on far more data than just medical literature and may respond improperly.
The Dartmouth study is an early sign that specially built tools using generative AI can be helpful in some cases, but that doesn't mean any AI chatbot can be your therapist. This was a controlled study with human experts monitoring it, and there are dangers in trying this on your own.
Remember that most general large language models are trained on oceans of data found on the internet. So, while they can sometimes provide some good mental health guidance, they also include bad information -- like how fictional therapists behaved, or what people posted about mental health on online forums.
"There's a lot of ways they behave in profoundly unsafe ways in health settings," he said.
Even a chatbot offering helpful advice might be harmful in the wrong setting. Jacobson said if you tell a chatbot you're trying to lose weight, it will come up with ways to help you. But if you're dealing with an eating disorder, that may be harmful.
Many people are already using chatbots to perform tasks that approximate the work of a therapist. Jacobson says you should be careful.
"There's a lot of things about it in terms of the way it's trained that very closely mirrors the quality of the internet," he said. "Is there great content there? Yes. Is there dangerous content there? Yes."
Treat anything you get from a chatbot with the same skepticism you would from an unfamiliar website, Jacobson said. Even though it looks more polished from a Gen AI tool, it may still be unreliable.
If you or someone you love are living with an eating disorder, contact the National Eating Disorder Association for resources that can help. If you feel like you or someone you know is in immediate danger, dial 988 or text "NEDA" to 741741 to connect with the Crisis Text Line.