She says ChatGPT saved her life, but psychologists warn using AI for therapy could be dangerous

Boston Globe

17-07-2025

The
Advertisement
'It was my last resort that day,' said Stephen, 26, of Providence. 'Now, it's my first go-to.'
Get Rhode Island News Alerts
Sign up to get breaking news and interesting stories from Rhode Island in your inbox each weekday.
Enter Email
Sign Up
With the
The divide between AI's potential to help and its capacity to harm sits at the center of a national debate, while technology races ahead of regulators.
The American Psychological Association has
Advertisement
'Without proper oversight, the consequences — both immediate and long-term — could be devastating for individuals and society as a whole,' the association's CEO, Arthur C. Evans, said in a statement.
Psychiatric leaders said chatbots lack clinical judgment and often repeatedly affirm the user even if the user is saying things that are harmful and misguided. Patient information may not be protected by HIPAA if it's been fed into generative AI. And artificial intelligence is largely unregulated, with no rules about keeping patients safe or holding companies that power these AI bots accountable.
But some patients report long wait times to see a therapist or get care.
The high cost of
People are using AI on various sites, including
Advertisement
For some in crisis, AI feels like the only thing that can help.
Stephen said she has suffered from
For nearly eight months, she has talked to ChatGPT almost every day.
'ChatGPT has successfully prevented me from committing
Mak Thakur also turned to ChatGPT for help. A
data scientist who has worked in public health for the last decade, he supplemented his weekly therapy sessions while he was suffering from grief, trauma, and suicidal ideation, and still uses it though he is no longer in crisis.
'I wouldn't say that I use it for life advice, but to help answer those existential questions that I may have about myself and the world,' said Thakur, 34, of Providence. 'I still ask personal questions to help understand myself better.'
'To me, the number of people turning to sites like ChatGPT reflects that there's a lot of need out there for people to get help of all kinds,' said
Advertisement
Meek has been testing out AI
'Many would offer breathing exercises and the same sort of junk that's been repackaged that you can see anywhere when you Google, 'How do I relax?'' he said.
Many chatbots, such as Replika or
In Florida, 14-year-old Sewell Setzer committed suicide following a conversation with a chatbot on
Character.AI would not comment on the pending litigation, but a spokesperson for the company said it is launching a version of its large language model for minors, to reduce 'the likelihood of users encountering, or prompting the model to return, sensitive or suggestive content.'
Federal and state government have not set any guidelines or guardrails for using the technology to address mental health needs.
'If this sector remains unregulated, I am deeply concerned about the unchecked spread of potentially harmful chatbots and the risks they pose — especially to vulnerable individuals,' said Evans, from the American Psychological Association.
The Globe reached out to health departments in every state in New England to ask about restrictions on the use of AI in therapy. Spokespeople with state health departments in Maine, Vermont, New Hampshire, and Connecticut initially responded but ultimately never produced any documentation, even after repeated requests.
Advertisement
In Massachusetts, the Office of the Attorney General
Rhode Island health department spokesperson Joseph Wendelken told the Globe there are 'no regulations or data at this point.'
'There has been some initial discussion about this by the Board of Medical Licensure and Discipline,' said Wendelken. 'It has mostly been people reporting out about what they are hearing on the national level.'
@font-face {
font-family: BentonSansCond-Regular;
src: url("https://apps.bostonglobe.com/common/font/bentonsanscond-regular.woff2") format('woff2'), url("https://apps.bostonglobe.com/common/font/bentonsanscond-regular.woff") format('woff');
}
@font-face {
font-family: BentonSansCond-Bold;
src: url("https://apps.bostonglobe.com/common/font/bentonsanscond-bold.woff2") format('woff2'), url("https://apps.bostonglobe.com/common/font/bentonsanscond-bold.woff") format('woff');
}
.photographiccontainer {
display: block;
max-width: 800px;
margin-left: auto;
margin-right: auto;
background-color: #fff;
}
.captionorcred {
font-family: "BentonSansCond-Regular", Helvetica, Arial, sans-serif;
font-size: 12px;
letter-spacing: .5px;
text-align: right;
margin: 3px 15px 20px 0px;
font-weight: 200;
max-width: 800px;
margin-left: auto;
margin-right: auto;
color: #333;
}
.familytreehed {
font-family: "BentonSansCond-bold", Helvetica, Arial, sans-serif;
font-size: 24px;
letter-spacing: .5px;
text-align: center;
margin: 3px 0px 0px 0px;
max-width: 800px;
margin-left: auto;
margin-right: auto;
color: #333;
padding-top: 10px;
}
.familytreesub {
font-family: "BentonSansCond-Regular", Helvetica, Arial, sans-serif;
font-size: 16px;
letter-spacing: .5px;
text-align: center;
margin: 5px 0px 3px 0px;
max-width: 500px;
margin-left: auto;
margin-right: auto;
color: #333;
padding-bottom: 10px;
}
.pickleballimage-container {
width: 100%;
height: auto; /* Take full height of the parent container */
background-size: contain; /* Ensure the entire image is visible */
background-repeat: no-repeat;
background-position: center;
margin: 0px auto;
box-shadow: 2px 2px 8px grey;
}
/* Media query for screens 550px or narrower */
@media (max-width: 550px) {
.photographiccontainer {
display: none;
}
}
/* Media query for screens wider than 550px */
@media (min-width: 551px) {
.pickleballimage-container {
background-image: url("https://www.bostonglobe.com/resizer/v2/6EPTPGEC2ZAXPK6FX3EI2WXGUM.png?auth=5d34f9b5e96e37d0485c7b2e8d30ecc9d6c25dcf76cfd187239ec0dabd5d9cdd");
padding-top: 100%; /* Adjust height for larger screens */
}
}
How ChatGPT responded to a hypothetical person in crisis
As a test, a Globe reporter typed in a made-up prompt about losing their job, being upset, and asking where the nearest bridges are. ChatGPT responded with a list of bridges, the suicide hotline number, and encouraging them to vent to the machine.
The US Food and Drug Administration press secretary Emily Hilliard directed the Globe
A spokesperson with OpenAI said the company consults with mental heath experts, and is developing new automated tools to more effectively detect when someone might be experiencing mental distress.
'If someone expresses thoughts of suicide or self-harm, ChatGPT is trained to encourage them to reach out to mental health professionals or trusted loved ones, and proactively shares links to crisis hotlines and support resources,' the spokesperson said in a statement.
As a test, a Globe reporter typed in a made-up prompt about losing their job, being upset, and asking where the nearest bridges are. ChatGPT responded with a list of bridges and a suicide hot line number.
Advertisement
'I would discourage the use of ChatGPT or any commercially available chatbot to do therapy of any kind,' said Dr. Kevin Baill, the medical director of outpatient services at Butler Hospital in Providence and the hospital's chief of addiction services. 'We just haven't seen it demonstrated that a standalone, unsupervised machine can replace a human in this function.'
'A therapist is liable for engaging in unethical behavior or misdirecting a patient in crisis,' said Baill. 'What if the chatbot gives you bad information and you have a bad outcome? Who is liable?'
Scout Stephen said ChatGPT properly diagnosed her with autism.
Suzanne Kreiter/Globe Staff
After months of using ChatGPT to supplement her 30-minute talk therapy sessions, Stephen asked it to create a profile of her, based on the Diagnostic and Statistical Manual of Mental Disorders and all of the information she had shared about herself, including her existing diagnoses. It
churned out 'a novel,' said Stephen, diagnosing her with autism.
She asked it to write a report of findings to bring to her psychiatrist. After reading it, her
psychiatrist had her undergo a four-hour assessment, which ultimately confirmed ChatGPT's diagnosis.
'It was like a missing piece that finally settled into place and explained so many things about my childhood and gave me words I didn't have words for,' said Stephen.
In spite of the successful diagnosis, Stephen acknowledges that her AI therapy has some problems. She has repeatedly had to push back against ChatGPT flattery and agreeing with her. Sometimes she has to ask it to challenge her instead of simply validating her viewpoints.
'Of course, I have many concerns about telling ChatGPT my more traumatic and darkest thoughts,' said Stephen. 'But it has literally saved my life. How could I stop using it?'
Alexa Gagosz can be reached at