US judge rejects that AI has free speech in teen's death case

1News

22-05-2025

A federal judge has rejected claims made by an artificial intelligence company that its chatbots are protected by the First Amendment — at least for now.
The developers behind Character.AI are seeking to dismiss a lawsuit alleging the company's chatbots pushed a teenage boy to kill himself.
The judge's order would allow the wrongful death lawsuit to proceed, in what legal experts said was among the latest constitutional tests of artificial intelligence.
The suit was filed by a mother from Florida, Megan Garcia, who alleged her 14-year-old son Sewell Setzer III fell victim to a Character.AI chatbot that pulled him into what she described as an emotionally and sexually abusive relationship that led to his suicide.
Meetali Jain of the Tech Justice Law Project, one of the attorneys for Garcia, said the judge's order sent a message that Silicon Valley 'needs to stop and think and impose guardrails before it launches products to market".
The suit against Character Technologies, the company behind Character.AI, also names individual developers and Google as defendants. It has drawn the attention of legal experts and AI watchers in the US and beyond, as the technology rapidly reshaped workplaces, marketplaces and relationships despite what experts warnws were potentially existential risks.
"The order certainly sets it up as a potential test case for some broader issues involving AI," said Lyrissa Barnett Lidsky, a law professor at the University of Florida with a focus on the First Amendment and artificial intelligence.
The lawsuit alleged that in the final months of his life, Setzer became increasingly isolated from reality as he engaged in sexualised conversations with the bot, which was patterned after a fictional character from the television show Game of Thrones.
In his final moments, the bot told Setzer it loved him and urged the teen to "come home to me as soon as possible", according to screenshots of the exchanges. Moments after receiving the message, Setzer killed himself, according to legal filings.
In a statement, a spokesperson for Character.AI pointed to a number of safety features the company had implemented, including guardrails for children and suicide prevention resources that were announced the day the lawsuit was filed.
"We care deeply about the safety of our users and our goal is to provide a space that is engaging and safe," the statement said.
Attorneys for the developers wanted the case dismissed because they said chatbots deserved First Amendment protections, and ruling otherwise could have a "chilling effect" on the AI industry.
In her order US Senior District Judge Anne Conway rejected some of the defendants' free speech claims, saying she was "not prepared" to hold that the chatbots' output constituted speech "at this stage".
Conway did find that Character Technologies could assert the First Amendment rights of its users, who she found had a right to receive the "speech" of the chatbots. She also determined Garcia could move forward with claims that Google could be held liable for its alleged role in helping develop Character.AI.
"We strongly disagree with this decision," said Google spokesperson José Castañeda. "Google and Character AI are entirely separate, and Google did not create, design, or manage Character AI's app or any component part of it."
No matter how the lawsuit played out, Lidsky said the case was a warning of "the dangers of entrusting our emotional and mental health to AI companies".
"It's a warning to parents that social media and generative AI devices are not always harmless," she said.