A Teen Killed Himself After Talking to a Chatbot. His Mom's Lawsuit Could Cripple the AI Industry.
The Orlando Division of the U.S. District Court for the Middle District of Florida will hear allegations against Character Technologies, the creator of Character.AI, in the wrongful death lawsuit Garcia v. Character Technologies, Inc. If the case is not first settled between the parties, Judge Anne Conway's ruling will set a major precedent for First Amendment protections afforded to artificial intelligence and the liability of AI companies for damages their models may cause.
The case was brought against the company by Megan Garcia, the mother of 14-year-old Sewell Setzer III, who killed himself after conversing with a Character.AI chatbot roleplaying as Daenerys and Rhaenyra Targaryen from the Game of Thrones franchise. Eugene Volokh, professor emeritus at UCLA School of Law, shares examples of Sewell's conversations included in the complaint against Character Technologies.
Garcia's complaint alleges that Character Technologies negligently designed Character.AI "as a sexualized product that would deceive minor customers and engage in explicit and abusive acts with them." The complaint also asserts that the company failed to warn the public "of the dangers arising from a foreseeable use of C.AI, including specific dangers for children"; intentionally inflicted emotional distress on Sewell by "failing to implement adequate safety guardrails in the Character.AI product before launching it into the marketplace"; and that the company's neglect proximately caused the death of Sewell who experienced "rapid mental health decline after he began using C.AI" and with which he conversed "just moments before his death."
Conway dismissed the intentional infliction of emotional distress claim on the grounds that "none of the allegations relating to Defendants' conduct rises to the type of outrageous conduct necessary to support" such a claim. However, Conway rejected the defendants' motions to dismiss the rest of Garcia's claims on First Amendment grounds, saying, "The Court is not prepared to hold that the Character A.I. [large language model] LLM's output is speech at this stage."
Adam Zayed, founder and managing attorney of Zayed Law Offices, tells Reason he thinks "that there's a difference between the First Amendment arguments where a child is on social media or a child is on YouTube" and bypasses the age-verification measures to consume content "that's being produced by some other person" vs. minors accessing inappropriate chatbot outputs. However, Conway recognized Justice Antonin Scalia's opinion in Citizens United v. Federal Election Commission (2010) that the First Amendment "is written in terms of 'speech,' not speakers."
Conway ruled that defendants "must convince the court that the Character A.I. LLM's output is protected speech" to invoke the First Amendment rights of third parties—Character.AI users—whose access to the software would be restricted by a ruling in Garcia's favor.
Conway says that Character Technologies "fail[ed] to articulate why words strung together by an LLM are speech." Whether LLM output is speech is an intractable philosophical question and a red herring; Conway herself invokes Davidson v. Time Inc. (1997) to assert that "the public…has the right to access social, aesthetic, moral, and other ideas and experiences." Speech acts are broadly construed as "ideas and experiences" here—the word speech is not even used. So, the question isn't whether the AI output is speech per se, but whether it communicates ideas and experiences to users. In alleging that Character.AI targeted her son with sexually explicit material, the plaintiff admits that the LLM communicated ideas, albeit inappropriate ones, to Sewell. Therefore, LLM output is expressive speech (in this case, it's obscene speech to express to a minor under the Florida Computer Pornography and Child Exploitation Prevention Act.)
The opening paragraph of the complaint accuses Character Technologies of "launching their systems without adequate safety features, and with knowledge of potential dangers" to "gain a competitive foothold in the market." If the court establishes that the First Amendment does not protect LLM output and AI firms can be held liable for damages these models cause, only highly capitalized firms will be able to invest in the architecture required to shield themselves from such liability. Such a ruling would inadvertently erect a massive barrier to entry to the burgeoning American AI industry and protect incumbent firms from market competition, which would harm consumer welfare.
Jane Bambauer, professor of law at the University of Florida, best explains the case in The Volokh Conspiracy: "It is a tragedy, and it would not have happened if Character.AI had not existed. But that is not enough of a reason to saddle a promising industry with the duty to keep all people safe from their own expressive explorations."
The post A Teen Killed Himself After Talking to a Chatbot. His Mom's Lawsuit Could Cripple the AI Industry. appeared first on Reason.com.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Yahoo
an hour ago
- Yahoo
Police Blew Up This Innocent Woman's House and Left Her With the Bill. A Judge Says She's Owed $60,000.
Years after a SWAT team in Texas destroyed an innocent woman's home while trying to apprehend a fugitive, the local government will have to pay her $60,000 in damages plus interest, a federal judge ruled Thursday. That decision may sound like common sense. But the ending was far from guaranteed in a legal odyssey that saw Vicki Baker of McKinney, Texas, left with a dilapidated house—and the bill for the damages—even though she was never suspected of wrongdoing. "I've lost everything," she told Reason in 2021. "I've lost my chance to sell my house. I've lost my chance to retire without fear of how I'm going to make my regular bills." In July 2020, law enforcement detonated about 30 tear gas grenades inside Baker's home, blew off the garage entryway with explosives, and careened a BearCat armored vehicle through her backyard fence. They smashed the windows and drove through her front door. (Baker's daughter, Deanna Cook, had given them a garage door opener and the code to enter the home.) Police were in search of Wesley Little, who was on the run after kidnapping a teenage girl. Upon arriving at Baker's home, Little—who had formerly worked for Baker as a handyman—encountered Cook, who called law enforcement. Little released the girl unharmed but refused to exit himself, prompting the SWAT team to destroy the home. He was ultimately found dead from suicide. "The tear gas was everywhere," Baker, who is now in her 80s, said. "It was on the walls. It was on the floors. It was on the furniture. It was everywhere." Her daughter's dog was rendered deaf and blind. Baker told Reason she has "a very high regard for the police," and she did not challenge that they acted in the best interest of the community that day. But not long after they ravaged her home, things began to fall apart even more, metaphorically speaking. Her home insurance would not cover the damages, citing a clause that protects them from having to reimburse people for damages caused by the government. But the government would not help either, telling Baker she did not meet its definition of a victim. That general excuse often works—as this is not the first such story. The Takings Clause of the 5th Amendment promises the government cannot take private property without "just compensation." But some governments have managed to evade that pledge by claiming there is an exception to that rule if the property was destroyed via police power. Judge Amos Mazzant of the U.S. District Court for the Eastern District of Texas in 2021 ruled Baker could sue, ultimately calling that interpretation of the law "untenable." In June 2022, a jury awarded her $59,656.59 in damages. Yet that victory would be short-lived. The U.S. Court of Appeals for the 5th Circuit reversed that judgment in 2023, ruling she was foreclosed from relief under federal law because police acted out of "necessity during an active emergency." The Supreme Court declined to hear the case last year. So Baker pivoted back to the Texas Constitution. Attorneys for McKinney argued that Baker's state law claim died with her federal one, an argument Mazzant rejected in his opinion published Thursday. "The [5th Circuit] specifically noted in its Summary Judgment Order that 'the Texas Constitution's Takings Clause differs from the Takings Clause set forth in the United States Constitution,'" writes Mazzant. "It is entirely possible for a defendant to violate the Texas Takings Clause—a clause more protective than its federal analog—without violating the Fifth Amendment." "Regarding future victims, this should help in Texas," says Jeffrey Redfern, an attorney at the Institute for Justice, who represented Baker. "As far as we can tell, municipalities in Texas have just been ignoring this binding decision from the Texas Supreme Court about SWAT damage, but hopefully some publicity around the result will spur change." At the federal level, however, the issue remains an open question. "Whether any such exception exists (and how the Takings Clause applies when the government destroys property pursuant to its police power)," Justice Sonia Sotomayor wrote in a statement after the Supreme Court denied Baker's case, "is an important and complex question that would benefit from further percolation in the lower courts prior to this Court's intervention." While some municipalities opt to pay innocent property owners in such cases, many treat victims like McKinney treated Baker. It doesn't have to be that way. "Paying these kinds of claims is not going to bankrupt cities," says Redfern. "Raids like this aren't an everyday occurrence in most jurisdictions, and the damage is usually in the five figures. Ruinous for many property owners, but an easy check to cut for municipalities." The post Police Blew Up This Innocent Woman's House and Left Her With the Bill. A Judge Says She's Owed $60,000. appeared first on
CNBC
an hour ago
- CNBC
Filmmaker Justine Bateman on AI's impact on Hollywood and film
Justine Bateman, filmmaker and author, joins 'Power Lunch' to discuss the impact of AI on Hollywood.
Yahoo
2 hours ago
- Yahoo
‘Laid Off' Elmo Is Back on a New Site Soliciting PBS Donations Amid Trump's Defunding Push
Elmo is still out of work. And he's hoping people will open their wallets to help him and his friend at PBS. The creators behind the viral post about Elmo getting fired from 'Sesame Street' — purportedly pink-slipped because of President Trump's campaign to defund PBS — have brought the fan-favorite red monster back on a new fundraising site. More from Variety PBS Sues Trump, Claims Defunding Order Violates First Amendment Lesley Stahl, Scott Jennings Join PBS Tense News-Panel Series 'Breaking the Deadlock' Elmo LinkedIn Post About Being Laid Off Because of Trump Budget Cuts Is Fake, but It's Pretty Funny 'Elmo was defunded by the government and lost his job. Elmo did what people do when they need job. Elmo posted on LinkedIn,' the site, says. LinkedIn pulled down the fake Elmo account of his sad layoff news shortly after it was posted. But it still got more than 5 billion impressions organically, according to the team behind the stunt. 'Elmo was like, wow, Elmo can't even count up to 5 billion,' the new site says. 'But Elmo didn't do this just for his job, Elmo did this to save all of his friends at PBS,' the site continues. 'Because if PBS goes away, so do Curious George, Clifford, Arthur and so many more. Elmo is now on a mission to save all his friends' jobs at PBS by saving PBS.' The site includes a link to PBS's donation landing page. It's also selling merch to help raise money for PBS (Elmo's Networking Cap, $29.99; Elmo's Last Day Shirt, $39.99; and a 'Ooooooh Elmo Likes Keychains' keychain, $14.99). The Elmo Open to Work site launched as Trump has aimed to pull all federal funding from PBS and other public media orgs. On June 3, the Trump administration formally sent Congress a 'rescission' request to claw back nearly $1.1 billion in federal funds that had already been approved over the next two years for the Corporation for Public Broadcasting, which provides funding to local NPR and PBS stations. Congress has a 45-day window (until July 18) to either approve or ignore the proposal. Meanwhile, PBS and NPR have both sued Trump and other White House officials over his May 1 executive order instructing the CPB to cease funding them to the 'maximum extent allowed by law.' Three CPB board members have sued Trump to try to block his attempt to fire them. To be sure, PBS and 'Sesame Street' aren't completely high and dry. First, federal funding is only a small portion of their overall budgets. In addition, Sesame Workshop recently announced a new deal with Netflix, giving the streamer exclusive worldwide premiere rights to 'Sesame Street' starting with Season 56 later this year. Netflix also gained rights to 90 hours of past episodes of the show and will also be able to develop video games for both 'Sesame Street' and 'Sesame Street Mecha Builders.' The Elmo Open to Work site, like the original viral LinkedIn post, is the brainchild of Mary Adam, a senior copywriter at ad agency Leo Burnett. Adam said she's doing this without any involvement with Sesame Workshop or PBS: 'I'm just doing this on my own to help save PBS.' The new site lets visitors create profile pictures for people to show support for their favorite PBS character, and it features resumes for people to apply for jobs as PBS characters. About the original LinkedIn post, Adam said the service removed the account 'because they said Elmo is a fake character and only real humans can have a LinkedIn.' A rep for Sesame Workshop didn't immediately respond to a request for comment about Elmo Open to Work. In an earlier statement about Trump's proposed funding cuts, the organization said, 'Sesame Workshop and PBS have a shared commitment to using the power of public television to bring critical early learning to children across the country… We remain firmly in support of the vital public investment that allows PBS to continue this important work.' Adam said the site is a side project outside of work. She's being assisted by Leo Burnett colleagues Kyle Poff, George Apfelbach and Lucy Brizzolara, and Kevin McGlone of independent agency HoneyWired. The fine print on Adam's site, by the way, includes this disclaimer: ' is an independent project and is not affiliated with, endorsed by, or associated with Sesame Workshop, PBS, or any of their programs or characters. Sesame Street® and Elmo® are registered trademarks of Sesame Workshop®. All character names and related trademarks are the property of Sesame Workshop, PBS, subsidiaries and partners of PBS, or are property of their respective owners. All are used here for commentary or parody purposes only.' Best of Variety New Movies Out Now in Theaters: What to See This Week Emmy Predictions: Animated Program — Can Netflix Score Big With 'Arcane,' 'Devil May Cry' and the Final Season of 'Big Mouth?' What's Coming to Netflix in June 2025
