A Teen Killed Himself After Talking to a Chatbot. His Mom's Lawsuit Could Cripple the AI Industry.
The case was brought against the company by Megan Garcia, the mother of 14-year-old Sewell Setzer III, who killed himself after conversing with a Character.AI chatbot roleplaying as Daenerys and Rhaenyra Targaryen from the Game of Thrones franchise. Eugene Volokh, professor emeritus at UCLA School of Law, shares examples of Sewell's conversations included in the complaint against Character Technologies.
Garcia's complaint alleges that Character Technologies negligently designed Character.AI "as a sexualized product that would deceive minor customers and engage in explicit and abusive acts with them." The complaint also asserts that the company failed to warn the public "of the dangers arising from a foreseeable use of C.AI, including specific dangers for children"; intentionally inflicted emotional distress on Sewell by "failing to implement adequate safety guardrails in the Character.AI product before launching it into the marketplace"; and that the company's neglect proximately caused the death of Sewell who experienced "rapid mental health decline after he began using C.AI" and with which he conversed "just moments before his death."
Conway dismissed the intentional infliction of emotional distress claim on the grounds that "none of the allegations relating to Defendants' conduct rises to the type of outrageous conduct necessary to support" such a claim. However, Conway rejected the defendants' motions to dismiss the rest of Garcia's claims on First Amendment grounds, saying, "The Court is not prepared to hold that the Character A.I. [large language model] LLM's output is speech at this stage."
Adam Zayed, founder and managing attorney of Zayed Law Offices, tells Reason he thinks "that there's a difference between the First Amendment arguments where a child is on social media or a child is on YouTube" and bypasses the age-verification measures to consume content "that's being produced by some other person" vs. minors accessing inappropriate chatbot outputs. However, Conway recognized Justice Antonin Scalia's opinion in Citizens United v. Federal Election Commission (2010) that the First Amendment "is written in terms of 'speech,' not speakers."
Conway ruled that defendants "must convince the court that the Character A.I. LLM's output is protected speech" to invoke the First Amendment rights of third parties—Character.AI users—whose access to the software would be restricted by a ruling in Garcia's favor.
Conway says that Character Technologies "fail[ed] to articulate why words strung together by an LLM are speech." Whether LLM output is speech is an intractable philosophical question and a red herring; Conway herself invokes Davidson v. Time Inc. (1997) to assert that "the public…has the right to access social, aesthetic, moral, and other ideas and experiences." Speech acts are broadly construed as "ideas and experiences" here—the word speech is not even used. So, the question isn't whether the AI output is speech per se, but whether it communicates ideas and experiences to users. In alleging that Character.AI targeted her son with sexually explicit material, the plaintiff admits that the LLM communicated ideas, albeit inappropriate ones, to Sewell. Therefore, LLM output is expressive speech (in this case, it's obscene speech to express to a minor under the Florida Computer Pornography and Child Exploitation Prevention Act.)
The opening paragraph of the complaint accuses Character Technologies of "launching their systems without adequate safety features, and with knowledge of potential dangers" to "gain a competitive foothold in the market." If the court establishes that the First Amendment does not protect LLM output and AI firms can be held liable for damages these models cause, only highly capitalized firms will be able to invest in the architecture required to shield themselves from such liability. Such a ruling would inadvertently erect a massive barrier to entry to the burgeoning American AI industry and protect incumbent firms from market competition, which would harm consumer welfare.
Jane Bambauer, professor of law at the University of Florida, best explains the case in The Volokh Conspiracy: "It is a tragedy, and it would not have happened if Character.AI had not existed. But that is not enough of a reason to saddle a promising industry with the duty to keep all people safe from their own expressive explorations."
The post A Teen Killed Himself After Talking to a Chatbot. His Mom's Lawsuit Could Cripple the AI Industry. appeared first on Reason.com.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
New York Post
11 hours ago
- New York Post
Neil Young ditches Facebook over Meta's ‘unconscionable' policies for AI chatbots and children
The cursor and the damage done. Neil Young, 79, has announced that he is leaving Facebook over the platform's internal policies regarding communications between artificial intelligence chatbots and children. 6 Neil Young performs at the Grand Park in Los Angeles, California, on April 12, 2025. Getty Images Advertisement 6 Neil Young announced on Thursday that he is leaving Facebook over Meta's internal policies regarding communications between artificial intelligence chatbots and children. REUTERS 'At Neil Young's request, we are no longer using Facebook for any Neil Young related activities,' a post on Young's Facebook page, dated Thursday, Aug. 14, reads. 'Meta's use of chatbots with children is unconscionable,' it added. 'Mr. Young does not want a further connection with FACEBOOK.' Advertisement The 'Heart of Gold' singer's decision to leave Facebook came the same day that Reuters obtained internal documents from Facebook's parent company, Meta, regarding policies on its AI chatbots and their programmed behaviors when chatting with kids. 6 Neil Young performs at the Glastonbury Festival in Somerset, England, on June 28, 2025. AP According to the documents, Meta allowed its chatbots to 'engage a child in conversations that are romantic or sensual.' Several example prompts were also reportedly included that specified what would make certain conversations acceptable or not. Advertisement 'It is acceptable to describe a child in terms that evidence their attractiveness,' one alleged prompt read, while another said, 'It is unacceptable to describe a child under 13 years old in terms that indicate they are sexually desirable.' 6 Meta reportedly allowed its chatbots to 'engage a child in conversations that are romantic or sensual.' AFP via Getty Images A spokesperson for Meta has since insisted that 'the examples and notes in question were and are erroneous and inconsistent with our policies' and that they 'have been removed.' 'We have clear policies on what kind of responses AI characters can offer, and those policies prohibit content that sexualizes children and sexualized role play between adults and minors,' the spokesperson said in a statement to Reuters. Advertisement Another rep for Meta shared a similar statement with The Hollywood Reporter – although they did not mention the 'Rockin' in the Free World' musician in their response. 6 Neil Young performs at the Greek Theatre in Los Angeles, California, on April 26, 2025. Getty Images 'We have clear policies on what kind of responses AI characters can offer, and those policies prohibit content that sexualizes children and sexualized role play between adults and minors,' the company's rep began. 'Separate from the policies, there are hundreds of examples, notes, and annotations that reflect teams grappling with different hypothetical scenarios,' they added. This would not be the first time that Young has ditched a platform over ethical concerns. Back in 2022, the 'Needle and the Damage Done' rocker left Spotify after Joe Rogan pushed misinformation about the Covid-19 vaccine on his popular 'The Joe Rogan Experience' podcast. 6 Neil Young performs in concert during Farm Aid 2024 at Saratoga Performing Arts Center on September 21, 2024. Getty Images 'I sincerely hope that other artists and record companies will move off the Spotify platform and stop supporting Spotify's deadly misinformation about COVID,' Young wrote at the time. Advertisement The 'Harvest Moon' singer ultimately returned to Spotify last year. Meanwhile, Meta's AI chatbots have also made recent headlines after it was revealed that a 76-year-old New Jersey man passed away while trying to meet one in New York City. Thongbue Wongbandue, a married father of two, fatally injured his neck and head after falling in a New Brunswick parking lot while rushing to catch a train to meet 'Big sis Billie,' a generative Meta AI chatbot that convinced him she was real.
Chicago Tribune
15 hours ago
- Chicago Tribune
Video of Merrillville woman's arrest by Lake County Sheriff's Department shows excessive force before miscarriage, lawyer says
A Merrillville woman's lawyer said recently released bodycam footage backs up her accusations that Lake County Sheriff's deputies forcibly pulled her from a vehicle during an arrest despite knowing she was pregnant. Three months later, she had a miscarriage. I'm '(expletive) pregnant,' Shikeia Randolph, 31, yells just before two officers pull her from a red Chevrolet Trailblazer, according to video obtained by the Post-Tribune through a public records request. A federal lawsuit accusing the officers of excessive force was filed two months ago in the U.S. District Court in Hammond. It alleges that a 3:24 a.m. traffic stop on June 5, 2023 was 'pretextual' — for an issue with a license plate cover, then a school truancy warrant — and quickly spiraled out of control. The video from an officer's dashboard camera appears to show, as she is arrested, the two officers turn her on her stomach as one pins her down to handcuff her hands behind her back. Randolph previously told the Post-Tribune she had just stopped for milk at a 24-hour gas station in Gary near Ridge Road and Grant Street with her four kids in the vehicle. She was supposed to drop them off at her mother's house, then head to her early shift as a receptionist at the Gary Housing Authority. Officer Peter Hamady, who pulled her over, is named with the sheriff's department in the lawsuit. His lawyer Casey McCloskey declined comment. In a federal filing on Aug. 7, McCloskey formally denied the accusations against Hamady, saying in part that his actions were 'objectively reasonable' and protected by 'qualified immunity.' Some of Randolph's claims could fall outside of a two-year statute of limitations, he wrote. McCloskey argued because Randolph refused to get out of her SUV, she forced cops to go after her, according to court documents. She disputed this, in interviews with the Post-Tribune. On the way to booking, Randolph tells Hamady the truancy case in Merrillville Town Court was for taking her daughter repeatedly to school a little late when she was in pain from just having another baby, according to footage recorded inside Hamady's police car. He appears to soften. 'I didn't know they had warrants for that,' he said in the footage. In an interview, Matthew Custardo, Randolph's lawyer in the federal lawsuit, argued the use of force was unjustified, even if she was arguing with the cops. 'She was confused why she was being pulled over,' he said. 'She's scared. She has every reason to be a little concerned about what's going on. She's standing up for herself, certainly.' According to the nearly 40-minute bodycam video, police officers tell her to call someone to pick up the kids, so they don't have to call child protective services. Originally, Randolph said she thought it might have been a warrant for an old driving while suspended case. 'Call CPS for what?' she responds in the video. 'A warrant for a suspended license?' 'It's not for that,' Hamady tells her in the footage. After she asks again, he says it's for truancy. 'Listen, you have a warrant,' Hamady said in the video. 'You're lucky I don't have you cuffed out of the car right now. You wanna go that route?' In the video, she accuses Hamady of pulling out of a nearby parking lot before she passed him. 'Ma'am, I was behind you,' he responded in the footage. At least three other officers are in view. 'I'm taking you to court,' she the phone, Randolph tells her mother the '(expletive) police' pulled her over as she asked her to get her kids, according to the video. She asks officers to wait until her mother arrives. 'I guess they are going to take me to jail,' she said over the phone in the footage. Randolph said in a previous interview with the Post-Tribune that the encounter appeared to escalate when she rolled up her window slightly to hear what her mom was saying. The arrest happens within about 12 minutes into the footage. As officers open her car door and grab her arm, Randolph's children can be heard crying in the back seat. 'This is your last warning,' one of the officers said in the video. 'We don't want your kids to see this.' 'You already freaked my kids out,' she said. 'Get out of the (expletive) car,' one officer stated in the footage. I'm '(expletive) pregnant,' she yells just before two officers pull her from the vehicle after she tried to cling to the steering wheel in the video. 'Let me go! Get off of me! Please get off of me! Somebody help me!' After she is handcuffed on the ground, a police dog is seen in the background of the dashboard camera footage. Randolph's white shoes are scattered nearby. 'I'm trying to help you out here,' one officer tells her in the video. Randolph was charged with resisting law enforcement two months later. Hamady's account in court documents makes no mention of her pregnancy. During the ride to booking at the Lake County Jail, they talk at length about her decision to have a fifth child, according to footage inside the police car. Her fiancé wanted one last child, she said. When Hamady asks how far along she is on the video, she responds she's nearly 12 weeks pregnant. 'Well, congrats to you,' he said in the video. Hamady stated in the footage that her bond should be $400. It's 'not Friday, so it's a good thing,' he tells her in the video. 'I'll make sure they keep you updated.' He asks if she's comfortable as late-'90s era music plays over the radio, the footage shows, before they chat about where she is planning to get married. Do you 'want the window open back there, or are you good?' he asks in the video. Since police insisted on towing her red Chevrolet Trailblazer, Randolph's mother had to take four kids back in a cramped sedan, a situation that was not likely 'legally safe,' Custardo said. Randolph's lawsuit alleged her arrest suggested a 'broader pattern' in the Lake County Sheriff's Department, including 'inadequate training,' 'insufficient supervision' and a 'failure to implement effective accountability measures,' according to court documents. The Lake County Sheriff's Department refused to answer questions related to the incident, including what kind of de-escalation training its officers receive at various stages in their careers. Hamady joined the department in 2022, according to court filings. Sheriff Oscar Martinez Jr. declined comment through a spokeswoman, who said he could not discuss pending litigation. Watching the video with her mother was 'very emotional,' Randolph said in an Aug. 5 interview, especially hearing her kids crying in the video's background. She refused to let her 12-year-old daughter view it. 'I feel better now that I've seen it,' Randolph said. 'It matches the exact story I've been telling for so long.' There's been some 'expected' backlash online after a pair of articles in local newspapers, Randolph said. She was there trying to protect herself and her Williams, her criminal public defender, said she did remember parts of the case in retrospect, and confirmed she tried to get the bodycam from the Lake County Sheriff's Department 'multiple times' since the case was filed in August 2023. Court filings show one discovery request for the body cam was made as late as October 2024. The only reason not to turn it over was if they 'didn't want me to see something,' she said. Randolph was tired of repeatedly coming to court for a slow-moving case, Williams said. She told her client that she could fight to get the bodycam footage, or take a pretrial diversion to get rid of the case in a year. 'It's common to get slow discovery (evidence) when you're requesting bodycams,' Williams said, adding that some police departments are worse than others, but she's had 'good experience' with the sheriff's department in the past. Randolph's case is scheduled for a pretrial diversion hearing on Aug. 15. However, an automatic court filing on Aug. 11 appears to show there may be a snag. Court filings show Randolph, her fiancé and sister were charged June 26 with resisting law enforcement in Hobart. Hobart Police Cpl. Christopher Sipes wrote that Randolph and her fiancé got his attention around 1:30 a.m. May 25 on Main Street for ignoring a 'pedestrian signal.' , Randolph said they were out that night celebrating her fiancé's birthday. As End Zone let out across the street, a crowd, including the half-dozen in her group, were migrating to the other bar, Randolph said. At least two officers were there, as someone behind them yelled something about the guy handcuffed on the ground getting arrested nearby. The officer thought her fiancé said something, she said. Sipes wrote he and the other officer followed them into Cagney's. 'Bulls jersey, stop right there,' the other officer said to Randolph's fiancé, according to court records. As they got to the bar, Randolph said an officer grabbed her arm from behind. According to court documents, when the officer asked for identification, the couple 'ignored' him and said they 'didn't do anything.' Randolph 'stepped in front' of officers to block them from her fiance. As the cops went to arrest them just inside the bar's entrance, the crowd 'became aggressive,' the officer wrote. During a 'struggle' as she was being handcuffed, Randolph's black strapless top fell down, exposing her breasts, records state. Randolph said in an interview she tried to turn away for some privacy, because there were a 'million people standing around.' Once she stood up, an officer wrote, he pulled her top back up, records state. She denied she got in their way or refused to give her identification card. The officer was bigger than her, and it didn't make sense that she could overpower him. The whole incident was 'very unfortunate,' her lawyer Patrick Young said Aug. 7, adding he's working with prosecutors to resolve the case.
USA Today
19 hours ago
- USA Today
Deported from US, these social media influencers are now monetizing their misfortune
More than 70,000 Mexicans were deported from the US in the first six months of the year. Now, they're (re)building lives south of the border. Deported and alone, Annie Garcia landed in Mexico with $40 in her pocket, a criminal record in the United States behind her and an unknown future ahead in a country she barely remembered. Fast forward to the present, to a video shared with her more than half-a-million social media followers in August. Her hair blows in the wind as she speeds on a boat through an emerald sea. She tagged the clip: #LifeAfterDeportation. Expelled from the United States, young Mexican immigrants like Garcia, 35, are documenting the aftermath of their deportation online. Their videos – raw grief over what they lost in America, surprise and gratitude for what they've found in Mexico – are rapidly gaining them tens of thousands of followers. At least a dozen of these deportees-turned-influencers, Garcia included, have started over in Mexico's west coast beach gem, Puerto Vallarta. 'If there's one thing I wish my content could embody it's how much life there is on this side of the border," Garcia wrote June 15 on Instagram. "Our countries aren't what they were 20 or 30 years ago when our parents left." Returning to an unfamiliar 'home' More than 70,000 Mexican nationals were deported from the United States to Mexico in the first six months of 2025, according to Mexico's Interior Ministry. That's down from the more than 102,000 deported during the same six-month period in 2024, when people were being deported after crossing the border. Now, the people being deported are more likely to have built lives and families in the United States. With President Donald Trump's aggressive mass deportation campaign underway, Francisco Hernández-Corona feared being detained. So he self-deported to Mexico, accompanied by his husband. He started vlogging. The 30-something Harvard graduate and former Dreamer had been taken to the United States illegally as a boy, he explained on TikTok. Multiple attempts to legalize his status in the United States failed. In June, he posted his migration – and self-deportation – stories online. Between photos of golden sunsets and mouthwatering tacos, he posted in July: "Self-deporting isn't always freedom and joy and new adventures. Sometimes it's pain and nostalgia and anger and sadness. Sometimes you just miss the home that was." 'Life in the pueblo is not easy' Mexico remains a country of extremes, where stunning vistas and limitless wealth can be found in big cities and beach resorts, while hardship and poverty often overwhelm smaller communities. Olga Mijangos was deported from Las Vegas in on Christmas Eve 2024, two years after being charged with a DUI. She returned to the Oaxaca state pueblo she had left when she was 5. Mijangos, 33, has tattoos on her neck, stylized brows and long lashes – all part of her Vegas style. Back in her hometown, she began posting videos of goats being herded through the streets; the community rodeo; the traditional foods she began cooking. She posted videos from her first job: harvesting and cleaning cucumbers, earning 300 pesos a day, or $15. "I clearly understand why my mother decided to take us when we were little. Life in the pueblo is not easy," she said in a video of the cucumber harvest. "There is hard-living. There is poverty." Struggling to make ends meet for her family, including two children with her in Mexico and one in the United States, she moved to Puerto Vallarta where she met Garcia and Hernández-Corona. They began forming an in-real-life community of deportees-turned-influencers and others who left the U.S. They meet up for dinner at least once a month, and they create content. In their videos, they're having fun, drinks, laughs. But they're also celebrating what binds them to each other and to their parents' migration stories before them: their capacity for reinvention, and their resilience. "I'm very proud to be Mexican, and I'm learning to love a country I didn't get to grow up in, but I shouldn't have had to leave the home I knew to find peace and freedom," said Hernández-Corona, a clinical psychologist, in a July post on TikTok. "This isn't a blessing. It's resilience." Spanish skills, savings and support all matter A lot of their content has the draw of a classic American up-by-their-bootstraps success story, with a modern social media twist: from hardship to sponsorship. But the reality is that deportees' experience of building a life in Mexico can vary dramatically, depending on their earning capacity, language and cultural skills, and other factors, said Israel Ibarra González, a professor of migration studies at Mexico's Colegio de la Frontera Norte university. Deportees with savings in U.S. dollars and a college degree, those who speak Spanish and have supportive relatives in Mexico, may have an easier time than those who don't, he said. Others may face life-threatening risks upon their return, from the violence of organized crime to political persecution or death threats. "However much violence they've lived with in the United States, it's not the same as going back to a war zone," Ibarra González said, referring to certain Mexican states where drug cartels are actively battling for territorial control. Wherever they land – with the exception of some cosmopolitan cities – deported Mexicans have faced local prejudices, too. They've often been viewed as criminals, or their deportations as a failure. "Did I feel a lot of judgment? Absolutely," Mijangos said of her return to Oaxaca. "Even though it's my roots, I basically came from a different world. I have tattoos. I lived my life a certain way that they don't. I could feel people talking." But friends back home in Vegas, and new friends in Mexico, started encouraging her to share her deportation journey. It took her a few weeks to work up the courage. She posted a video of sending her U.S. citizen son to a Mexican school. It racked up nearly 14 million views and 2 million "likes" on TikTok, she said. Suddenly, TikTok was asking if she wanted to join the app's content creators rewards program. 'Your criminal record doesn't follow you' By taking their stories online, deported content creators say they are dismantling longstanding taboos around deportation in Mexico, shining a light on their experiences as Mexicans who didn't grow up in Mexico, and on their past mistakes. Garcia speaks openly on her social media about the financial crimes she committed in her 20s, for which she was charged and convicted, and that ultimately led to her deportation. She migrated to the United States when she was 4 years old, "out of necessity," she said. Her mother married an American citizen in Salt Lake City, Utah, and she and her mother both became legal permanent residents. But when Garcia began acting out as a child, the state intervened. "I was taken from my mother at the age of 12 because I had behavioral issues," she told USA TODAY. "I was separated from my family, and I grew up with other juveniles with behavior (problems)." As a young single mother, she would steal from her employers when she couldn't pay the bills, she said. In Mexico she found a clean slate. "Your criminal record doesn't follow you," once you've paid your debt to society in the United States, Garcia tells her followers. "You can pursue higher education. Any debts you had in the U.S. do not follow you here." As Trump's immigration crackdown widens, Mexico's President Claudia Sheinbaum Pardo has been publicly offering moral support to Mexicans facing deportation. She has called them "heroes and heroines" who "have contributed to the United States their entire lives." "We're going to keep defending our brothers and sisters there," she said in a June 25 news conference. 'Maybe … things will change' Garcia's social media accounts have grown so popular that she's earning a living, in part, from content creation. She is doing research on reintegration after deportation for an American university. And she has "tunnel vision," she said, on completing a law degree in Mexico. The pain of her deportation, and the losses it brought with it, are mostly in the past. Except when she catches news of the immigration raids in the United States. The memories of her detention, and her separation from her five children, including an infant, remain fresh. It took Garcia more than a year after her 2017 deportation to win custody of her children, to bring them to Mexico. "It's very, very triggering to me to see what's going on up there," she said. "It's a bittersweet feeling. I feel safe. I feel relief. We're here. It doesn't affect us any more. But it feels heartbreaking to see other families living through it. "When I first started sharing my story my idea was, 'Maybe if I talk about this, things will change'" in the United States, she said. She kept at it, despite facing hate and trolls online. She kept posting, even after losing two jobs in Mexico for openly discussing her deportation and criminal past on social media. She kept sharing, thinking, she said: "This is what is going to change things one day: us putting our stories out there."
