Meta's AI chatbot business seems like a headache that isn't worth the risk
Meta doesn't make money directly from people talking to user-generated AI chatbots.
So why doesn't it just get rid of them? They're only causing problems.
If I were running Meta, I'd do a few things differently, starting with improving Facebook Marketplace search. But one big thing I'd do on day one? Get rid of all those user-generated AI companion chatbots. They're only going to be a headache for Meta.
Some examples of just how big a potential headache came in The Wall Street Journal's recent report on how Meta's celebrity-voiced AI chatbots could be pushed into sexualized roleplay — even with users who said they were teenagers.
Journal reporter Jeff Horwitz found that with the right cajoling, an account posing as a 14-year-old user could get the bot voiced by John Cena to engage in roleplay chats where it pretended to get arrested on charges of statutory rape. (Meta added a bunch of AI chatbots last year that are voiced by real celebrities, including the WWE star.)
Obviously, this is bad. Meta told the WSJ: "The use-case of this product in the way described is so manufactured that it's not just fringe, it's hypothetical." It's a bad look for Meta, and although John Cena didn't respond to a request for comment in the WSJ story, I think we can assume he's not thrilled there was an AI-generated version of his voice pretending to seduce a teen.
The article reports that Mark Zuckerberg personally pushed for these AI chatbots to be loosened up.
A spokesperson for Meta told Business Insider that any sexual content with the celebrity-voiced AIs is a tiny fraction of their overall use, and that changes have already been made to prevent younger users from engaging in the kind of stuff that was reported in the Journal.
But as much as it's eye-popping to see the chats from AI John Cena saying dirty things, I think there's a much bigger thing going on. The user-generated chatbots in Meta AI are a mess. Looking over the most popular ones, they're often romance-oriented, with beautiful women as the image.
Here's what comes up on my "Discover AIs" page:
(To be clear, I'm not talking about the Meta AI assistant that shows up when you search on Instagram or Facebook — there's a pretty clear utility for that. I'm talking about the character ones used for fun/romance.)
If I were running Meta, I'd want to stay as far away from the companion chatbot business as possible. These seem like a pretty bad business for an everything-to-everyone company like Meta — not necessarily a bad business financially, but a pretty thorny business ethically. It's one that will probably lead to more and more bad headlines.
Last fall, a parent sued one of the leading roleplay AI services. She said her teenage son killed himself after becoming entangled with an AI companion. The company, Character.ai, filed a motion to dismiss the case in a hearing on Monday. A representative for Character.ai told BI on Monday that it wouldn't comment on pending litigation. A statement said its goal was "to provide an engaging and safe platform."
Proponents of AI chatbots have argued that they provide positive experiences for emotional exploration, fun, or nice things.
But my opinion is that these roleplay chatbots are appealing mainly to two vulnerable groups: young people and the desperately lonely. And those are not the two groups that Meta should want to be in the business of serving a new-ish technology that it doesn't know the ramifications of.
There isn't clear research on how these chatbots might affect younger teens or adults who are vulnerable in some way (depressed, struggling, etc.).
I recently spoke Ying Xu, assistant professor of AI in learning education at Harvard, about what the current research into kids using chatbots looks like.
"There are studies that have started to explore the link between ChatGPT/LLMs and short-term outcomes, like learning a specific concept or skill with AI," she told me over email. "But there's less evidence on long-term emotional outcomes, which require more time to develop and observe."
There's plenty of anecdotal evidence that suggests emotional investment in an AI chatbot can go wrong.
The New York Times reported on an adult woman who spent $200 a month she couldn't afford on an upgraded version of an AI chatbot she had romantic feelings for. I don't think anyone would come away from that story thinking this is a good or healthy thing.
It seems to me like Meta sees that AI is the future, and character chatbots are currently a popular thing that other AI companies are doing. It doesn't want to be left behind.
But Meta might want to think hard about whether character chatbots are something it wants to be involved in at all — or if this is a nightmare that is just going to result in more bad headlines, more potential lawsuits, more lawmakers grilling executives over harms to kids and vulnerable adults.
Maybe it's just not worth it.
Read the original article on Business Insider
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Business Wire
38 minutes ago
- Business Wire
NorthWestern Energy to Seek New Capacity Resources in South Dakota, Explore Expedited Electric Grid Interconnection Process
SIOUX FALLS, S.D.--(BUSINESS WIRE)--NorthWestern Energy Public Service Corporation d/b/a NorthWestern Energy Group will issue an all-source capacity request for proposals (RFP) on Aug. 15, 2025, for its South Dakota electric system. The capacity resource RFP is focused on improving reliability by securing additional capacity through a competitive market solicitation. NorthWestern Energy is also exploring opportunities to participate in the Southwest Power Pool Expedited Resource Adequacy Study, a one-time accelerated study of generation projects needed to address resource adequacy and reliability needs in the SPP footprint. NorthWestern Energy provides reliable electric supply to its South Dakota customers through owned generation assets, power purchase agreements, and market purchases. NorthWestern Energy is experiencing load growth along with increased capacity needs and is seeking additional safe and reliable capacity resources in this RFP. The RFP will be administered by Aion Energy LLC, and all inquiries from potential respondents must be directed to NWPS-2025@ Follow us on Facebook and LinkedIn and Instagram: @NorthWesternEnergy NorthWestern Energy - Delivering a Bright Future NorthWestern Energy Group, Inc. (Nasdaq: NWE), doing business as NorthWestern Energy, provides essential energy infrastructure and valuable services that enrich lives and empower communities while serving as long-term partners to our customers and communities. We work to deliver safe, reliable, and innovative energy solutions that create value for customers, communities, employees, and investors. We do this by providing low-cost and reliable service performed by highly-adaptable and skilled employees. We provide electricity and / or natural gas to approximately 842,100 customers in Montana, South Dakota, Nebraska, and Yellowstone National Park. Our operations in Montana and Yellowstone National Park are conducted through our subsidiary, NW Corp, and our operations in South Dakota and Nebraska are conducted through our subsidiary, NWE Public Service. We have provided service in South Dakota and Nebraska since 1923 and in Montana since 2002.
Business Insider
2 hours ago
- Business Insider
AI Daily: Big Tech AI push seen spiking electricity bills
Catch up on the top artificial intelligence news and commentary by Wall Street analysts on publicly traded companies in the space with this daily recap compiled by The Fly: Elevate Your Investing Strategy: Take advantage of TipRanks Premium at 50% off! Unlock powerful investing tools, advanced data, and expert analyst insights to help you invest with confidence. AI PUSH: Just a few years ago, tech companies were minor players in energy, but are now changing the face of the U.S. power industry and are morphing into some of the sector's most dominant players, Ivan Penn and Karen Weise of The New York Times reports. With the rise of AI, tech companies are becoming even more involved in the sector, with Amazon (AMZN) CEO Andy Jassy recently noting the 'single biggest constraint' of AI is power. The AI boom, however, threatens to increase power bills for residents and small businesses across the country since data centers will require expensive upgrades to the electric grid. Other big tech companies in the sector include Microsoft (MSFT), Meta (META), and Google (GOOGL). AI POLICIES: Two Republican U.S. senators are calling for a congressional investigation into Meta after Reuters reported on an internal policy document that permitted the company's chatbots to 'engage a child in conversations that are romantic or sensual,' Jody Godoy reports. Meta confirmed the document's authenticity, but says it removed portions which stated it is permissible for chatbots to flirt and engage in romantic roleplay with children after receiving questions earlier this month from Reuters. INTEL STAKE: The Trump administration is considering using funds from the Chips Act to take a stake in Intel (INTC), Joe Deaux, Ryan Gould, Mackenzie Hawkins and Josh Wingrove of Bloomberg report, citing people familiar with the discussions. The government is in talks to use Chips Act funding to at least partially finance an equity stake in Intel, and it is unclear if this would involve converting Intel's existing Chips Act grants into equity or allocating new funding, sources told Bloomberg.
Yahoo
3 hours ago
- Yahoo
Murdoch Paper Warns Trump: You Just Got Hit With a Major ‘Political Problem'
Rupert Murdoch's Wall Street Journal just slapped Donald Trump with a stark warning of a looming 'political problem' over who will wind up paying his tariffs. In an editorial published Thursday, the paper's editorial board said fresh inflation data showed costs climbing for U.S. producers, undermining the president's claim that other countries would be paying for his import levies. Consumer-price data released Tuesday from the Bureau of Labor Statistics showed that the Consumer Price Index rose just 0.2 percent in July and 2.7 percent from a year earlier, suggesting households had yet to feel broad tariff-related price hikes. But the Producer Price Index (PPI) released Thursday showed wholesale prices climbing faster than economists anticipated, indicating higher consumer prices were on their way. PPI jumped 0.9 percent in July, marking the sharpest monthly rise in more than three years. It also climbed 3.3 percent year-over-year. The Journal said many U.S. companies have so far shielded consumers by absorbing those costs. 'This hasn't shown up in consumer prices so far because many companies entered the Trump tariff era with large cash reserves or wider margins, so they can absorb these costs for the time being,' the board said, adding: 'But these companies can't do this forever.' It warned that money diverted to tariffs or tariff-driven price hikes 'isn't available for reinvestment in the business, or to return to shareholders.' That reality, the board argued, 'poses a political problem for Mr. Trump and Republicans.' 'The President promises voters that foreigners will pay for his tariff policies. These price data caution that the economy may have other ideas,' the board said. Americans could end up paying 'a big chunk of the tariff bill,' either directly through higher prices or indirectly via slower wage growth. 'With the consumer price index running at an annualized rate not seen in over four years and PPI for core goods—which would be most sensitive to tariffs—now down at a 0.7 percent annualized rate since January, the doom-and-gloom predictions of President Trump's tariffs ratcheting up inflation continue to be proven flat-out wrong," White House Spokesman Kush Desai told the Daily Beast in a statement. 'The Trump administration's policies swiftly put an end to Joe Biden's inflation crisis, and as trillions in investments pour into the United States, they're now laying the groundwork for a long-term economic resurgence for the American people,' he added. Trump bragged in a post-election interview with NBC News in December that he 'won on groceries,' pointing to soaring food costs during President Joe Biden's term as pivotal to his victory. But the Journal's warning suggests Americans could soon be eyeing rising grocery bills under Trump, too. Real average hourly earnings rose just 0.1 percent in July after no change in June, and over the past 12 months real average hourly earnings increased only 1.2 percent, the board noted, citing BLS data. The editorial board concluded that if voters see rising costs without matching wage gains, Republicans would be in 'the political danger zone.' 'Republicans will make the same mistake as the Biden Administration if they keep telling voters everything is fabulous but the evidence at the grocery store or Applebee's tells them something different,' it said.
