Latest news with #AlexiosMantzarlis
Yahoo
2 days ago
- Yahoo
Meta sues AI ‘nudify' app Crush AI for advertising on its platforms
Meta has sued the maker of a popular AI 'nudify' app, Crush AI, that reportedly ran thousands of ads across Meta's platforms. In addition to the lawsuit, Meta says it's taking new measures to crack down on other apps like Crush AI. In a lawsuit filed in Hong Kong, Meta alleged Joy Timeline HK, the entity behind Crush AI, attempted to circumvent the company's review process to distribute ads for AI nudify services. Meta said in a blog post that it repeatedly removed ads by the entity for violating its policies, but claims Joy Timeline HK continued to place additional ads anyway. Crush AI, which uses generative AI to make fake, sexually explicit images of real people without their consent, reportedly ran more than 8,000 ads for its 'AI undresser' services on Meta's platform in the first two weeks of 2025, according to the author of the Faked Up newsletter, Alexios Mantzarlis. In a January report, Mantzarlis claimed that Crush AI's websites received roughly 90% of their traffic from either Facebook or Instagram, and that he flagged several of these websites to Meta. Crush AI reportedly evaded Meta's ad review processes by setting up dozens of advertiser accounts and frequently changed domain names. Many of Crush AI's advertiser accounts, according to Mantzarlis, were named 'Eraser Annyone's Clothes' followed by different numbers. At one point, Crush AI even had a Facebook page promoting its service. Facebook and Instagram are hardly the only platforms dealing with such challenges. As social media companies like X and Meta race to add generative AI to their apps, they've also struggled to moderate how AI tools can make their platforms unsafe for users, particularly minors. Researchers have found that links to AI undressing apps soared in 2024 on platforms like X and Reddit, and on YouTube, millions of people were reportedly served ads for such apps. In response to this growing problem, Meta and TikTok have banned keyword searches for AI nudify apps, but getting these services off their platforms entirely has proven challenging. In a blog post, Meta said it has developed new technology to specifically identify ads for AI nudify or undressing services 'even when the ads themselves don't include nudity.' The company said it is now using matching technology to help find and remove copycat ads more quickly, and has expanded the list of terms, phrases and emoji that are flagged by its systems. Meta said it is also applying the tactics it has traditionally used to disrupt networks of bad actors to these new networks of accounts running ads for AI nudify services. Since the start of 2025, Meta said, it has disrupted four separate networks promoting these services. Outside of its apps, the company said it will begin sharing information about AI nudify apps through Tech Coalition's Lantern program, a collective effort between Google, Meta, Snap and other companies to prevent child sexual exploitation online. Meta says it has provided more than 3,800 unique URLs with this network since March. On the legislative front, Meta said it would 'continue to support legislation that empowers parents to oversee and approve their teens' app downloads.' The company previously supported the US Take It Down Act, and said it's now working with lawmakers to implement it. Sign in to access your portfolio
TechCrunch
2 days ago
- TechCrunch
Meta sues AI ‘nudify' app Crush AI for advertising on its platforms
Meta has sued the maker of a popular AI 'nudify' app, Crush AI, that reportedly ran thousands of ads across Meta's platforms. In addition to the lawsuit, Meta says it's taking new measures to crack down on other apps like Crush AI. In a lawsuit filed in Hong Kong, Meta alleged Joy Timeline HK, the entity behind Crush AI, attempted to circumvent the company's review process to distribute ads for AI nudify services. Meta said in a blog post that it repeatedly removed ads by the entity for violating its policies, but claims Joy Timeline HK continued to place additional ads anyway. Crush AI, which uses generative AI to make fake, sexually explicit images of real people without their consent, reportedly ran more than 8,000 ads for its 'AI undresser' services on Meta's platform in the first two weeks of 2025, according to the author of the Faked Up newsletter, Alexios Mantzarlis. In a January report, Mantzarlis claimed that Crush AI's websites received roughly 90% of their traffic from either Facebook or Instagram, and that he flagged several of these websites to Meta. Crush AI reportedly evaded Meta's ad review processes by setting up dozens of advertiser accounts and frequently changed domain names. Many of Crush AI's advertiser accounts, according to Mantzarlis, were named 'Eraser Annyone's Clothes' followed by different numbers. At one point, Crush AI even had a Facebook page promoting its service. Facebook and Instagram are hardly the only platforms dealing with such challenges. As social media companies like X and Meta race to add generative AI to their apps, they've also struggled to moderate how AI tools can make their platforms unsafe for users, particularly minors. Researchers have found that links to AI undressing apps soared in 2024 on platforms like X and Reddit, and on YouTube, millions of people were reportedly served ads for such apps. In response to this growing problem, Meta and TikTok have banned keyword searches for AI nudify apps, but getting these services off their platforms entirely has proven challenging. In a blog post, Meta said it has developed new technology to specifically identify ads for AI nudify or undressing services 'even when the ads themselves don't include nudity.' The company said it is now using matching technology to help find and remove copycat ads more quickly, and has expanded the list of terms, phrases and emoji that are flagged by its systems. Techcrunch event Save $200+ on your TechCrunch All Stage pass Build smarter. Scale faster. Connect deeper. Join visionaries from Precursor Ventures, NEA, Index Ventures, Underscore VC, and beyond for a day packed with strategies, workshops, and meaningful connections. Save $200+ on your TechCrunch All Stage pass Build smarter. Scale faster. Connect deeper. Join visionaries from Precursor Ventures, NEA, Index Ventures, Underscore VC, and beyond for a day packed with strategies, workshops, and meaningful connections. Boston, MA | REGISTER NOW Meta said it is also applying the tactics it has traditionally used to disrupt networks of bad actors to these new networks of accounts running ads for AI nudify services. Since the start of 2025, Meta said, it has disrupted four separate networks promoting these services. Outside of its apps, the company said it will begin sharing information about AI nudify apps through Tech Coalition's Lantern program, a collective effort between Google, Meta, Snap and other companies to prevent child sexual exploitation online. Meta says it has provided more than 3,800 unique URLs with this network since March. On the legislative front, Meta said it would 'continue to support legislation that empowers parents to oversee and approve their teens' app downloads.' The company previously supported the US Take It Down Act, and said it's now working with lawmakers to implement it.
Engadget
2 days ago
- Business
- Engadget
Meta is cracking down on AI 'nudify' apps
Meta is finally cracking down on "nudify" apps that use AI to generate nonconsensual nude and explicit images of celebrities, influencers and others. The company is suing one app maker that's frequently advertised such apps on Facebook and Instagram, and taking new steps to prevent ads for similar services. The crackdown comes months after several researchers and journalists have raised the alarm about such apps. A recent report from CBS News identified at least "hundreds" of ads on Meta's platform promoting apps that allow users to "remove clothing" from images of celebrities and others. One app in particular, called Crush AI, has apparently been a prolific advertiser on Facebook and Instagram. Researcher Alexios Mantzarlis, Director of Cornell Tech's Security, Trust and Safety Initiative, reported back in January that Crush AI had run more than 8,000 ads on Facebook and Instagram since last fall. Now, Meta says it has filed a lawsuit against Joy Timeline HK Limited, the Hong Kong-based company behind Crush AI and other nudify apps. "This follows multiple attempts by Joy Timeline HK Limited to circumvent Meta's ad review process and continue placing these ads, after they were repeatedly removed for breaking our rules," the company wrote in a blog post. Joy Timeline HK Limited didn't immediately respond to a request for comment. Meta also says it's taking new steps to prevent apps like these from advertising on its platform. "We've developed new technology specifically designed to identify these types of ads — even when the ads themselves don't include nudity — and use matching technology to help us find and remove copycat ads more quickly," Meta wrote. "We've worked with external experts and our own specialist teams to expand the list of safety-related terms, phrases and emojis that our systems are trained to detect within these ads." The social network says it also plans to work with other tech platforms, including app store owners, to share relevant details about entities that abuse its platform. Nudify apps aren't the only entities that have exploited Meta's advertising platform to run ads featuring celebrity deepfakes. Meta has also struggled to contain shady advertisers that use AI-manipulated video of public figures to promote scams . The company's independent Oversight Board, which weighs in on content moderation issues affecting Facebook and Instagram, recently criticized Meta for under-enforcing its rules prohibiting such ads.
Yahoo
27-03-2025
- Politics
- Yahoo
How a TikTok network spreading Spanish-language immigration misinformation gained traction
A rumor falsely claiming green card holders are banned from leaving and re-entering the U.S. spread like wildfire on TikTok this week, garnering more than 21 million views across dozens of videos shared by a network of nearly 40 accounts, many posing as Spanish-language news outlets to share immigration-related misinformation. The accounts have been sharing doctored videos that contain artificial intelligence-generated content and the voices of well-known professional journalists to spread misinformation about immigration and other divisive topics that tend to go viral on social media, according to Alexios Mantzarlis, director of the Security, Trust, and Safety Initiative at Cornell Tech, Cornell University's graduate campus in New York City. Most of the TikTok accounts are named with generic terms that include the words "noticias" (news) or "noticiero" (newscast), and eight of them falsely use the logos of two major Spanish-language news networks, Univision and Telemundo. (Telemundo and NBC News are owned by NBCUniversal, a subsidiary of Comcast Corp.) This newest Spanish-language disinformation network "hijacks both the kind of viral mechanics of the TikTok algorithm, plus the very human peer-to-peer nature of immigrant communities," Mantzarlis, who also runs Faked Up, a newsletter about digital disinformation, told NBC News. Mantzarlis first started tracking the network of accounts last month after he saw a series of posts falsely claiming supermarkets would start requiring identification to buy groceries. In his newsletter, he said, some of the videos included messaging that pointed to "a clear effort to target undocumented immigrants." He noticed the accounts mainly focused on posting about President Donald Trump and immigration. This week, at least 25 videos shared by the network of TikTok accounts spreading false information about green card holders impersonated the voice of Univision reporter Javier Díaz. Díaz did not respond to a request for comment and Univision did not comment. The 25 videos were deleted from TikTok after NBC News sent links for the videos when requesting comment from the social media platform on Tuesday. NBC News also sent the company links for the 38 TikTok accounts Mantzarlis found were part of the network. By Wednesday afternoon, all 38 TikTok accounts were taken down. In a statement to NBC News, a TikTok spokesperson said, "We continue to protect our Spanish-speaking community from misleading content and accounts by investing in moderation teams and technologies, partnering with fact-checkers, and building media literacy tips and tools." According to the spokesperson, TikTok took down the videos and accounts because they violated the company's integrity and authenticity policies that prohibit harmful misinformation, misleading AI-generated content, impersonation and deceptive behaviors. A false narrative shared by one of the accounts in the network last month used the voice of former Telemundo anchor María Celeste Arrarás to make it look like she was reporting on Trump considering a plan to provide a legal pathway for undocumented immigrants without a criminal record. Arrarás posted the clip on her Instagram account to let her followers know the news in the video "IS FAKE." "They've edited several reports I presented on television years ago to mislead the public. IT'S AN AUDIO CUT to make it seem like I'm reporting news that ISN'T REAL. Please spread the word," she wrote in Spanish. As of Tuesday, that video featuring Arrarás' voice appeared to have already been deleted from the TikTok account. But the account remained functional until Wednesday morning along with several other videos containing misinformation, including a new one published the day before using Arrarás' voice. Before TikTok shut down the accounts Wednesday afternoon, the videos that included misinformation about green card holders and legal pathways for undocumented immigrants had reached the feed of an immigrant woman in New York City whom Amelia Scdoris, a community organizer at Cabrini Immigrant Services, had been helping. It's an example, Scdoris said, of why her organization holds regular meetings to help immigrants parse through misinformation they encounter and serve as a trustworthy source of immigration-related information. According to the TikTok spokesperson, the platform relies on automated technology, user reports, proactive searches, trend reports from experts and fact-checkers to detect misinformation on the platform across different languages. During the third quarter of 2024, over 97% of the content taken down from TikTok for violating its integrity and authenticity policies "was removed proactively before someone reported it to us." They added that TikTok has thousands of content moderators monitoring misinformation globally but declined to say exactly how many. As an example, TikTok's operation in the European Union region had 531 Spanish-language content moderators and 1,524 English-language content moderators as of December 2024, according to the company's most recent digital transparency report that it's required to file to operate in that region. Despite these efforts, combating online misinformation remains a daunting task. Latinos, including Spanish-speaking immigrant communities, are more likely to rely on social media to stay informed than other groups, according to a Nielsen report. TikTok is particularly popular among Latino communities in the U.S., with roughly half of Hispanic adults reporting they use the platform. Luba Cortés, the civil rights and lead immigration organizer at the advocacy group Make the Road New York, said she's seen firsthand how the news consumption habits of Spanish-speaking immigrant communities make them more vulnerable to the kind of misinformation spread on social media. "They see it and they believe that it's true," Cortés told NBC News about the AI-generated content and misinformation people are seeing online. "That causes a lot of mass panic." What's more, she said, the panic motivates viewers to share the false videos within their networks, since they mistakenly think the TikToks contain helpful information. Immigration-related falsehoods shared online often gain more traction when they include 'misleading cuts of evidence' connected to 'a real crisis,' according to Mert Bayar, who researches rumors about immigrants and noncitizens as a postdoctoral scholar at the University of Washington's Center for an Informed Public. That seems to be the case with the storyline falsely claiming green card holders, or lawful permanent residents, can't leave and re-enter the U.S. It's been circulating after news stories about green card holders now fighting deportation after participating in pro-Palestinian campus protests and traveling abroad. Last week, the National Korean American Service and Education Consortium, an immigration advocacy group, said in an Instagram post that some green card holders have unknowingly signed an I-407 form relinquishing their lawful permanent status. Cortés said she has not come across a case like that in her organization, but 'I do think, now, there are concerns.' Because the network of TikTok accounts sharing videos with false information both "denounce Trump's immigration policy and others that say he's about to save everyone," Mantzarlis said it's unclear if the operation is politically motivated or if it's laying the foundation to identify potential victims for future financial scams. As part of his previous research, Mantzarlis has seen how users who interact with this kind of content later become targets for monetary scams. "The only thing we can say for certain is that they're just trying to build a public, build a large audience," he said. "Afterwards, what they choose to do with that is kind of up to them." This article was originally published on
NBC News
26-03-2025
- Politics
- NBC News
How a TikTok network spreading Spanish-language immigration misinformation gained traction
A rumor falsely claiming green card holders are banned from leaving and re-entering the U.S. spread like wildfire on TikTok this week, garnering more than 21 million views across dozens of videos shared by a network of nearly 40 accounts, many posing as Spanish-language news outlets to share immigration-related misinformation. The accounts have been sharing doctored videos that contain artificial intelligence-generated content and the voices of well-known professional journalists to spread misinformation about immigration and other divisive topics that tend to go viral on social media, according to Alexios Mantzarlis, director of the Security, Trust, and Safety Initiative at Cornell Tech, Cornell University's graduate campus in New York City. Most of the TikTok accounts are named with generic terms that include the words "noticias" (news) or "noticiero" (newscast), and eight of them falsely use the logos of two major Spanish-language news networks, Univision and Telemundo. (Telemundo and NBC News are owned by NBCUniversal, a subsidiary of Comcast Corp.) This newest Spanish-language disinformation network "hijacks both the kind of viral mechanics of the TikTok algorithm, plus the very human peer-to-peer nature of immigrant communities," Mantzarlis, who also runs Faked Up, a newsletter about digital disinformation, told NBC News. Mantzarlis first started tracking the network of accounts last month after he saw a series of posts falsely claiming supermarkets would start requiring identification to buy groceries. In his newsletter, he said, some of the videos included messaging that pointed to " a clear effort to target undocumented immigrants." He noticed the accounts mainly focused on posting about President Donald Trump and immigration. This week, at least 25 videos shared by the network of TikTok accounts spreading false information about green card holders impersonated the voice of Univision reporter Javier Díaz. Díaz did not respond to a request for comment and Univision did not comment. The 25 videos were deleted from TikTok after NBC News sent links for the videos when requesting comment from the social media platform on Tuesday. NBC News also sent the company links for the 38 TikTok accounts Mantzarlis found were part of the network. By Wednesday afternoon, all 38 TikTok accounts were taken down. In a statement to NBC News, a TikTok spokesperson said, "We continue to protect our Spanish-speaking community from misleading content and accounts by investing in moderation teams and technologies, partnering with fact-checkers, and building media literacy tips and tools." According to the spokesperson, TikTok took down the videos and accounts because they violated the company's integrity and authenticity policies that prohibit harmful misinformation, misleading AI-generated content, impersonation and deceptive behaviors. A fake message — and messenger A false narrative shared by one of the accounts in the network last month used the voice of former Telemundo anchor María Celeste Arrarás to make it look like she was reporting on Trump considering a plan to provide a legal pathway for undocumented immigrants without a criminal record. Arrarás posted the clip on her Instagram account to let her followers know the news in the video " IS FAKE." "They've edited several reports I presented on television years ago to mislead the public. IT'S AN AUDIO CUT to make it seem like I'm reporting news that ISN'T REAL. Please spread the word," she wrote in Spanish. As of Tuesday, that video featuring Arrarás' voice appeared to have already been deleted from the TikTok account. But the account remained functional until Wednesday morning along with several other videos containing misinformation, including a new one published the day before using Arrarás' voice. Before TikTok shut down the accounts Wednesday afternoon, the videos that included misinformation about green card holders and legal pathways for undocumented immigrants had reached the feed of an immigrant woman in New York City whom Amelia Scdoris, a community organizer at Cabrini Immigrant Services, had been helping. It's an example, Scdoris said, of why her organization holds regular meetings to help immigrants parse through misinformation they encounter and serve as a trustworthy source of immigration-related information. According to the TikTok spokesperson, the platform relies on automated technology, user reports, proactive searches, trend reports from experts and fact-checkers to detect misinformation on the platform across different languages. During the third quarter of 2024, over 97% of the content taken down from TikTok for violating its integrity and authenticity policies "was removed proactively before someone reported it to us." They added that TikTok has thousands of content moderators monitoring misinformation globally but declined to say exactly how many. As an example, TikTok's operation in the European Union region had 531 Spanish-language content moderators and 1,524 English-language content moderators as of December 2024, according to the company's most recent digital transparency report that it's required to file to operate in that region. Despite these efforts, combating online misinformation remains a daunting task. Latinos, including Spanish-speaking immigrant communities, are more likely to rely on social media to stay informed than other groups, according to a Nielsen report. TikTok is particularly popular among Latino communities in the U.S., with roughly half of Hispanic adults reporting they use the platform. Luba Cortés, the civil rights and lead immigration organizer at the advocacy group Make the Road New York, said she's seen firsthand how the news consumption habits of Spanish-speaking immigrant communities make them more vulnerable to the kind of misinformation spread on social media. "They see it and they believe that it's true," Cortés told NBC News about the AI-generated content and misinformation people are seeing online. "That causes a lot of mass panic." What's more, she said, the panic motivates viewers to share the false videos within their networks, since they mistakenly think the TikToks contain helpful information. Immigration-related falsehoods shared online often gain more traction when they include 'misleading cuts of evidence' connected to 'a real crisis,' according to Mert Bayar, who researches rumors about immigrants and noncitizens as a postdoctoral scholar at the University of Washington's Center for an Informed Public. That seems to be the case with the storyline falsely claiming green card holders, or lawful permanent residents, can't leave and re-enter the U.S. It's been circulating after now fighting deportation after participating in pro-Palestinian campus protests and traveling abroad. Last week, the National Korean American Service and Education Consortium, an immigration advocacy group, said in an Instagram post that some green card holders have unknowingly signed an I-407 form relinquishing their lawful permanent status. Cortés said she has not come across a case like that in her organization, but 'I do think, now, there are concerns.' Politically motivated or a potential financial scam? Because the network of TikTok accounts sharing videos with false information both "denounce Trump's immigration policy and others that say he's about to save everyone," Mantzarlis said it's unclear if the operation is politically motivated or if it's laying the foundation to identify potential victims for future financial scams. As part of his previous research, Mantzarlis has seen how users who interact with this kind of content later become targets for monetary scams. "The only thing we can say for certain is that they're just trying to build a public, build a large audience," he said. "Afterwards, what they choose to do with that is kind of up to them."
