'Tool for grifters': AI deepfakes push bogus sexual cures
By Anuj Chopra and Marisha Goldhamer
Holding an oversized carrot, a brawny, shirtless man promotes a supplement he claims can enlarge male genitalia -- one of countless AI-generated videos on TikTok peddling unproven sexual treatments.
The rise of generative AI has made it easy -- and financially lucrative -- to mass-produce such videos with minimal human oversight, often featuring fake celebrity endorsements of bogus and potentially harmful products.
In some TikTok videos, carrots are used as a euphemism for male genitalia, apparently to evade content moderation policing sexually explicit language.
"You would notice that your carrot has grown up," the muscled man says in a robotic voice in one video, directing users to an online purchase link.
"This product will change your life," the man adds, claiming without evidence that the herbs used as ingredients boost testosterone and send energy levels "through the roof."
The video appears to be AI-generated, according to a deepfake detection service recently launched by the Bay Area-headquartered firm Resemble AI, which shared its results with AFP.
"As seen in this example, misleading AI-generated content is being used to market supplements with exaggerated or unverified claims, potentially putting consumers' health at risk," Zohaib Ahmed, Resemble AI's chief executive and co-founder, told AFP. "We're seeing AI-generated content weaponized to spread false information."
The trend underscores how rapid advances in artificial intelligence have fueled what researchers call an AI dystopia, a deception-filled online universe designed to manipulate unsuspecting users into buying dubious products.
They include everything from unverified -- and in some cases, potentially harmful -- dietary supplements to weight loss products and sexual remedies.
"AI is a useful tool for grifters looking to create large volumes of content slop for a low cost," misinformation researcher Abbie Richards told AFP.
"It's a cheap way to produce advertisements," she added.
Alexios Mantzarlis, director of the Security, Trust, and Safety Initiative at Cornell Tech, has observed a surge of "AI doctor" avatars and audio tracks on TikTok that promote questionable sexual remedies.
Some of these videos, many with millions of views, peddle testosterone-boosting concoctions made from ingredients such as lemon, ginger and garlic.
More troublingly, rapidly evolving AI tools have enabled the creation of deepfakes impersonating celebrities such as actress Amanda Seyfried and actor Robert De Niro.
"Your husband can't get it up?" Anthony Fauci, former director of the National Institute of Allergy and Infectious Diseases, appears to ask in a TikTok video promoting a prostate supplement.
But the clip is a deepfake, using Fauci's likeness.
Many manipulated videos are created from existing ones, modified with AI-generated voices and lip-synced to match what the altered voice says.
"The impersonation videos are particularly pernicious as they further degrade our ability to discern authentic accounts online," Mantzarlis said.
Last year, Mantzarlis discovered hundreds of ads on YouTube featuring deepfakes of celebrities -- including Arnold Schwarzenegger, Sylvester Stallone, and Mike Tyson -- promoting supplements branded as erectile dysfunction cures.
The rapid pace of generating short-form AI videos means that even when tech platforms remove questionable content, near-identical versions quickly reappear -- turning moderation into a game of whack-a-mole.
Researchers say this creates unique challenges for policing AI-generated content, requiring novel solutions and more sophisticated detection tools.
AFP's fact checkers have repeatedly debunked scam ads on Facebook promoting treatments -- including erectile dysfunction cures -- that use fake endorsements by Ben Carson, a neurosurgeon and former U.S. cabinet member.
Yet many users still consider the endorsements legitimate, illustrating the appeal of deepfakes.
"Scammy affiliate marketing schemes and questionable sex supplements have existed for as long as the internet and before," Mantzarlis said. "As with every other bad thing online, generative AI has made this abuse vector cheaper and quicker to deploy at scale."
© 2025 AFP
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Nikkei Asia
4 days ago
- Nikkei Asia
TikTok Shop in Japan and Danang's hub dreams
Hello everyone, this is Cissy in Hong Kong, your #techAsia host this week. In recent months, China's live commerce industry has continued to gain ground, starkly contrasting with global markets. In the first six months of the year, China's online retail sales exceeded 7.4 trillion yuan ($1.03 trillion), up 8.5%, partially fueled by live commerce and instant retail. A recent Cyberspace Administration report even highlighted live commerce as a "vital driver of consumption and export." Overseas, even foreign streamers are benefiting from China's robust and efficient supply chain, with many products shipped directly from China. For instance, Southeast Asian customers ordering via a TikTok live stream can receive their purchases in just five or six days. In China, cross-border e-commerce is booming, with 5,080 new companies registered in the segment in early 2025, almost doubling from the same time in 2024. Since the start of the year, TikTok Shop has expanded into Mexico, Brazil and Japan, leveraging its "short video plus live streaming" model to blend entertainment and shopping to lure global consumers. In contrast, the U.S. and Europe have been laggards in adopting live commerce. Japan also appears less than keen to jump on the bandwagon. Launched in Japan in late June, TikTok Shop described its performance there as "very impressive." But the company didn't disclose its first-month gross merchandise value (GMV), and other signs point to a less rosy picture. TikTok trepidation TikTok's live commerce feature launched in Japan a month ago amid high expectations that it would challenge e-commerce giants. But the foray appears to be faltering as many companies are reluctant to try a marketing medium that remains relatively untested in the country, writes Nikkei's Ryo Sato. Although TikTok users in Japan spend an average of 44 minutes a day on the app, double the figure for Instagram, that hasn't translated into sales. Early adopters included Unilever Japan, Ya-Man, Lacoste Japan and Nissin Foods, but while the service got off to a brisk start, the number of companies joining has since fallen off. The slow uptake stems from three key hurdles. First, companies doubt live commerce will catch on in Japan's cautious market. Second, hesitation persists due to TikTok's Chinese ownership, raising concerns about buyer trust. Third, the need for specialized talent and facilities to run live streams poses logistical challenges for businesses. Cost of doing business Donald Trump said he may allow Nvidia to sell a more advanced artificial intelligence chip in China, after confirming he had "negotiated a little deal" to give the U.S. a share of the chipmaker's revenues from the country, write the Financial Times' Demetri Sevastopulo and Michael Acton. The unprecedented deal the president struck with Nvidia's chief executive Jensen Huang, revealed by the FT, involves the U.S. taking 15% of Chinese sales of its H20 chip, which it introduced in 2023 to comply with Biden-era export controls. The performance of the H20 chips sold to China is restricted compared with those available to U.S. customers. Trump on Monday said that he planned to discuss a new deal with Huang to allow Nvidia to sell chips to China based on its latest, and most advanced, Blackwell platform. Approval would unlock billions of dollars in sales for Nvidia after lobbying by Huang for access to the Chinese market, despite security concerns in Washington. Race for the stars Rocket Lab, a New Zealand-founded, California-based aerospace company, is making waves in Asia, competing head-on with Japanese launch providers, as Japan's satellite operators increasingly rely on Rocket Lab's services to boost its regional presence, writes Nikkei Asia's Mitsuru Obe. Japanese players like Canon's Space One, Toyota-backed Interstellar Technologies and a Honda research unit, which tested vertical takeoff and landing in June, are emerging but lag in terms of global outreach as they traditionally focused on domestic needs. Rocket Lab's focus on cost and speed positions it to capture more of Asia's expanding space market. In 2024, Rocket Lab's primary launch site in New Zealand made it the world's fourth most-active launch market, surpassing Japan, which ranked fifth. The company's agility in seizing opportunities - building satellites, selling components, and launching low-cost missions to Mars and Venus - sets it apart. Its 18-meter Electron rocket, capable of carrying 300 kilograms, has become the second most-used launch vehicle globally after SpaceX's Falcon 9, with over 56 orbital launches since 2017. Thinking big Danang, renowned for its soft sand, clear waters, and iconic Golden Bridge, is looking to transform itself into Vietnam's finance, technology and trade hub. In an exclusive interview with Nikkei Asia's Yuji Nitta and Mai Nguyen, Luong Nguyen Minh Triet, chairman of the Danang People's Committee, outlined plans for an International Financial Center (IFC) to drive the city's growth and support Vietnam's goal of double-digit GDP expansion from 2026. The IFC, approved by Vietnam's top leaders alongside one in Ho Chi Minh City, will specialize in green finance, fintech, digital assets, and financial innovation. It will be headquartered in a 22-story building at Danang Software Park 2, with additional infrastructure planned on the Son Tra peninsula. To bolster the IFC, Danang is exploring artificial islands in Danang Bay to expand its financial ecosystem. This ambitious shift aims to attract global capital, positioning the city as a key player in Southeast Asia's economic landscape. China hits the gas on automotive chips Welcome to the Tech Latest podcast. Hosted by our tech coverage veterans, Katey Creel and Shotaro Tani, every Tuesday we deliver the hottest trends and news from the sector. In this episode, our host Katey speaks with Hong Kong correspondent Cissy Zhou about how Chinese automakers are ramping up their push to live without Nvidia chips. Find us on Apple Podcasts | Spotify | Amazon Music | Voicy | YouTube | YouTube Music Suggested reads 1. Tencent says it has ample chips for AI training amid reported purchase halt (Nikkei Asia) 2. Beijing puts pressure on Chinese tech giants over purchases of Nvidia's H20 chips (FT) 3. Applied Materials sued by China rival over alleged trade secrets theft (Nikkei Asia) 4. South Korea's Upstage enters global AI race (FT) 5. SMIC says more Chinese chip players 'perfectly' replacing foreign rivals (Nikkei Asia) 6. Indosat opens AI center in Papua in symbol of 'digital awakening' (Nikkei Asia) 7. Manus and Benchmark: the AI deal that upset China and the US (FT) 8. Synopsys says new era dawning in chip design due to AI, robotics demand (Nikkei Asia) 9. India's IT services groups race to reinvent themselves for AI age (FT) 10. China wants US to relax export controls on chips as part of trade deal (FT)
Japan Today
5 days ago
- Japan Today
Eyeing robotaxis, Tesla hiring New York test car operator
Tesla is eyeing robotaxi operations in New York and other American cities Tesla is recruiting a motorist to test its driver-assistance technology in New York with an eye towards autonomous driving, according to a job listing reviewed by AFP. The role of "Vehicle Operator, Autopilot" involves the driving of an "engineering vehicle" for extended periods, "conducting dynamic audio and camera data collection for testing and training purpose," according to the job listing. The position is based in Flushing, New York in the borough of Queens. The functions described in the full-time position are still many steps away from providing autonomous or robotaxi service in New York City. New York State law currently limits the use of autonomous cars to testing. Waymo in June said it applied for a permit to begin testing self-driving cars. Billionaire CEO Elon Musk has described Tesla as poised for potential rapid deployment of autonomous vehicles, emphasizing the company's use of artificial intelligence to analyze real-world data that has been gathered by the company's existing fleet of vehicles. Tesla in June finally launched limited robotaxi service in Austin, Texas following many delays. On a conference call in July, Musk predicted Tesla would "probably have autonomous ride-hailing in probably half the population of the U.S. by the end of the year" -- an ambitious target that looks highly unlikely. Musk acknowledged that the rollout depends on regulatory approvals, adding that the company is being "very cautious" in light of safety concerns. © 2025 AFP
Japan Today
5 days ago
- Japan Today
Tainted fentanyl blamed for 87 hospital deaths in Argentina
Relatives of victims of fentanyl contamination have staged protests to demand justice in Argentina At least 87 people have died in Argentina after being administered bacteria-infected fentanyl in hospitals, a judge investigating the deaths said Wednesday. Since May, a court has been trying to determine how many deaths are linked to batches of the drug contaminated with the bacteria Klebsiella pneumoniae and Ralstonia pickettii. The suspected toll has soared since then, with family members holding protests outside hospitals demanding justice. The probe arose from a complaint filed by Argentina's drug regulatory agency, Anmat, which had received a report from a hospital that discovered the tainted drug in its supply, an Anmat employee told AFP on condition of anonymity. The deaths have occurred in hospitals in the city and province of Buenos Aires, and in three other provinces, the judge leading the investigation, Ernesto Kreplak, told The Nation daily newspaper in an article published Wednesday. At least 24 people have been questioned and had their assets frozen by the court. They include Ariel Furfaro Garcia, owner of HLB Pharma and Ramallo Laboratories, which allegedly produced and sold the contaminated drug, The Nation reported. He denied the claims and blamed a former colleague for having planted the story in the media. "This was fabricated by the media. All the clinical histories show that the patients had other, more dangerous bacteria, people with serious problems," Furfaro Garcia told the daily newspaper Clarin. According to The Nation, at least five contaminated batches were distributed to eight hospitals and health centers in the country, although the investigation is looking at clinical histories from 200 hospitals. Two weeks ago, relatives of victims demonstrated outside the Italian Hospital in La Plata, 60 kilometers south of Buenos Aires, where the first deaths were reported, demanding "justice for the fentanyl victims." "The fentanyl caused his death in a matter of days," Alejandro Ayala, whose brother Leonel died at the age of 32, told AFP. Experts have warned that the death toll could rise as new medical records are reviewed and cases are confirmed in hospitals that to date had not reported any fentanyl-linked deaths. Argentina's Congress on Wednesday submitted 26 questions related to the case to the Executive branch of the government, but no deadline has been set for their response. © 2025 AFP
