'Tool for grifters': AI deepfakes push bogus sexual cures
The rise of generative AI has made it easy -- and financially lucrative -- to mass-produce such videos with minimal human oversight, often featuring fake celebrity endorsements of bogus and potentially harmful products.
In some TikTok videos, carrots are used as a euphemism for male genitalia, apparently to evade content moderation policing sexually explicit language.
"You would notice that your carrot has grown up," the muscled man says in a robotic voice in one video, directing users to an online purchase link.
"This product will change your life," the man adds, claiming without evidence that the herbs used as ingredients boost testosterone and send energy levels "through the roof."
The video appears to be AI-generated, according to a deepfake detection service recently launched by the Bay Area-headquartered firm Resemble AI, which shared its results with AFP.
"As seen in this example, misleading AI-generated content is being used to market supplements with exaggerated or unverified claims, potentially putting consumers' health at risk," Zohaib Ahmed, Resemble AI's chief executive and co-founder, told AFP.
"We're seeing AI-generated content weaponized to spread false information."
'Cheap way'
The trend underscores how rapid advances in artificial intelligence have fueled what researchers call an AI dystopia, a deception-filled online universe designed to manipulate unsuspecting users into buying dubious products.
They include everything from unverified -- and in some cases, potentially harmful -- dietary supplements to weight loss products and sexual remedies.
"AI is a useful tool for grifters looking to create large volumes of content slop for a low cost," misinformation researcher Abbie Richards told AFP.
"It's a cheap way to produce advertisements," she added.
Alexios Mantzarlis, director of the Security, Trust, and Safety Initiative at Cornell Tech, has observed a surge of "AI doctor" avatars and audio tracks on TikTok that promote questionable sexual remedies.
Some of these videos, many with millions of views, peddle testosterone-boosting concoctions made from ingredients such as lemon, ginger and garlic.
More troublingly, rapidly evolving AI tools have enabled the creation of deepfakes impersonating celebrities such as actress Amanda Seyfried and actor Robert De Niro.
"Your husband can't get it up?" Anthony Fauci, former director of the National Institute of Allergy and Infectious Diseases, appears to ask in a TikTok video promoting a prostate supplement.
But the clip is a deepfake, using Fauci's likeness.
'Pernicious'
Many manipulated videos are created from existing ones, modified with AI-generated voices and lip-synced to match what the altered voice says.
"The impersonation videos are particularly pernicious as they further degrade our ability to discern authentic accounts online," Mantzarlis said.
Last year, Mantzarlis discovered hundreds of ads on YouTube featuring deepfakes of celebrities -- including Arnold Schwarzenegger, Sylvester Stallone, and Mike Tyson -- promoting supplements branded as erectile dysfunction cures.
The rapid pace of generating short-form AI videos means that even when tech platforms remove questionable content, near-identical versions quickly reappear -- turning moderation into a game of whack-a-mole.
Researchers say this creates unique challenges for policing AI-generated content, requiring novel solutions and more sophisticated detection tools.
AFP's fact checkers have repeatedly debunked scam ads on Facebook promoting treatments -- including erectile dysfunction cures -- that use fake endorsements by Ben Carson, a neurosurgeon and former US cabinet member.
Yet many users still consider the endorsements legitimate, illustrating the appeal of deepfakes.
"Scammy affiliate marketing schemes and questionable sex supplements have existed for as long as the internet and before," Mantzarlis said.
"As with every other bad thing online, generative AI has made this abuse vector cheaper and quicker to deploy at scale."
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
France 24
an hour ago
- France 24
Dutch suggest social media ban for under-15s
The advice, which is non-binding, comes after Australia and New Zealand proposed social media bans for under-16s, and several European countries have issued similar guidelines. "Intensive screen and social media use can be bad for the (mental) health and development of children," said the Dutch ministry for health, wellbeing and sport. "Think of sleeping problems, panic attacks, depressive symptoms, reduced concentration and a negative self-image." The ministry distinguished between smartphone use, messaging apps such as WhatsApp and Signal, and social media apps like TikTok. Children younger than their last year of primary school (typically 11 or 12) should not be allowed a smartphone, the government advised. From secondary school (age 12 or 13), message apps should be permitted but no social media apps before 15, according to the guidelines. "A step-by-step approach helps: first learn to communicate via chat, then get acquainted with social media," said the government. The ministry also issued guidelines on screen time: none at all before the age of two, while children over 12 should not be in front of a screen for more than three hours. Healthy screen use is more than just time limits, the government said. "It's also about balancing screen time with other activities, using media together, and fostering positive online experiences." The advice brings the Netherlands into line with other age guidelines in Europe, said the government. Backed by France and Spain, Greece has spearheaded a proposal for how the European Union should limit children's use of online platforms. France, Greece and Denmark believe there should be a ban on social media for under-15s, while Spain has suggested a ban for under-16s. In the Netherlands, a children's advocacy group said earlier this month the "unchecked expansion" of social media platforms is driving an unprecedented global mental health crisis in kids and teens. The KidsRights report said what it termed "problematic" social media use was on the rise, with a direct link between heavy internet use and suicide attempts. However, blanket bans are not the answer, the group warned.
France 24
2 hours ago
- France 24
Gaza rescuers say Israel army kills more than 50 people near aid site
The Gaza Strip has been ravaged by more than 20 months of war between Israel and Palestinian militant group Hamas, with the situation continuing to deteriorate on the ground amid shortages of food, fuel and clean water. Civil defence spokesman Mahmud Bassal told AFP that at least 53 people were killed and some 200 wounded as thousands of Palestinians gathered to receive flour at a World Central Kitchen (WCK) aid centre in the morning. "Israeli drones fired at the citizens. Some minutes later, Israeli tanks fired several shells at the citizens, which led to a large number of martyrs and wounded," he said. The Israeli army said it was "aware of reports regarding a number of injured individuals from (Israeli military) fire following the crowd's approach" in Khan Yunis, and that the details of the incident were "under review". It said that "a gathering was identified adjacent to an aid distribution truck that got stuck in the area of Khan Yunis, and in proximity to (Israeli) troops operating in the area." Bassal said that four additional people were killed by Israeli fire Tuesday near the southern Gaza city of Rafah. Chaotic scenes Israeli restrictions on media in the Gaza Strip and other difficulties in accessing some areas mean AFP is unable to independently verify the tolls and details provided by the civil defence agency. The health ministry in Hamas-run Gaza reported that as a result of the incident, "51 martyrs and more than 200 injuries have arrived at Nasser Medical Complex, including 20 in critical condition". In early March, Israel imposed a total aid blockade on the Gaza Strip amid an impasse in truce negotiations, only partially easing restrictions in late May. The US- and Israel-backed Gaza Humanitarian Foundation (GHF) began distributing aid in late May, but its operations have been marred by chaotic scenes and dozens of deaths. The UN's humanitarian agency OCHA said Monday that during recent aid distributions several children have been "temporarily separated from their families due to mass movements around militarised distribution points." Workers raced to restore Al-Ahli hospital in Gaza City on Tuesday, one of the last remaining functioning health facilities in Gaza's north, an area particularly hard-hit by the war. They cleared piles of rubble out of the courtyard to make space for ambulances, breaking large chunks of concrete from a collapsed storey with sledgehammers. Amer Abu Safiya, a patient at the hospital who suffered from a wound on his hand, told AFP there was little doctors could do to help him. 'There's no medication' "Every day we are being bombed from the north to the south. Al-Ahli Hospital has been destroyed. Medical services are halted. As you can see, there's nothing to wrap around my hand, and there's no medication", he said, holding up his swollen hand while laying down on a makeshift bed in the hospital's backyard. "We are reactivating the emergency department as well as the physiotherapy. This is important", Alessandro Maracchi, head of the UN Development Program's Gaza's office, told AFP. OCHA further reported that its humanitarian partners in Gaza "continue to warn of the risk of famine in Gaza, amid catastrophic levels of acute food insecurity". The war was triggered by an unprecedented Hamas attack on Israel on October 7, 2023, which resulted in the deaths of 1,219 people, according to official Israeli figures. The health ministry in Hamas-run Gaza said on Monday that 5,194 people have been killed since Israel resumed strikes on the territory on March 18 following a truce. The overall death toll in Gaza since the war broke out on October 7, 2023 has reached 55,493 people, according to the health ministry.
France 24
4 hours ago
- France 24
Why stablecoins are gaining popularity
But as stablecoins move closer to the financial mainstream, experts warn that weak regulation could leave investors and the financial system vulnerable. - What are stablecoins?- Stablecoins play a key role in crypto markets, enabling users to trade digital assets without relying on traditional banks, instead using a decentralised register known as blockchain. Unlike volatile cryptocurrencies like bitcoin, stablecoins track the value of traditional assets -- such as the US dollar or gold -- enabling greater stability. Tether and USDC, for example, are pegged to the dollar, backed by reserves held by their issuing companies. They are useful internationally because they enable "fast, low-cost cross-border payments", Dessislava Aubert, analyst at crypto insights firm Kaiko, told AFP. This is "especially valuable in emerging markets where access to hard currency and traditional banking services is often limited", Aubert added. Examples include Argentina, Nigeria and Turkey, The market value of stablecoins soared to $246 billion in May, up from $20 billion in 2020, according to Deutsche Bank. And the total number of transactions in 2024 surpassed those of Visa and Mastercard. USDC's issuer, Circle, made a splash this month when it was listed on the New York Stock Exchange. Why is the US regulating them? To ensure the stability of stablecoins, the United States is pushing for issuers to hold sufficient low-risk, liquid assets -- such as dollars and Treasury bills. This could also boost demand for US debt and the greenback. The proposed legislation would require major stablecoin issuers to undergo regular audits and make it tougher to launch new tokens. These safeguards became more urgent after the collapse of the Terra stablecoin in 2022, which showed how these tokens can "depeg", or lose the link to the asset they are meant to track. There's a risk that an organisation may not be trustworthy or could be hacked, making audits and checks vital, explained Murat Kantarcioglu, a computer science professor at Virginia Tech University in the United States. Another possibility is that a loss of trust in the stablecoin may ripple beyond the crypto world, hitting the assets that back these tokens. Does the bill go far enough? "The new rules could make it harder for start-ups to issue stablecoins, creating a risk that a few big companies -- such as tech giants -- could dominate the market," Aubert told AFP over email. According to the Wall Street Journal, Amazon and Walmart are considering issuing their own stablecoin, which their customers could use for purchases. Democrats opposing the bill say risks of speculation, money laundering and political conflicts of interest associated with stablecoins are not sufficiently addressed. Notably, President Donald Trump's family has helped launch a stablecoin called USD1, used by Emirati fund MGX. And even with new regulations, in the event of issuer bankruptcy, stablecoin losses "are not explicitly covered by government insurance programs", unlike "bank deposits, which are insured up to $250,000", Aubert pointed out. How do other countries regulate them? In Europe, regulation on cryptocurrencies (MiCA), effective since the end of December, provides a framework for issuing stablecoins. The UK, South Korea and Brazil are moving forward with regulations. China banned cryptocurrencies in 2021 and is instead developing its own central bank digital currency, the e-yuan. Russia is considering a stablecoin backed by the rouble or friendly currencies such as the yuan. © 2025 AFP
