AI deepfakes are pushing bogus sexual cures
Countless AI-generated videos are peddling unproven sexual treatments. (Envato Elements pic)
WASHINGTON : Holding an oversized carrot, a brawny, shirtless man promotes a supplement he claims can enlarge male genitalia – one of countless AI-generated videos on TikTok peddling unproven sexual treatments.
The rise of generative AI has made it easy – and financially lucrative – to mass-produce such videos with minimal human oversight, often featuring fake celebrity endorsements of bogus and potentially harmful products.
In some TikTok videos, carrots are used as a euphemism for male genitalia, apparently to evade content moderation policing sexually explicit language.
'You would notice that your carrot has grown up,' the muscled man says in a robotic voice in one video, directing users to an online purchase link.
'This product will change your life,' the man adds, claiming without evidence that the herbs used as ingredients boost testosterone and send energy levels 'through the roof.'
The video appears to be AI-generated, according to a deepfake detection service recently launched by the Bay Area-headquartered firm Resemble AI, which shared its results with AFP.
'As seen in this example, misleading AI-generated content is being used to market supplements with exaggerated or unverified claims, potentially putting consumers' health at risk,' Zohaib Ahmed, Resemble AI's chief executive and co-founder, told AFP.
'We're seeing AI-generated content weaponised to spread false information.'
'Cheap way'
The trend underscores how rapid advances in artificial intelligence have fuelled what researchers call an AI dystopia, a deception-filled online universe designed to manipulate unsuspecting users into buying dubious products.
They include everything from unverified – and in some cases, potentially harmful – dietary supplements to weight loss products and sexual remedies.
'AI is a useful tool for grifters looking to create large volumes of content slop for a low cost,' misinformation researcher Abbie Richards told AFP.
'It's a cheap way to produce advertisements,' she added.
Alexios Mantzarlis, director of the Security, Trust, and Safety Initiative at Cornell Tech, has observed a surge of 'AI doctor' avatars and audio tracks on TikTok that promote questionable sexual remedies.
Some of these videos, many with millions of views, peddle testosterone-boosting concoctions made from ingredients such as lemon, ginger and garlic.
More troublingly, rapidly evolving AI tools have enabled the creation of deepfakes impersonating celebrities such as actress Amanda Seyfried and actor Robert De Niro.
'Your husband can't get it up?' Anthony Fauci, former director of the National Institute of Allergy and Infectious Diseases, appears to ask in a TikTok video promoting a prostate supplement.
But the clip is a deepfake, using Fauci's likeness.
'Pernicious'
Many manipulated videos are created from existing ones, modified with AI-generated voices and lip-synced to match what the altered voice says.
'The impersonation videos are particularly pernicious as they further degrade our ability to discern authentic accounts online,' Mantzarlis said.
Last year, Mantzarlis discovered hundreds of ads on YouTube featuring deepfakes of celebrities – including Arnold Schwarzenegger, Sylvester Stallone, and Mike Tyson – promoting supplements branded as erectile dysfunction cures.
The rapid pace of generating short-form AI videos means that even when tech platforms remove questionable content, near-identical versions quickly reappear – turning moderation into a game of whack-a-mole.
Researchers say this creates unique challenges for policing AI-generated content, requiring novel solutions and more sophisticated detection tools.
AFP's fact checkers have repeatedly debunked scam ads on Facebook promoting treatments – including erectile dysfunction cures – that use fake endorsements by Ben Carson, a neurosurgeon and former US cabinet member.
Yet many users still consider the endorsements legitimate, illustrating the appeal of deepfakes.
'Scammy affiliate marketing schemes and questionable sex supplements have existed for as long as the internet and before,' Mantzarlis said.
'As with every other bad thing online, generative AI has made this abuse vector cheaper and quicker to deploy at scale.'
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Malay Mail
2 hours ago
- Malay Mail
AI won't kill all jobs but it's hiring faster than you think
PARIS, June 15 — Predictions of imminent AI-driven mass unemployment are likely overblown, but employers will seek workers with different skills as the technology matures, a top executive at global recruiter ManpowerGroup told AFP at Paris's Vivatech trade fair. The world's third-largest staffing firm by revenue ran a startup contest at Vivatech in which one of the contenders was building systems to hire out customisable autonomous AI 'agents', rather than humans. Their service was reminiscent of a warning last month from Dario Amodei, head of American AI giant Anthropic, that the technology could wipe out half of entry-level white-collar jobs within one to five years. For ManpowerGroup, AI agents are 'certainly not going to become our core business any time soon,' the company's Chief Innovation Officer Tomas Chamorro-Premuzic said. 'If history shows us one thing, it's most of these forecasts are wrong.' An International Labour Organisation (ILO) report published in May found that around 'one in four workers across the world are in an occupation with some degree of exposure' to generative AI models' capabilities. 'Few jobs are currently at high risk of full automation,' the ILO added. But the UN body also highlighted 'rapid expansion of AI capabilities since our previous study' in 2023, including the emergence of 'agentic' models more able to act autonomously or semi-autonomously and use software like web browsers and email. 'Soft skills' Chamorro-Premuzic predicted that the introduction of efficiency-enhancing AI tools would put pressure on workers, managers and firms to make the most of the time they will save. 'If what happens is that AI helps knowledge workers save 30, 40, maybe 50 per cent of their time, but that time is then wasted on social media, that's not an increase in net output,' he said. Adoption of AI could give workers 'more time to do creative work' — or impose 'greater standardisation of their roles and reduced autonomy,' the ILO said. There's general agreement that interpersonal skills and an entrepreneurial attitude will become more important for knowledge workers as their daily tasks shift towards corralling AIs. Employers identified ethical judgement, customer service, team management and strategic thinking as top skills AI could not replace in a ManpowerGroup survey of over 40,000 employers across 42 countries published this week. Nevertheless, training that adopts those new priorities has not increased in step with AI adoption, Chamorro-Premuzic lamented. 'For every dollar you invest in technology, you need to invest eight or nine on HR, culture transformation, change management,' he said. He argued that such gaps suggest companies are still chasing automation, rather than the often-stated aim of augmenting human workers' capabilities with AI. AI hiring AI? One of the areas where AI is transforming the world of work most rapidly is ManpowerGroup's core business of recruitment. But here candidates are adopting the tools just as quickly as recruiters and companies, disrupting the old way of doing things from the bottom up. 'Candidates are able to send 500 perfect applications in one day, they are able to send their bots to interview, they are even able to game elements of the assessments,' Chamorro-Premuzic said. That extreme picture was not borne out in a survey of over 1,000 job seekers released this week by recruitment platform TestGorilla, which found just 17 per cent of applicants admitting to cheating on tests, and only some of those to using AI. Jobseekers' use of consumer AI tools meets recruiters doing the same. The same TestGorilla survey found almost two-thirds of the more-than-1,000 hiring decision-makers polled used AI to generate job descriptions and screen applications. But a far smaller share are already using the technology to actually interview candidates. Where employers today are focused on candidates' skills over credentials, Chamorro-Premuzic predicted that 'the next evolution is to focus on potential, not even skills even if I know the skills you bring to the table today, they might be obsolete in six months.' 'I'm better off knowing that you're hard-working, that you are curious, that you have good people skills, that you're not a jerk — and that, AI can help you evaluate,' he believes. — AFP
Free Malaysia Today
11 hours ago
- Free Malaysia Today
Empower AI to gain competitive edge, media industry told
FMT's head of Bahasa Malaysia and video unit, Zahrul Alam Yahya Shahir, said the media should not fear AI as it has long survived alongside it. (Bernama pic) KUALA LUMPUR : The media industry has been urged to empower artificial intelligence (AI) by actively shaping its development and use to gain a competitive advantage. FMT Sdn Bhd's head of Bahasa Malaysia and video unit, Zahrul Alam Yahya Shahir, said the media plays a crucial role in providing the data that AI systems learn from. Speaking at the National Journalists' Day (Hawana) 2025 Media Forum today, he noted that AI performs best when trained on structured content, as the technology prefers content that clearly outlines the issue and explains the problem. 'The content that AI gives us comes from us. That's why I say it's time for us to empower AI with our content… the analytic content that we put into social media and what not. 'Don't fear AI; we (the media) have long survived alongside it,' he said at the forum titled 'AI & Media: Tool, Threat or Opportunity?' He said that media practitioners should be the ones empowering AI, not the other way around. Another panellist, Astro Awani senior editor Nazri Kahar, emphasised that technologies such as AI should not be seen as a threat but an opportunity that the media industry must embrace. He said journalism is at a critical crossroads, demanding that media practitioners quickly adapt to technological advancements. 'This is a crucial moment for us to seize the opportunity. We should not view technologies as enemies that will erase the media industry. Instead, we must set boundaries, adapt and move forward,' he said. Nazri recounted a personal experience, going live for the Sibu by-election in 2010 using only an iPhone instead of expensive conventional broadcast equipment such as satellite systems. 'It was a new approach at that time. We used Skype (a telecommunication application to make voice and video calls and send instant messages), and the results were impressive,' he said, adding that the experience revealed the potential of technology to many people in journalism. Elaborating, he said that media organisations should not be bound by traditional approaches, focusing on concerns over audio and visual quality; instead, they should prioritise news value and timeliness in delivering information.
Free Malaysia Today
15 hours ago
- Free Malaysia Today
Govt allocates RM30mil to help media adapt to AI, digital shift
Prime Minister Anwar Ibrahim announced an allocation of RM30 million to help future-proof journalism while upholding ethical standards. (Bernama pic) PUTRAJAYA : The government has set aside RM30 million to help media organisations embrace digital transformation and the rise of artificial intelligence, Prime Minister Anwar Ibrahim announced today. Calling it the largest-ever allocation for the industry, Anwar said the funds would be used for training, capacity-building and exposure for journalists, particularly younger ones, to ensure the profession is not left behind. 'I have agreed to allocate a larger sum to support media organisations in the country as an initial step,' he said at the National Journalists Day (Hawana) celebration at World Trade Centre Kuala Lumpur today. 'It is to ensure the world of journalism is not left behind in keeping up with changes brought by the emergence of AI, while also ensuring a balance between AI and journalistic ethics.' At a press conference later, communications minister Fahmi Fadzil said the mechanism for media companies to apply for funding will be announced soon. He said the new allocation is a continuation of Project Sigma, a collaboration between his ministry and Google, that had helped over 30 media organisations in finding new revenue sources. 'We called it Project Sigma, in conjunction with Gen Z who likes to use the word 'sigma'. It intends to help media organisations better understand the new generation's way of viewing and consuming media,' he said. Following the encouraging results from the project, he said the ministry applied for a further RM30 million fund to re-create similar efforts. 'This is to help them achieve financial sustainability,' he said. Stay clear of 'manufacturing dissent' Anwar also urged the media to resist becoming propaganda tools for those with vested interests. He said media freedom is a fundamental pillar of democracy and crucial for protecting the freedom to give criticism and provide feedback. However, he said, the government will not tolerate the 'distortion of perceptions' to insult and attack others. '(American philosopher) Noam Chomsky called this 'manufacturing consent' because people are forced to accept the narrative. 'However, we must remember that we also have to reject 'manufacturing dissent'. 'There must be a balance between taking advantage of our freedom of information and ensuring it is not exploited to oppress any racial groups,' he said.
