'Tool for grifters': AI deepfakes push bogus sexual cures
HighlightsA brawny, shirtless man in AI-generated TikTok videos promotes unproven supplements that claim to enlarge male genitalia, using euphemisms like 'carrot' to evade content moderation. Experts warn that the rise of generative artificial intelligence has led to a surge in misleading content, with researchers noting a proliferation of AI-generated 'doctor' avatars endorsing questionable sexual remedies. The rapid generation of AI videos complicates content moderation, as tech platforms struggle to keep up with the near-immediate reappearance of similar misleading ads, turning the effort into a game of whack-a-mole.
Holding an oversized carrot, a brawny, shirtless man promotes a supplement he claims can enlarge male genitalia -- one of countless AI-generated videos on
TikTok
peddling unproven sexual treatments.
The rise of
generative AI
has made it easy -- and financially lucrative -- to mass-produce such videos with minimal human oversight, often featuring
fake celebrity endorsements
of bogus and potentially harmful products.
In some TikTok videos, carrots are used as a euphemism for male genitalia, apparently to evade content moderation policing sexually explicit language.
"You would notice that your carrot has grown up," the muscled man says in a robotic voice in one video, directing users to an online purchase link.
"This product will change your life," the man adds, claiming without evidence that the herbs used as ingredients boost testosterone and send energy levels "through the roof."
The video appears to be AI-generated, according to a deepfake detection service recently launched by the Bay Area-headquartered firm Resemble AI, which shared its results with AFP.
"As seen in this example, misleading
AI-generated content
is being used to market supplements with exaggerated or unverified claims, potentially putting consumers' health at risk," Zohaib Ahmed, Resemble AI's chief executive and co-founder, told AFP.
"We're seeing AI-generated content weaponized to spread false information."
'Cheap way'
The trend underscores how rapid advances in artificial intelligence have fueled what researchers call an AI dystopia, a deception-filled online universe designed to manipulate unsuspecting users into buying dubious products.
They include everything from unverified -- and in some cases, potentially harmful -- dietary supplements to weight loss products and sexual remedies.
"AI is a useful tool for grifters looking to create large volumes of content slop for a low cost," misinformation researcher Abbie Richards told AFP.
"It's a cheap way to produce advertisements," she added.
Alexios Mantzarlis, director of the Security, Trust, and Safety Initiative at Cornell Tech, has observed a surge of "AI doctor" avatars and audio tracks on TikTok that promote questionable sexual remedies.
Some of these videos, many with millions of views, peddle testosterone-boosting concoctions made from ingredients such as lemon, ginger and garlic.
More troublingly, rapidly evolving AI tools have enabled the creation of deepfakes impersonating celebrities such as actress Amanda Seyfried and actor Robert De Niro.
"Your husband can't get it up?" Anthony Fauci, former director of the National Institute of Allergy and Infectious Diseases, appears to ask in a TikTok video promoting a prostate supplement.
But the clip is a deepfake, using Fauci's likeness.
'Pernicious'
Many manipulated videos are created from existing ones, modified with AI-generated voices and lip-synced to match what the altered voice says.
"The impersonation videos are particularly pernicious as they further degrade our ability to discern authentic accounts online," Mantzarlis said.
Last year, Mantzarlis discovered hundreds of ads on YouTube featuring deepfakes of celebrities -- including Arnold Schwarzenegger, Sylvester Stallone, and Mike Tyson -- promoting supplements branded as erectile dysfunction cures.
The rapid pace of generating short-form AI videos means that even when tech platforms remove questionable content, near-identical versions quickly reappear -- turning moderation into a game of whack-a-mole.
Researchers say this creates unique challenges for policing AI-generated content, requiring novel solutions and more sophisticated detection tools.
AFP's fact checkers have repeatedly debunked scam ads on Facebook promoting treatments -- including erectile dysfunction cures -- that use fake endorsements by Ben Carson, a neurosurgeon and former US cabinet member.
Yet many users still consider the endorsements legitimate, illustrating the appeal of deepfakes.
"Scammy affiliate marketing schemes and questionable sex supplements have existed for as long as the internet and before," Mantzarlis said.
"As with every other bad thing online, generative AI has made this abuse vector cheaper and quicker to deploy at scale."
bur-ac/md
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Time of India
an hour ago
- Time of India
Why GenZ are so mindful about the amount of alcohol they are drinking
The new generation, Gen Z's, are skipping alcohol. Unlike previous generations, for whom drinking alcohol was a rite of passage, Gen Z is opting out and embracing sobriety as a personal lifestyle choice. Tired of too many ads? go ad free now They are doing this not due to external pressure but because they don't feel like drinking. They're proving that they can have a good time and enjoy themselves without consuming booze. According to TheConversation, Gen Z consumes about one-third less beer and wine than previous generations. They are also shifting toward alcohol-free beverages at a significantly higher rate. They have even called Gen Z 'the soberest generation.' The health risks of consuming alcohol Gen Z's efforts to bridge the knowledge gap about alcohol's effects have gained momentum, as they're more aware of harmful and adverse effects of alcohol than the older generations. With Gen Z experiencing higher rates of anxiety and depression, mental health awareness has played a significant role in this shift. However, they are also more likely to seek therapy from mental health professionals. According to the World Health Organisation, alcohol is linked to over 200 health conditions. Public awareness of these risks has been comparatively low. Initiatives must be taken to create awareness among the public about severe health conditions that can be caused by the consumption of alcohol. Gen Z's mindful approach to drinking alcohol Social media platforms like Instagram and TikTok are shaping Gen Z's view on alcohol by sharing its effects and promoting alternatives, helping to normalize taking breaks from drinking, also known as 'intermittent sobriety'. Tired of too many ads? go ad free now Influencers have shared their experiences with 'dry months' and improved mental conditions and clarity, inspiring others to follow the same. Gen Z's sense of self-awareness is reflected in movements which encourages people to make conscious efforts to evaluate their consumption of alcohol. This trend aligns with a broader cultural shift among younger generations toward mindfulness and intentional decision-making. Digital impact Previously, alcohol consumption has often been a centerpiece of social gatherings. But today, activities like wellness retreats, sober-friendly activities, and even dry bars are becoming popular. Social media has shifted the focus of social interaction. Today's population is aware about the lasting impact of whatever they post on digital platforms. Gen Z understands that actions, especially those influenced by alcohol, can be immortalized online, so they are more cautious about engaging in behavior that they might later regret. Businesses also need to rethink how they will cater to a generation drinking less alcohol. Many younger people are redefining what it means to have fun without the need for alcohol. For them, social experience is more about creating an inclusive and intentional environment. Also read |
Time of India
3 hours ago
- Time of India
Chennai IT company employees get surprise Creta SUVs as gifts as it marks 10th anniversary
Who says companies can't be generous towards employees? A Chennai-based firm's rather extravagant gesture of gratitude and goodwill towards loyal staff has grabbed eyeballs on the Internet. To commemorate a decade of its operations, Chennai-headquartered tech services firm Agilisium celebrated its 10-year milestone with a heartfelt gesture—gifting 25 brand-new Hyundai Creta SUVs to some of its most loyal employees. These individuals have been part of Agilisium's journey since its foundation, having devoted ten continuous years of service. The commemorative event was held at the company's office located in the World Trade Center, Chennai. It witnessed an enthusiastic gathering of more than 500 team members. The highlight of the occasion—the surprise SUV handover—had been meticulously kept secret, which intensified the joy and emotional reaction when the announcement was made. According to The Hindu Businessline, the moment left both the employees and their families overwhelmed and proud. In addition to the unexpected vehicle gifts, Agilisium rolled out pay increases based on individual performance across every level in the organization. This move, seen as a sign of the company's long-term commitment to its workforce, was especially appreciated during a time when the broader tech industry is navigating uncertainty and conservative spending. Play Video Pause Skip Backward Skip Forward Unmute Current Time 0:00 / Duration 0:00 Loaded : 0% 0:00 Stream Type LIVE Seek to live, currently behind live LIVE Remaining Time - 0:00 1x Playback Rate Chapters Chapters Descriptions descriptions off , selected Captions captions settings , opens captions settings dialog captions off , selected Audio Track default , selected Picture-in-Picture Fullscreen This is a modal window. Beginning of dialog window. Escape will cancel and close the window. Text Color White Black Red Green Blue Yellow Magenta Cyan Opacity Opaque Semi-Transparent Text Background Color Black White Red Green Blue Yellow Magenta Cyan Opacity Opaque Semi-Transparent Transparent Caption Area Background Color Black White Red Green Blue Yellow Magenta Cyan Opacity Transparent Semi-Transparent Opaque Font Size 50% 75% 100% 125% 150% 175% 200% 300% 400% Text Edge Style None Raised Depressed Uniform Drop shadow Font Family Proportional Sans-Serif Monospace Sans-Serif Proportional Serif Monospace Serif Casual Script Small Caps Reset restore all settings to the default values Done Close Modal Dialog End of dialog window. by Taboola by Taboola Sponsored Links Sponsored Links Promoted Links Promoted Links You May Like The Top 25 Most Beautiful Women In The World Articles Vally Undo Raj Babu, Agilisium's founder and CEO, emphasized that the company's core strength has always been its people. He noted that such meaningful gestures not only acknowledge contributions but also reinforce and energize the company's shared values and internal culture. Agencies The celebration didn't end with the event. Agilisium later shared snapshots and highlights from the occasion on its LinkedIn page, underscoring the deeper intention behind the celebration. The post described the gesture not just as recognition for completing ten years, but as a celebration of the company's foundational values—generosity, employee-centric thinking, and a focus on nurturing long-term growth and trust. According to the post, it was a reflection of what Agilisium had become: a thriving community that celebrates loyalty, invests in progress, and honors the stories behind each employee's professional journey. The post rapidly garnered attention across social platforms, with users appreciating the company's genuine approach to acknowledging commitment. Among the many responses, one viewer praised Agilisium's efforts to highlight employee loyalty with such a significant and thoughtful reward. With this move, Agilisium not only commemorated its own growth but also reinforced its image as an employer that deeply values and celebrates the people who shape its success. In an industry where recognition often gets overlooked, this initiative stood out as a benchmark in thoughtful corporate gratitude.
Time of India
4 hours ago
- Time of India
Startups show off ocean-preserving tech at Paris trade fair
Harnessing ocean currents to boost fuel efficiency of vessels, or tracking whales using sensor data and AI -- startups at Paris trade fair Vivatech have been showing off the latest innovations aimed at protecting the environment. Recently developed AI programs capable of learning from vast datasets have boosted projects trying to understand and predict real-world phenomena, several company founders told AFP. "We have to use AI because in the natural world there are too many variables" to deal with manually, said Emily Charry Tissier, a biologist and founder of Canadian startup Whale Seeker, which is developing technology to track sea mammals. by Taboola by Taboola Sponsored Links Sponsored Links Promoted Links Promoted Links You May Like Access all TV channels anywhere, anytime Techno Mag Learn More Undo Powered by "neural network" systems that ape the functioning of the human brain, the learning systems behind today's AI models "can calculate a weather forecast 1,000 times faster than a standard digital model running on a supercomputer", agreed oceanographer Alexandre Stegner. He flagged an AI model developed by his firm, Amphitrite, that he said could predict ocean currents by crunching "several layers of satellite data corresponding to different physical variables". Live Events It can forecast currents up to 10 days in advance, he said, offering sea captains "a simple way to save fuel" by slightly changing course and using currents to gain a speed boost of up to four knots. Discover the stories of your interest Blockchain 5 Stories Cyber-safety 7 Stories Fintech 9 Stories E-comm 9 Stories ML 8 Stories Edtech 6 Stories That could save operators money on fuel, reduce the carbon emissions from shipping, and avoid the classic solution of telling sea captains to reduce their speed. Global protection push Technologies like these were being shown off in the halls of Vivatech as the UN Ocean Conference (UNOC) drew to a close hundreds of kilometres to the south in French Mediterranean city Nice. The conference has pushed a treaty to protect 60% of the world's oceans closer to becoming law, with 55 signatures -- just five shy of the number required for its enactment. New technologies could be "a very good thing" for the oceans, said Andre Abreu, International Affairs Director at the Paris-based Tara Ocean Foundation. But he warned that innovation should not be harnessed to allow more fish to be caught. "That would mean shooting ourselves in the foot" on goals like preserving marine biodiversity, he said. That ambiguity can be seen in technology from OceanEyes, a Japanese startup using AI analysis of satellite data to predict sea conditions. The company hopes to cut the time fishing boats spend tracking down a catch. "A big problem in Japan is the efficiency of the fishery operations. Many fishers spend a lot of time searching for fish in the water," said boss Yusuke Tanaka. With less fuel burnt, operators will save money and greenhouse emissions can be slashed. Anticipating concerns about overfishing, OceanEyes said it also aimed to help vessels comply with recently updated Japanese regulations that oblige fishers to "ensure sustainable use of marine resources". 'Can' vs 'should' Whale Seeker's Tissier said technology could be used in a considered way to find sustainable solutions. "I'd like the market to recognise its own limits -- not the limits of what we can do, but what we should do," she told AFP. That attitude pushed her to refuse to work with a company that wanted to use whale detection to identify nearby fish to catch. But startups cannot grow without funding and, in the context of oceans, investments are likely to come from big firms keen to make a saving -- from fishing and ship management companies to haulage and logistics giants. This could well limit their ability to stand on principle. Stegner called for "regulations that would push the maritime sector to reduce carbon emissions". But Charry Tissier said the initiative could come from business. "Technology is developing so much faster than regulation... what I'd like is for big companies to decide for themselves to be responsible," she said.
