AI deepfakes are pushing bogus sexual cures
WASHINGTON : Holding an oversized carrot, a brawny, shirtless man promotes a supplement he claims can enlarge male genitalia – one of countless AI-generated videos on TikTok peddling unproven sexual treatments.
The rise of generative AI has made it easy – and financially lucrative – to mass-produce such videos with minimal human oversight, often featuring fake celebrity endorsements of bogus and potentially harmful products.
In some TikTok videos, carrots are used as a euphemism for male genitalia, apparently to evade content moderation policing sexually explicit language.
'You would notice that your carrot has grown up,' the muscled man says in a robotic voice in one video, directing users to an online purchase link.
'This product will change your life,' the man adds, claiming without evidence that the herbs used as ingredients boost testosterone and send energy levels 'through the roof.'
The video appears to be AI-generated, according to a deepfake detection service recently launched by the Bay Area-headquartered firm Resemble AI, which shared its results with AFP.
'As seen in this example, misleading AI-generated content is being used to market supplements with exaggerated or unverified claims, potentially putting consumers' health at risk,' Zohaib Ahmed, Resemble AI's chief executive and co-founder, told AFP.
'We're seeing AI-generated content weaponised to spread false information.'
'Cheap way'
The trend underscores how rapid advances in artificial intelligence have fuelled what researchers call an AI dystopia, a deception-filled online universe designed to manipulate unsuspecting users into buying dubious products.
They include everything from unverified – and in some cases, potentially harmful – dietary supplements to weight loss products and sexual remedies.
'AI is a useful tool for grifters looking to create large volumes of content slop for a low cost,' misinformation researcher Abbie Richards told AFP.
'It's a cheap way to produce advertisements,' she added.
Alexios Mantzarlis, director of the Security, Trust, and Safety Initiative at Cornell Tech, has observed a surge of 'AI doctor' avatars and audio tracks on TikTok that promote questionable sexual remedies.
Some of these videos, many with millions of views, peddle testosterone-boosting concoctions made from ingredients such as lemon, ginger and garlic.
More troublingly, rapidly evolving AI tools have enabled the creation of deepfakes impersonating celebrities such as actress Amanda Seyfried and actor Robert De Niro.
'Your husband can't get it up?' Anthony Fauci, former director of the National Institute of Allergy and Infectious Diseases, appears to ask in a TikTok video promoting a prostate supplement.
But the clip is a deepfake, using Fauci's likeness.
'Pernicious'
Many manipulated videos are created from existing ones, modified with AI-generated voices and lip-synced to match what the altered voice says.
'The impersonation videos are particularly pernicious as they further degrade our ability to discern authentic accounts online,' Mantzarlis said.
Last year, Mantzarlis discovered hundreds of ads on YouTube featuring deepfakes of celebrities – including Arnold Schwarzenegger, Sylvester Stallone, and Mike Tyson – promoting supplements branded as erectile dysfunction cures.
The rapid pace of generating short-form AI videos means that even when tech platforms remove questionable content, near-identical versions quickly reappear – turning moderation into a game of whack-a-mole.
Researchers say this creates unique challenges for policing AI-generated content, requiring novel solutions and more sophisticated detection tools.
AFP's fact checkers have repeatedly debunked scam ads on Facebook promoting treatments – including erectile dysfunction cures – that use fake endorsements by Ben Carson, a neurosurgeon and former US cabinet member.
Yet many users still consider the endorsements legitimate, illustrating the appeal of deepfakes.
'Scammy affiliate marketing schemes and questionable sex supplements have existed for as long as the internet and before,' Mantzarlis said.
'As with every other bad thing online, generative AI has made this abuse vector cheaper and quicker to deploy at scale.'
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
The Star
19 minutes ago
- The Star
Nasdaq slides again as AI jitters rattle tech investors
NEW YORK (Reuters) -Tech stocks are leading declines on Wall Street, with worries about AI spurring debates about its future. The Nasdaq Composite is down around 2.4% over the last two days, the worst two-day fall since April. The semiconductor index was down 1.5%, while the information technology sector was the second biggest decliner in the S&P 500, dropping 1.1% on Wednesday. Market participants attributed the selloff to a range of factors including a technical pullback after driving much of the stock market's recovery in the weeks after April 2nd "Liberation Day." Analysts also cited deepening concerns of government interference with companies, as the Trump administration looked into taking equity stakes in chip companies such as Intel in exchange for grants under the CHIPS Act. COMMENTS: ART HOGAN, MARKET STRATEGIST, B. RILEY WEALTH MANAGEMENT, BOSTON: "Technology in general is up 40% from its April lows, and the group clearly got ahead of itself. Also, if there's anything to the market consensus that we'll see a Fed rate cut, then there will be room for other things to work as well – and there are 493 other stocks in the S&P 500 that are lagging the Mag 7 right now. So I think there's a bit of a rotation." "I don't know how long it will last, but if it does keep going, well, August and September (are) the weak period seasonally in which it could do so. Also, there are some people who are beginning to question the pace at which we need to be chasing AI capital spending. If you put all this together: when technology stocks take a breather, this is what it looks like. Nvidia and other blue chips in the group are seeing relatively steady drawdowns, but things on the speculative edge are clearly seeing more selling pressure. Palantir has gone from trading at 200 times sales to 150 times its sales, for instance." MICHAEL ASHLEY SCHULMAN, CHIEF INVESTMENT OFFICER, RUNNING POINT, EL SEGUNDO, CALIFORNIA: "Tuesday' s U.S. technology stock swoon and its continuation today looks like multiple compression meeting a little margin math, but the timing makes it hard to ignore the new elephant in the server room. Names that had been sprinting on AI dreams pulled back hard, with Nvidia, AMD, and Palantir Technologies among the drags." "DeepSeek's update landed on Tuesday represents a serious cocktail of capability and availability and traders well remember the original harsh tech-market pullback DeepSeek caused when it was first broadly recognized in January of this year." BRIAN JACOBSEN, CHIEF ECONOMIST, ANNEX WEALTH MANAGEMENT, BROOKFIELD, WISCONSIN: "When you go from rally to rout, it shows how vulnerable the names were to even a scent of bad news. It could have been (Sam)Altman's valuation warning and then Meta restructuring its AI division threw fuel on the fire." PHIL BLANCATO, CHIEF EXECUTIVE OFFICER, LADENBURG THALMANN ASSET MANAGEMENT, NEW YORK: "It's much more about profit-taking and temporary rebalancing here. If you get a Federal Reserve cut or a mention of it on Friday, this will reverse pretty quickly, but this is a lot to do with names pushed up to really lofty levels and profit taking across the board." SETH HICKLE, PORTFOLIO MANAGER, MINDSET WEALTH MANAGEMENT, INDIANAPOLIS: "I think we are starting to see a little bit of rotation. It's always healthy to see a little bit of a pullback to that way, the markets can kind of get re-oriented." "To me, tech was overbought. Maybe it was justified, but it could have been kind of a buy on the rumor, sell it on the news type of thing where we had tech runup into earnings. We had really good earnings, and now it's kind of natural for the market just to sell some of that good news." "I wouldn't be surprised if we see a little bit of rotation into some smaller cap or into healthcare names, or consumer staples. And to me, that's kind of a healthy rotation. But honestly, I don't believe it will be a longer-term trend. It'll probably be a shorter-term trend. I think we'll see money flow back into tech in the next couple months." STEVESOSNICK, CHIEF STRATEGIST, INTERACTIVE BROKERS, CONNECTICUT "The tech-led selloff that we saw yesterday resumed this morning. That said, dip buyers stepped in around 11am EDT and we've now recovered about half our losses. It's somewhat inevitable to expect them to arrive promptly, though it did take a bit longer than usual." "I believe that some of the early declines are related to profit-taking and risk squaring ahead of (Fed Chair Jerome)Powell's speech on Friday. That is merely rotation and relatively benign, though it gets magnified because of megacap tech stocks' heavy weighting in key some of the ferocity of the early drop was related to the President's calls for Lisa Cook's resignation." "Note that futures broke through their pre-market lows shortly after he posted on Truth Social. Markets were not perturbed that there are inquiries into the propriety of her personal mortgage applications. She gets a presumption of innocence until proven guilty, like any other person. But when the President weighed in even before the process began, then it raised the specter of politicization. That put markets on the wrong foot early, and negative momentum ruled again – at least for a couple of hours." ADAM SARHAN, CHIEF EXECUTIVE, 50 PARK INVESTMENTS, NEW YORK: "To see a little pullback here after a big move up is perfectly normal and healthy. If the selling gets worse then you'll see a rotation out of tech and into undervalued areas of the market like biotech stocks or healthcare stocks or small cap stocks because those areas have not participated this year." (Reporting by Carolina Mandl, Johann Cherian, Laura Matthews, Suzanne McGee, and Gertrude Chavez-Dreyfuss)
The Star
19 minutes ago
- The Star
Analysis-US tech-stock stumble shows vulnerability in AI trade
NEW YORK (Reuters) -U.S. technology shares are showing signs of vulnerability after a massive run, which has some investors pointing to overdone AI-driven gains while funds have taken steps to position away from the high-flying sector. Investors are looking to de-risk portfolios or lock in profits during a seasonally difficult period for stocks. Friday's looming speech by Federal Reserve Chair Jerome Powell at the annual Jackson Hole symposium is creating caution, investors said, with the potential for volatility if his comments fail to meet growing market expectations that the central bank is poised to cut interest rates. "When you have overcrowding and you have had such strong performance, it doesn't take much to see an unwind of that," said Keith Lerner, co-chief investment officer at Truist Advisory Services. "At the same time this week, everyone is waiting for the Fed, and there is repositioning ahead of that." The heavyweight S&P 500 tech sector fell sharply for a second consecutive session on Wednesday, putting its decline on the week at about 2.5%, while the tech-heavy Nasdaq Composite was off about 2% for the week. Shares of some highflyers, including Nvidia Corp and Palantir Technologies, were getting hit particularly hard. The pullback comes after a huge rally in which the tech sector soared over 50% through last week since the market's low for the year in April. That easily topped the 29% gain of the broader S&P 500 during that period and drove up valuations of tech stocks to lofty levels. Investors cited wariness about the artificial intelligence trade, which has been a key driver of tech stocks and the broader market as indexes have soared to record highs this year. Shares of Nvidia, the semiconductor giant that has symbolized the AI trade, have gained about 30% this year while shares of AI-focused data and analytics firm Palantir have roughly doubled year-to-date. Indeed, the tech sector's price-to-earnings ratio recently reached about 30 times expected earnings for the next 12 months, its highest level in a year, according to LSEG Datastream, while tech's share of the overall S&P 500's market value is nearly its highest since 2000. Recent cautionary signs included a study from researchers at the Massachusetts Institute of Technology that found that 95% of organizations are getting no return on AI investments, as well as comments by OpenAI CEO Sam Altman, who told tech news website the Verge last week that investors may be getting overexcited about AI. Since last week, some AI-linked shares have pulled back sharply: Nvidia has dropped about 5% while shares of Palantir have slumped some 16%. In Europe, stocks of so-called AI adopters have been under pressure over concerns over how powerful new AI models could disrupt the software sector. Still, some investors said, the caution is unlikely to be a sign that enthusiasm over AI is fizzling. 'These are price corrections," said Andrew Almeida, director of investments at financial planning network XYPN. "But if you look at the big picture, it's clear that more people will be investing more dollars in AI infrastructure. This is certainly not a 'reckoning' with the AI theme." JACKSON HOLE SEEN AS CRITICAL Investors also could be paring back their stock exposure during a traditionally rocky period for equities. August and September rank as the worst-performing months on average for the S&P 500 over the past 35 years, according to the Stock Trader's Almanac. "Valuations were stretched, these names have not taken a breather, and we're going into a tougher season for stocks," said King Lip, chief strategist at Baker Avenue Wealth Management. Other sectors such as consumer staples, healthcare and financials were up on the week, while relative strength for the equal-weight S&P 500 signaled to some investors a possible start of broadening of gains beyond the massive tech stocks that have led indexes higher. Powell's upcoming speech comes as Fed fund futures on Wednesday were indicating an 84% chance that the central bank will cut rates at its next meeting on September 16-17. Investors will be watching to see if Powell gives any indication that the central bank is on track for such a move or if he pushes back on the market's expectation for easing, which could spark volatility. Tech stocks tend to carry higher valuations which could make them sensitive to higher-than-expected interest rates going forward. "There are a lot of people who have overweighted tech, and it has worked for them," said Chuck Carlson, chief executive officer at Horizon Investment Services. "They don't want to get caught on the wrong side of that if in fact, the Fed doesn't do anything in September. So I think that is also causing (investors) to maybe not necessarily get out of tech, but to reduce the overweight a little bit." (Reporting by Lewis Krauskopf in New York; Additional reporting by Suzanne McGee and Laura Matthews in New York, Akash Sriram in Bengaluru and Lucy Raitano in London; Editing by Megan Davies and Matthew Lewis)
The Star
2 hours ago
- The Star
Pace of digital disruption accelerating, AI reshaping industries
KUALA LUMPUR: Artificial intelligence (AI) and rapid digital disruption are reshaping industries, compelling companies to eliminate inefficiencies, leverage data and reskill workers to stay competitive, according to speakers at the inaugural chief financial officer (CFO) forum 2025. UEM Sunrise Bhd board member and financial adviser Chari Thandalam Veeravalli Thirumala said the pace of digital disruption is accelerating, with companies increasingly displacing one another in the market. 'Short Message Service or SMS used to bring in billions in revenue, and then another company came along and took that business away – that's WhatsApp,' he said, pointing to how quickly established business models can be displaced. 'We went from renting digital versatile discs or DVDs to watching Netflix. Companies like Grab, Uber and Amazon have disrupted sectors.' Chari, who was also group chief financial officer of Malaysia's Axiata Group and Celcom Telecommunication for nine years until December 2016, stressed the importance of eliminating inefficiencies and using technology to create value. He pointed to examples such as Visa working with telecommunications companies to tackle credit card fraud, and Airbnb's rapid growth from zero to 650,000 rooms in 192 countries in just four years – compared with Marriott's 679,000 rooms across 80 countries built over 88 years – to illustrate how technology can fundamentally change the game. He noted that data-driven decision-making is key to maximising the benefits of digitalisation. 'Digitalisation is not about building a great website or an app. Every company should be a technology and data company,' he told a plenary session entitled 'The ultimate talent and tech synergy'. 'Don't jump into tech for the sake of tech. If technology helps and contributes to the bottom line, then tap into it.' On AI, Chari said the next wave, including agentic AI, would have an even bigger impact. 'AI is going to change the game even more. There are companies that have appointed AI as a board observer, and even those founded by a single founder but run by AI,' he said. 'The idea is not to know C++, but to know if there is a use case.' When asked whether the efficiency brought by AI would lead companies to cut headcount, Chari called for a broader perspective. 'If you look at history, when computers were introduced, people feared they would take away all the jobs. Yet, people didn't go away. The pie simply got bigger and technology has helped grow the pie,' he told StarBiz on the sidelines of the event. He drew parallels with past workplace changes, such as the absence of stenographers after bosses began typing their own correspondence on computers. 'That doesn't mean all those men and women lost their jobs – they were upskilled and moved into other roles,' he said, adding that AI would similarly create new opportunities. While he acknowledged that AI would take over some jobs, Chari stressed the importance of adapting. 'Upskill yourself – the water will find its level,' he said. Cargill South-East Asia director of finance Humayun Habib said innovation is not just about finding and fixing problems, but also about creating value for the business. He said companies should always look for ways to improve their processes and enhance the outcomes they deliver. This, he said, required recognising the role of teams on the ground in shaping and refining how work gets done. 'Generally, as leaders, we only know so much. The teams who are actually on the ground, executing business transactions, are the ones most familiar with the processes,' he said during another plenary session entitled 'Driving growth and innovation: initiative and culture', adding that this insight should be harnessed to close gaps and drive meaningful change. The forum, which was held here from Aug 13-14, was organised by Star Media Group Bhd with Airwallex as a gold sponsor. Airwallex is a financial technology company that assists businesses to accept and manage global payments in multiple currencies from one platform, streamline financial operations and improve overall efficiency.
