Why AI-generated nudes are pushing minors to suicide; and tech firms can't stop it

Malay Mail

5 days ago

WASHINGTON, July 17 — After a Kentucky teenager died by suicide this year, his parents discovered he had received threatening texts demanding US$3,000 to suppress an AI-generated nude image of him.
The tragedy underscores how so-called sextortion scams targeting children are growing around the world, particularly with the rapid proliferation of 'nudify' apps – AI tools that digitally strip off clothing or generate sexualized imagery.
Elijah Heacock, 16, was just one of thousands of American minors targeted by such digital blackmail, which has spurred calls for more action from tech platforms and regulators.
His parents told US media that the text messages ordered him to pay up or an apparently AI-generated nude photo would be sent to his family and friends.
'The people that are after our children are well organized,' John Burnett, the boy's father, said in a CBS News interview.
'They are well financed, and they are relentless. They don't need the photos to be real, they can generate whatever they want, and then they use it to blackmail the child.'
US investigators were looking into the case, which comes as nudify apps – which rose to prominence targeting celebrities – are being increasingly weaponized against children.
The FBI has reported a 'horrific increase' in sextortion cases targeting US minors, with victims typically males between the ages of 14 and 17. The threat has led to an 'alarming number of suicides,' the agency warned.
Instruments of abuse
In a recent survey, Thorn, a non-profit focused on preventing online child exploitation, found that six percent of American teens have been a direct victim of deepfake nudes.
'Reports of fakes and deepfakes – many of which are generated using these 'nudifying' services – seem to be closely linked with reports of financial sextortion, or blackmail with sexually explicit images,' the British watchdog Internet Watch Foundation (IWF) said in a report last year.
'Perpetrators no longer need to source intimate images from children because images that are convincing enough to be harmful – maybe even as harmful as real images in some cases – can be produced using generative AI.'
The IWF identified one 'pedophile guide' developed by predators that explicitly encouraged perpetrators to use nudifying tools to generate material to blackmail children. The author of the guide claimed to have successfully blackmailed some 13-year-old girls.
The tools are a lucrative business.
A new analysis of 85 websites selling nudify services found they may be collectively worth up to US$36 million a year.
The analysis from Indicator, a US publication investigating digital deception, estimates that 18 of the sites made between US$2.6 million and US$18.4 million over the six months to May.
Most of the sites rely on tech infrastructure from Google, Amazon, and Cloudflare to operate, and remain profitable despite crackdowns by platforms and regulators, Indicator said.
'Whack-a-mole'
The proliferation of AI tools has led to new forms of abuse impacting children, including pornography scandals at universities and schools worldwide, where teenagers created sexualized images of their own classmates.
A recent Save the Children survey found that one in five young people in Spain have been victims of deepfake nudes, with those images shared online without their consent.
Earlier this year, Spanish prosecutors said they were investigating three minors in the town of Puertollano for allegedly targeting their classmates and teachers with AI-generated pornographic content and distributing it in their school.
In the United Kingdom, the government this year made creating sexually explicit deepfakes a criminal offense, with perpetrators facing up to two years in jail.
And in May, US President Donald Trump signed the bipartisan 'Take It Down Act,' which criminalizes the non-consensual publication of intimate images, while also mandating their removal from online platforms.
Meta also recently announced it was filing a lawsuit against a Hong Kong company behind a nudify app called Crush AI, which it said repeatedly circumvented the tech giant's rules to post ads on its platforms.
But despite such measures, researchers say AI nudifying sites remain resilient.
'To date, the fight against AI nudifiers has been a game of whack-a-mole,' Indicator said, calling the apps and sites 'persistent and malicious adversaries.' — AFP