Child-safe phone will block nudes and abuse sent to minors

Telegraph

01-03-2025

The world's first phone that can prevent children from seeing harmful content online including abuse is set to be launched this summer after an AI breakthrough backed by the Government.
HMD, Europe's largest manufacturer of smartphones, has embedded the AI technology in a phone that is capable of automatically detecting and blocking harmful content before it reaches a child.
Unlike conventional parental controls which block access to entire sites and can be bypassed, the technology developed by UK AI specialists SafeToNet cannot be disabled as it is embedded in the phone's operating system.
It allows a child to remain on social media or other platforms but stops them seeing harmful content within them. The AI technology also extends to the phone's camera and video which prevents them taking any sexual 'selfies' or images that could be shared with a sexual predator or friends.
The phone's AI technology, known as HarmBlock, has been initially trained to block children from seeing any sexual content but will be extended to prevent them accessing 'gore', extreme violence, self-harm and suicide content, say Richard and Sharon Pursey, the founders of SafeToNet.
The online safety measures – to be announced on Sunday alongside a new starter phone designed with parents for children – have been welcomed by the Government whom SafeToNet has consulted in developing its technology.
A government spokesman said: 'Technological innovations offer opportunities to improve safety online, especially for children, which is why we wholeheartedly welcome this important work by partners like SafeToNet.
'It is encouraging to see a handset manufacturer use their technology to respond to real public concerns, particularly amongst parents, around safeguards for children.
'The Government will not accept child sexual abuse in any form. It is vital that legislation is robust enough to keep children safe, and last month we introduced new laws to crack down on this.'
The Purseys, husband and wife founders of SafeToNet and parents to four children, have spent 13 years developing the online safety technology which compresses huge gigabytes of tech so it can run on a phone at super-fast speeds without burning its battery.
'It must not destroy the user's experience of the application or the device. And of course, it has to be accurate, and accuracy is linked quite obviously to safety, because if we spot the harm and block the harm and prevent it from being seen, then we've safeguarded the child,' said Mr Pursey.
The AI technology has been trained to detect and block sexual imagery through the Internet Watch Foundation (IWF), a Cambridge-based charity that works with police, the Government and industry to seek out and remove child abuse imagery worldwide from the internet.
The tech has to be able to distinguish a benign holiday snap taken on a beach from a sexualised image that could be a precursor to a child being harmed or put at risk.
'We won't deploy unless we are accurate'
'We won't get this deployed unless we are highly accurate,' said Mrs Pursey. 'So we train our models on millions and millions of images to make sure that accuracy levels are as high as possible.'
The tech is targeted at protecting children when they livestream or use video as these modes of communication pose the biggest risk of online harm.
It is also why the AI models have to operate at superfast speeds and be capable of detecting potential threats even when there is background noise and little light.
The AI has also been trained to detect and block text messages that could be the precursor to a child being groomed by a paedophile although this element of the technology will not be immediately deployed.
The Purseys want to extend it to all devices used by children. 'Our vision in a perfect world is that we'd be able to produce our technology so that it doesn't matter who the manufacturer is or what the device is, you should be able to just press a button and download it,' said Mr Pursey.