Mind-reading device allows paralysed people to speak fluently

Telegraph

31-03-2025

A mind-reading device that allows paralysed patients to speak fluently just by thinking has been developed.
The technology can quickly decode brain signals produced by the motor cortex when a person wants to say a word before translating them into sound waves that can be 'spoken' by a synthesised voice.
Although similar devices have been trialled in the past, there has always been a lengthy delay between a person thinking a word and it being said out loud, making it tricky to form coherent sentences.
However, a team at the University of California have used the same rapid speech-decoding capacity of AI devices such as Alexa and Siri to help speed up the process and produce more natural speech.
'Intercepting signals'
Cheol Jun Cho, a doctoral student at UC Berkeley, said: 'We are essentially intercepting signals where the thought is translated into articulation.
'So what we're decoding is after a thought has happened, after we've decided what to say, after we've decided what words to use and how to move our vocal-tract muscles.
'This proof-of-concept framework is quite a breakthrough. We will continue to push the algorithm to see how we can generate speech better and faster.'
Many people are unable to speak because of paralysis, disease or injury and current speech generators – which often involve gazing at individual words or letters on a screen – are time-consuming and laborious.
The new device was trialled on a paralysed patient named Ann, who had an electronic array implanted over her motor cortex, to pick up her brain signals.
She was then asked to look at phrases on a screen, such as 'Hey, how are you?' and silently attempt to speak the sentences.
The programme was able to pick out chunks of neural activity behind certain sounds so they could be reproduced as a synthesised voice.
Prof Gopala Anumanchipalli, an assistant professor of electrical engineering and computer sciences at UC Berkeley, said: 'Within one second, we are getting the first sound out. And the device can continuously decode speech, so Ann can keep speaking without interruption.'
He added: 'Our streaming approach brings the same rapid speech decoding capacity of devices like Alexa and Siri to neuroprostheses.
'Using a similar type of algorithm, we found that we could decode neural data and, for the first time, enable near-synchronous voice streaming. The result is more naturalistic, fluent speech synthesis.'
The team used samples of Ann's pre-injury voice for the synthesised audio so that it would sound more like her, and found that even when she was thinking quickly, the algorithm could keep up.
The researchers believe that the technology may also work without the need for invasive electrodes, by using sensors on the face to measure muscle activity.
They also want to improve the algorithm so that it can pick up and convey the changes in tone, pitch and loudness that occur during speech, such as when someone is excited.
'Long-standing problem'
Kaylo Littlejohn, a doctoral student at UC Berkeley's department of electrical engineering and computer sciences, said: 'Previously, it was not known if intelligible speech could be streamed from the brain in real time.
'That's ongoing work, to try to see how well we can actually decode these paralinguistic features from brain activity.
'This is a long-standing problem even in classical audio synthesis fields and would bridge the gap to full and complete naturalism.'
Edward Chang, the senior co-principal investigator of the study who leads the clinical trial at UC San Francisco, added: 'This new technology has tremendous potential for improving quality of life for people living with severe paralysis affecting speech.
'It is exciting that the latest AI advances are greatly accelerating brain computer interfaces for practical real-world use in the near future.'