5 days ago
All you need to know about voice spoofing and audio deepfakes
Analysis: Biometric fraud like voice spoofing and audio deepfakes are part of broader social engineering attacks by scammers and criminals
Voice spoofing involves using digital technology such as artificial intelligence to mimic someone's voice so accurately that it can deceive both humans and automated speaker verification systems. With recent rapid advancements in AI, creating these fake voices—often called "audio deepfakes"—has become alarmingly easy. Today, with just a few seconds of recorded speech from platforms like podcasts, YouTube or TikTok, machine learning models can generate highly realistic synthetic voices that mimic real individuals. It is a type of biometric fraud and often part of broader social engineering attacks.
How does voice spoofing work?
AI-powered tools analyses the unique patterns of a person's speech—such as tone, pitch, and rhythm—and use this data to produce synthetic speech that closely resembles the original voice. The technology have become so advanced that distinguishing between a real voice and a fake one is increasingly challenging.
From RTÉ Radio 1's The Business, BBC's File on 4. reporter Paul Connolly on how criminals are now using AI-generated voices to scam people out of their money
Typically, the process usually begins with an attacker collecting voice clips from online sources like social media or videos. Specialized AI models, like VGGish or YAMNet analyze these voice samples to extract important acoustic patterns from the voice, turning them into digital fingerprints called embeddings. These embeddings are then fed into voice generation systems such as Tacotron, WaveNet, or FastSpeech that produce new speech mimicking the original voice. The resulting fake voice can be used in phone calls or apps to impersonate someone in real time.
How is this going to impact us in the real world?
Financial scams are a growing problem and we've all had a (fairly ridiculous) phone call where a robot voice purporting to be from a company tries to get information or money, but more sophisticated versions have worked. In the UK, fraudsters used AI-generated voices to impersonate financial advisors, leading to a multi-million euro scam targeting crypto investors. In the US, the FBI has warned about scammers using AI to mimic senior US officials' voices, deceiving individuals into sharing confidential information.
There have also even been cases where scammers cloned the voices of loved ones, calling individuals and pretending to be in distress to extract money. These incidents highlight the disturbing reality that even the sound of someone's voice can no longer be trusted.
From CNN, can Donie O'Sullivan's parents tell the difference between RealDonie's voice and AI-Donie's voice?
Celebrities, politicians, and influencers are particularly at risk because their voices are widely available online. The more audio content (voice data) available publicly, the easier it is for AI tools to replicate their voice. This is a basic principle of AI: more data = better performance. However, it's not just public figures who are at risk. If you've ever posted a video or audio clip on platforms like Facebook, Instagram, or YouTube, your voice could potentially be cloned.
What are the difficulties in detecting voice spoofing?
Detecting synthetic voices is a complex task. Most traditional security systems and speaker verification systems often rely on voice recognition for authentication, but AI-generated voices have become sophisticated enough to deceive these systems. Some of the core technical challenges include:
Spectro-temporal similarity Fake voices closely mimic both pitch and timing patterns of natural speech.
Data imbalance: Real-world datasets typically contain fewer examples of spoofed voices, making it harder for AI to recognize these cases.
Generalisation: Many detection models struggle when faced with spoofing methods they weren't specifically trained on.
How to protect yourself
While the threat is real, there are steps you can take to safeguard against voice spoofing:
Be sceptical: If you receive an unexpected call requesting sensitive information or money, verify the caller's identity through another channel.
Use safe words: Establish a unique code word with family and close contacts that can be used to confirm identities during emergencies.
Limit voice sharing: Be cautious about sharing voice recordings online, especially on public platforms.
Stay informed: Keep abreast of the latest scams and educate yourself on how to recognize potential threats.
Voice spoofing poses a growing threat as AI continues to advance, making it easier than ever to mimic someone's voice convincingly. Whether you're a public figure or an everyday social media user, the potential to become a victim of voice cloning exists.
From RTÉ Radio 1's Ray D'Arcy Show, AI expert Henry Ajder talks on how deepfakes are damaging online trust and what some platforms are doing to rebuild it
How our research work is helping
Our recent research proposes an innovative and effective approach for detecting voice spoofing by using a hybrid deep learning (DL) architecture called VGGish-LSTM. We used VGGish, a pre-trained model developed by Google, to extract robust acoustic embeddings from audio data. These embeddings capture detailed features that are often not noticeable by human listeners but are critical in distinguishing synthetic voices.
Once extracted, these acoustic features are then analysed by a Long Short-Term Memory (LSTM) network, a type of artificial neural network designed specifically to detect long-term patterns and dependencies in sequential data. These networks excel at identifying variations in speech rhythm, tone, and pitch that could indicate synthetic or manipulated speech.
The advice for users is to stay vigilant, limit how much voice data you share online and adopt simple safety practices
Evaluated on the widely used ASV Spoof 2019 dataset, our model achieved an outstanding accuracy of over 90%. This performance demonstrates our model's ability to detect spoofing effectively and can be used in real-world scenarios such as banking authentication, call centre security, or smart home voice verification systems.
With ongoing research into detection technologies, such as the VGGish-LSTM model described here, we can continue developing robust defences to cope with voice spoofing scams. But for users, the advice is to stay vigilant, limit how much voice data you share online and adopt simple safety practices.
