Can people tell a real voice from an AI-generated one? We put it to the test
These were the innocuous sentences used in a simple experiment to find out if people in my social circle could distinguish my real voice from an AI-cloned version.
The result was some confusion - but more importantly, the ease with which the imitation was generated suggests that more attention should be paid to the phenomenon of deepfake voice phishing, or vishing.
Millions of dollars have been lost to scammers using cheap yet increasingly sophisticated artificial intelligence tools to impersonate the voices of real people.
In Asia-Pacific, the trend of AI and deepfake-enabled fraud is accelerating even faster than the rest of the world, according to cybersecurity firm Group-IB.
AI-related fraud attempts surged by 194 per cent in 2024 compared to 2023, with deepfake vishing emerging as one of the most commonly used methods, said the company's senior fraud analyst for the region Yuan Huang.
An increase has also been observed in Singapore. While exact vishing figures are not publicly available, Ms Huang pointed to a study which found that about 56 per cent of businesses here have experienced deepfake audio fraud.
The Cyber Security Agency of Singapore told CNA that audio deepfakes are a rising concern, due to how they exploit people's natural inclination to trust familiar voices, such as those seemingly from a family member or a colleague.
COPYING MY VOICE
My experiment, which was facilitated by Ms Huang from Group-IB at the start of August, showed that cloning a voice really did not take much: A publicly available online tool – of which there are several – and samples of my voice as short as 10 seconds.
A more convincing replication would require longer samples and more tone variations.
The subscription plan for the platform we chose was priced at less than US$10 for the first month. Elsewhere, fees can start at US$3 a month.
Under our plan, we could generate high-definition audio output and cloned voices in other languages.
Seconds after uploading my samples, my deepfaked voice was ready.
On the user-friendly interface, cloned voices can be fine-tuned by adjusting speeds or tweaking settings to be more monotonous or more dramatic. None of this required any particular technical skills; practically anyone can generate a cloned voice.
When I finally played back my AI-generated voice in full, I was taken by surprise.
I already knew it was supposed to sound like me. But the level of similarity and accuracy – down to the pauses I habitually take in between words – was something else.
The technology was not entirely flawless. The voice clone seemed to have a hint of an American accent; and the more we replayed it, the less it sounded like me.
But it would suffice to fool most people, according to studies at least.
A 2024 poll by the University of California in the United States found that participants mistook the identity of an AI voice for its real counterpart 80 per cent of the time.
When researchers at the University College London played audio clips to 100 people, just 48 per cent were able to tell which was a human voice and which was created using AI.
A recent public awareness survey by CSA found that a majority (78 per cent) had confidence in their ability to identify deepfakes. But only one in four could distinguish legitimate videos from AI-manipulated ones. No data or trends specific to audio content were available.
In an April 2024 reply to a parliamentary question, it was also revealed that the Singapore police has not been tracking the number of deepfake-related scams using video, voice and other media.
Home Affairs Minister K Shanmugam said then: "While we have received some reports where the complainants had alleged that deepfake techniques were used by the scammers, the number is not high."
PUT TO THE TEST
After the cloning came the fun part of the experiment.
We typed out the statement on checking out a restaurant - something I would say to friends - on the online platform; then separately recorded myself saying the same thing.
Then I called four contacts - two colleagues as well as two friends I've known for almost a decade - and played them the two audio clips.
One of my colleagues initially mistook my real voice for AI. But in the end, all four could tell the difference: According to them, no amount of tweaking of settings could take away the more robotic and monotonous flavour of the cloned voice.
One possible factor could be that the Singaporean accent is not as easy to clone, since most AI models are trained on American or British accents, said Associate Professor Terence Sim from the National University of Singapore's (NUS) school of computing.
My friends were also young adults in their 20s, who may be more aware of the hallmarks of AI use - and on higher alert in the environment of phone calls.
THE VISHING APPEAL
Experts CNA spoke to nonetheless noted the growing accessibility of low-cost and sometimes even free generative AI tools to convincingly mimic and create human voices.
A CSA spokesperson said that with audio deepfakes, "while there are traditional indicators like unnatural pauses in speech, robotic intonation, or unusual word choices, these tells are becoming harder to spot as AI technology advances".
"What makes audio deepfakes particularly challenging to detect is that humans are naturally attuned to trust voice communications, especially in urgent or emotional situations," he added.
"When someone appears to sound exactly like a family member or colleague in distress, the emotional response often overrides our usual caution."
The AI tools also allow scammers to spoof trusted institutions with accuracy, making it easier to manipulate targets and gain financial and sensitive information, said Assistant Professor Saifuddin Ahmed from the Wee Kim Wee School of Communication and Information at Nanyang Technological University (NTU).
"Combined with the availability of stolen personal data from breaches or user mistakes, the technology empowers scammers to craft highly convincing, personalised attacks."
Group-IB's Ms Huang also highlighted that social media is often the primary source for obtaining voice samples.
Other avenues include radio or TV interviews, webinars and public recordings available online.
"In some cases, scammers record a victim's voice during a phone call, particularly during impersonation or vishing attempts, to later use it for further attacks."
Scammers, being financially motivated, prefer targets who can potentially bring such high returns, Ms Huang said. This includes chief executives, chief financial officers and finance personnel.
But they can also target the elderly, particularly those unfamiliar with AI-generated voice technology, or those who can be emotionally manipulated.
Prominent vishing cases
In 2019, a CEO of an energy company based in the United Kingdom was convinced he was on the phone with his boss from their German parent company.
The CEO said he recognised a subtle German accent and that it even carried the "melody" of his boss' voice.
He ended up transferring US$243,000 to a fraudster who had used AI to spoof the German boss. The call had been made from an Austrian phone number, and the money was moved from a Hungarian bank account to one in Mexico before being spread out to other locations.
Earlier in 2025, a Hong Kong merchant lost about US$18.5 million in cryptocurrency to scammers impersonating a financial manager of a Chinese company over WhatsApp.
The merchant, who wanted to buy cryptocurrency mining equipment, communicated with who he thought was the financial manager over WhatsApp. He even received voice messages from the "manager" during purchase negotiations.
While these cases happened outside of Singapore, similar tactics have been observed in this region, especially targeting finance departments and C-level executives in trusted industries, said Group-IB's Ms Huang.
WHAT TO DO ABOUT IT
Attempting to regulate online voice cloning platforms would be a complex matter, with almost all of them hosted outside of Singapore's jurisdiction, experts told CNA.
Asst Prof Saifuddin from NTU said cooperation between governments and tech companies would be needed to set boundaries to safeguard users.
At an industry level, telecom providers can also implement advanced call authentication and improve their spam-blocking technologies, he said.
Asst Prof Saifuddin's advice to individuals was to use a different communication channel where possible, such as text or email, to confirm the request. "Never rely solely on the incoming call for verification," he said.
NUS' Assoc Prof Sim pointed to skills that could be picked up, such as how to listen out for hissing, mismatched background sounds in different segments of the speech, a lack of background sound, or abrupt transitions from one sentence to the next.
But he also acknowledged that these were not foolproof, with voice cloning technology constantly improving.
CSA said to be on alert when receiving urgent and unsolicited requests for monetary transfers or sensitive details such as passwords and banking credentials.
"For voice calls from a supposed friend or family member, ask questions that only they would know ... If in doubt, do hang up and call the friend or family member directly on their known number," said the agency's spokesperson.
In the case of callers claiming to be from institutions, dial their official hotline to verify.
"A healthy dose of scepticism is important in today's digital world," said CSA. "Not everything we see or hear online is what it appears to be, and if it's too good to be true, then it probably is."
That was also my takeaway from the experiment, topped off with a dose of wariness. The next time I get a call from an unknown number, I won't be the first to speak.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
CNA
an hour ago
- CNA
Wall Street stocks fall, traders focus on Fed
PARIS :European stock markets fell on Thursday and Wall Street opened lower, as traders avoided making big moves and waited for the Federal Reserve's three-day annual Jackson Hole symposium. Jackson Hole starts on Thursday and focus will be on Fed Chair Jerome Powell's speech on Friday, which could move markets as traders look for hints about the likelihood of a September rate cut. Stocks stayed near recent highs during Asian trading, and Australia's benchmark hit a new record. But markets struggled to make gains during the European session. 'People are sitting on their hands. You have a couple of big known unknowns coming, with Jackson Hole tomorrow and the Fed in September," said Tim Graf, head of macro strategy for EMEA at State Street Markets. 'This is the time to send the message, if you're going to ease, that it's coming," he said. "But I can also see them saying, we don't know the full effect of tariffs and inflation pressure is still not quite out of the economy and being a bit more balanced." Traders had ramped up bets for a September cut following a surprisingly weak payrolls report at the start of this month, and were further encouraged after consumer price data showed limited upward pressure from tariffs. But they lowered their expectations slightly following the release of minutes from the Fed's July meeting. By Thursday, markets were pricing in a 79.6 per cent chance of a September rate cut, compared to 83 per cent on Wednesday, according to LSEG data. At 1355 GMT, the S&P 500 was down 0.4 per cent on the day in its fourth consecutive day of losses, while the Dow Jones Industrial Average was down 0.5 per cent and the Nasdaq 100 was down 0.4 per cent. The pan-European STOXX 600 was down by 0.1 per cent on the day. The MSCI World Equity Index was down 0.2 per cent on the day. Analysts attributed a pullback in tech stocks this week to concerns that AI investments were not delivering returns. Euro zone business activity accelerated in August, PMI data showed, with Germany registering its fastest growth since March and France's downturn easing. Euro zone bond yields were mostly higher, with the benchmark 10-year German Bund at 2.7527 per cent . The 10-year U.S. Treasury yield was at 4.3277 per cent . The U.S. dollar index was up 0.3 per cent on the day at 98.48 , and the euro was down 0.2 per cent at $1.1622. U.S. President Donald Trump intensified his effort to influence the Federal Reserve on Wednesday, calling on Federal Reserve Governor Lisa Cook to resign on the basis of allegations made by one of his political allies about mortgages she holds in Michigan and Georgia. Cook said she had "no intention of being bullied to step down" from her position at the central bank. Deutsche Bank analysts in a research note attributed a rise in gold overnight to renewed concerns about the Fed's independence. "The news was a reminder of the lingering concerns over future Fed independence and risks of fiscal dominance, though the extent of the market reaction was fairly modest," Deutsche Bank said. State Street Markets' Tim Graf said that although central bank independence is considered "sacrosanct" by markets, it was not yet problematic. 'Markets quite rightly look through this, price maybe a little bit of risk premium for sure, but it's not something that I think really upsets the apple cart too much," he said.
CNA
an hour ago
- CNA
Surge in Chinese EV sales, rising incomes pushing COE prices to near record highs, experts say
SINGAPORE: The growing popularity of mass market Chinese electric vehicles (EVs), combined with increasing household incomes, could explain the highest Certificate of Entitlement (COE) premiums for cars in almost two years, experts said. One analyst said that the market has not reached peak COE supply, and that prices will likely drop only towards the end of the decade. Category A premiums rose to S$104,524 (US$81,300) in the latest bidding exercise on Wednesday (Aug 20), approaching the all-time high of S$106,000 recorded in October 2023. Prices for larger cars in Category B reached S$124,400, the highest since December 2023, while Open Category COEs climbed to S$125,001. Analysts had earlier projected that premiums could start falling from 2024 due to the 10-year cycle of COE supply, which had been increasing since 2014. WHY ARE COE PREMIUMS STILL INCHING HIGHER? One reason for the sustained high premiums is the influx of mass market Chinese EVs, said Associate Professor Walter Theseira from the Singapore University of Social Sciences. He noted that these vehicles offer 'excellent value' due to lower import costs and high build quality, while subsidies like the EV Early Adoption Incentive and the Vehicular Emissions Scheme rebate – which can shave up to S$40,000 off upfront costs – further spur demand. 'This is likely a contributing factor to higher COE premiums in recent months,' said Assoc Prof Theseira. Agreeing, transport analyst Terence Fan said that subsidies and the lower prices of EV cars soften the blow of high COE prices. 'Motorists dive in thinking they save some on the car, and pay more for the COEs, and they think that's okay,' said Assistant Professor Fan, from the Singapore Management University. Assoc Prof Theseira added that with incentives set to end this year, prices could moderate. Rising household incomes and economic optimism are also driving car purchases, said Dr Fan. While concerns about global trade due to the US tariffs cast a shadow earlier this year, it has become apparent that 'the world hasn't fallen apart'. 'Singapore registered gains in its GDP, so some of these gains are going to translate into people's income and the outlook at how (the economy) is going to be,' he said. If residents here are 'buoyant at the prospects' they will continue to make purchases such as for cars, said Dr Fan. WHY HASN'T SUPPLY CAUGHT UP TO DEMAND? Still, some may wonder why the rising supply of COEs has not yet eased premiums. Assoc Prof Theseira explained that while supply has been growing, it remains well below historical peaks. COE supply began to rise in 2014 but did so from a low base. From February 2013 to April 2014, fewer than 400 COEs were available in Category A in each bidding exercise. The COE quota then began steadily increasing, surpassing 1,400 in Category A in May 2015. For the next four years after, the supply remained relatively high at above 1,000 in each bidding exercise. COE quotas have increased steadily in the past few years. In 2022 and 2023, COE quotas for Category A did not surpass 1,000 in any given bidding exercise, but went across that figure in 2024, and has stayed above 1,000 so far in 2025. 'Although COE quota supply has risen significantly over the last year, it is still quite far below the peak supply years,' said Assoc Prof Theseira. He said that it typically takes a few years of increasing supply for prices to fall. For instance, while 2014 was when COE quota supply began to increase, it was only 'much later' in 2016 and 2017 that the supply 'went up'. It was in 2016 that there was a peak quota supply of 2,272 for one bidding exercise for that 10-year cycle. Assoc Prof Theseira added that this 10-year cycle is not 'perfect', as some cars are deregistered early, and others are renewed and may be deregistered at any time. WERE CUT-AND-FILL, COE INJECTIONS EFFECTIVE? Analysts say that measures such as the cut-and-fill strategy and the injection of up to 20,000 COEs were aimed more at stabilising prices than reducing them. The cut-and-fill strategy ensures COE quota supply can be brought forward from peak years to fill the current supply troughs while maintaining a zero-vehicle growth policy. This helps to ensure that there are no large fluctuations in COE prices, said Associate Professor Raymond Ong from the National University of Singapore. 'If let's say (the government) doesn't do cut and fill, what happens as in past few cycles, or even in this year's cycle, we will see a much reduced supply for COEs, which means there is an uncontrolled variation that is very likely to happen in the COE prices,' said Assoc Prof Ong, who is the associate head of research at NUS' Department of Civil and Environmental Engineering. 'It's this uncontrolled variation that we don't want to see,' he added. He noted that though COE premiums inched higher, they have remained relatively stable over the past months. Since the start of the year, Category A premiums have been between S$85,000 and S$104,524, while Category B premiums between S$109,598 and S$124,400. Agreeing, Assoc Prof Theseira said that the cut-and-fill method was designed to address the fundamental imbalances between COE supply across years where 'buyers in some years pay tens of thousands, maybe even a hundred thousand, more than buyers in other years'. As for the injection of 20,000 COEs, Assistant Professor Lee Kwok Hao said that the additional supply would have put premiums 'on a lower trajectory than without them'. However, the impact on lowering premiums is limited, as the 20,000 COEs only make up a small 2 per cent proportion of the vehicle population, said Dr Lee, who is from the Department of Strategy and Policy at the NUS Business School. 'Introduced over several years, I expect their impact on prices to be small relative to the secular impact of continued strong demand,' he said. Former Transport Minister Chee Hong Tat said last year that more of such injections can be considered if distance based charging under the ERP 2.0 system is implemented. WHEN WILL COE PREMIUMS FALL? Analysts were divided about whether COE premiums will fall significantly in the coming years. Assoc Prof Theseira believes that the 10-year cycle will eventually lead to a dip in prices towards the end of the decade, similar to the trend in the 2010s. Periods of high supply are usually a 'self-reinforcing' concept, he added. This is because car owners try to 'time' when to scrap their car, and when COE prices are low, owners are more likely to deregister their cars early. When many do this at the same time, it creates a surge of COEs, which keeps prices low. He expects this to occur in the late 2020s. 'Thus, for car buyers who can wait, (such as those whose) current COE doesn't expire for another few years, I would still recommend waiting,' he said. However, other analysts were more cautious. Assoc Prof Ong said that with rising incomes, more people are likely to enter the car market. 'The increased demand is unlikely to result in drop of price in the near term,' he said. Dr Lee said: 'If the current zero vehicle growth regime continues, given anticipated population growth, we should see prices continue to rise gradually.'
CNA
an hour ago
- CNA
ADB to provide $410 million package for Barrick-run Pakistan mine, sources say
ISLAMABAD :The Asian Development Bank will provide a $410 million financing package to help develop Pakistan's Reko Diq copper mine, one of the world's largest untapped deposits, which will be operated by Barrick Gold, two sources told Reuters on Thursday. Islamabad hopes the project will serve as a springboard to draw more foreign interest to its mineral sector, particularly to exploit rare earth deposits. Pakistan has already attracted interest from the Trump administration and offered future concessions to U.S. companies. The loans and a financing guarantee will support development of Reko Diq, which is expected to produce copper and gold from 2028 and generate about $70 billion in free cash flow over its lifespan. The financing is composed of two loans totalling $300 million to Barrick and a $110 million financing guarantee for the government of Pakistan, both sources said ahead of the official announcement. The $6.6 billion project in Balochistan is 50 per cent owned by Barrick with the other half held by the federal and provincial governments. ADB, the petroleum ministry and Barrick did not immediately respond to requests for comment. The project aims to raise upwards of $2 billion and has a previous agreement for $700 million in financing from the International Finance Corporation, the World Bank's private investment arm. The project's developers are in talks with other prospective financiers, including the U.S. Export-Import Bank, Export Development Canada and Japan's JBIC and expect to sign term sheets this quarter, project director Tim Cribb told Reuters in April. Reko Diq, delayed for years by a legal dispute that was settled in 2022, will produce 200,000 metric tons of copper annually in its first phase, rising to 400,000 tons after an expansion.
