I scammed my bank

Business Insider

02-05-2025

I may be a tech reporter, but I am not tech savvy. Something breaks, I turn it off and back on, and then I give up. But even I was able to deepfake my own bank with relative ease.
Generative AI has made it way easier to impersonate people's voices. For years, there have been deepfakes of politicians, celebrities, and the late pope made to sow disinformation on social media. Lately, hackers have been able to deepfake people like you and me. All they need is a few seconds of your voice, which they might find in video posts on Instagram or TikTok, and maybe some information like your phone or debit card number, which they might be able to find in data leaks on the dark web.
In my case — for the purposes of this story — I downloaded the audio of a radio interview I sat for a few weeks ago, trained a voice generator on it after subscribing to a service for a few dollars, and then used a text-to-voice function to chat with my bank in a voice that sounded a bit robotic but eerily similar to my own. Over the course of a five-minute call, first with the automated system and then a human representative, my deepfake seemingly triggered little to no suspicion.
It's a tactic scammers are increasingly adopting. They take advantage of cheap, widely available generative-AI tools to deepfake people and gain access to their bank accounts, or even open accounts in someone else's name. These deepfakes are not only getting easier to make but also getting harder to detect. Last year, a financial worker in Hong Kong mistakenly paid out $25 million to scammers after they deepfaked the company's chief financial officer and other staff members in a video call.
That's one major oopsie, but huge paydays aren't necessarily the goal. The tech allows criminal organizations to imitate people at scale, automating deepfake voice calls they use to scam smaller amounts from tons of people. A report from Deloitte predicts that fraud losses in the US could reach $40 billion by 2027 as generative AI bolsters fraudsters, which would be a jump from $12.3 billion in 2023. In a recent Accenture survey of 600 cybersecurity executives at banks, 80% of respondents said they believed gen AI was ramping up hackers' abilities faster than banks could respond.
These scammers can take gen-AI tools and target accounts at a massive scale. "They're the best engineers, the best product managers, the best researchers," says Ben Colman, the CEO of Reality Defender, a company that makes software for governments, financial institutions, and other businesses to detect the likelihood that content was generated by AI in real time. "If they can automate fraud, they will use every single tool." In addition to stealing your voice or image, they can use gen AI to falsify documents, either to steal an identity or make an entirely new, fake one to open accounts for funneling money.
In a recent Accenture survey of 600 cybersecurity executives at banks, 80% of respondents said they believed gen AI was ramping up hackers' abilities faster than banks could respond.
The scammers are playing a numbers game. Even when a financial institution blocks them, they can try another account or another service. By automating the attempts, "the attackers don't have to be right very often to do well," Colman says. And they don't care about going after only the richest people; scamming lots of people out of small amounts of money can be even more lucrative over time. According to the FBI's Internet Crime Complaint Center, the average online scam in 2024 came out to just under $20,000 across more than 250,000 complaints the FBI received from people of all ages (those over 60 filed the most complaints and saw the biggest losses, but even people under 20 lost a combined $22.5 million). "Everybody is equally a target," he says.
Colman says some banks have tried to get ahead of the deepfake problem in the past few years, while others didn't see it as a pressing issue. Now, more and more are using software to protect their clients. A 2024 survey of business executives (who worked across industries, not just in banking) found that more than 10% had faced an attempted or successful deepfake fraud. More than half said that their employees had not been trained to identify or address such attacks.
I reached out to several of the largest banks in the US, asking them what they're doing to detect and shut down deepfake fraud. Several did not respond. Citi declined to share any details of its fraud detection methods and technology. Darius Kingsley, the head of consumer banking practices at JPMorgan Chase, told me the bank sees "the challenges posed by rapidly evolving technologies that can be exploited by bad actors" and is "committed to staying ahead by continuously advancing our security protocols and investing in cutting-edge solutions to protect our customers."
Spotting deepfakes is tricky work. Even OpenAI discontinued its AI-writing detector shortly after launching it in 2023, reasoning that its accuracy was too low to even reliably detect whether something was generated by its own ChatGPT. Image, video, and audio generation have all been rapidly improving over the past two years as tools become more sophisticated: If you remember how horrifying and unrealistic AI Will Smith eating spaghetti looked just two years ago, you'll be shocked to see what OpenAI's text-to-video generator, Sora, can do now. Generative AI has gotten leaps and bounds better at covering its tracks, which is great news for scammers.
On my deepfake's call with my bank, I had fake me read off information like my debit card number and the last four digits of my Social Security number. Obviously, this was info I had on hand, but it's disturbingly easy these days for criminals to buy this kind of personal data on the dark web, as it may have been involved in a data leak. I generated friendly phrases that asked my bank to update my email address, please, or change my PIN. Fake me repeatedly begged the automated system to connect me to a representative, and then gave a cheery, "I'm doing well today, how are you?" greeting to the person on the other line. I had deepfake me ask for more time to dig up confirmation codes sent to my phone and then thank the representative for their help.
Authorities are starting to sound the alarm on how easy and widespread deepfakes are becoming. In November, the Financial Crimes Enforcement Network put out an alert to financial institutions about gen AI, deepfakes, and the risk of identity fraud. Speaking at the Federal Reserve Bank of New York in April, Michael Barr, a governor of the Federal Reserve, said that the tech "has the potential to supercharge identity fraud" and that deepfake attacks had increased twentyfold in the past three years. Barr said that we'll need new policies that raise the cost for the attacker and lower the burden on banks. Right now, it's relatively low risk and low cost for scammer organizations to carry out a massive number of attacks, and impossible for banks to catch each and every one.
It's not just banks getting odd calls; scammers will also use deepfakes to call up people and impersonate someone they know or a service they use. There are steps we can take if suspicious requests come our way. "These scams are a new flavor of an old-school method that relies on unexpected contact and a false sense of urgency to trick people into parting with their money," Ashwin Raghu, the head of scam policy and innovation at Citi, tells me in an email. Raghu says people should be suspicious of urgent requests and unexpected calls — even if they're coming from someone who sounds like a friend or family member. Try to take time to verify the caller or contact the person in a different way. If the call seems to be from your bank, you may want to hang up and call the bank back using the phone number on your card to confirm it.
For all the data on you that scammers can dig up using AI, there will be things that only two people can ever know. This past summer, an executive at Ferrari was able to catch a scammer deepfaking the company CEO's voice when he asked the caller what book he had recommended just days earlier. Limiting what you share on social media and to whom is one way to crack down on the likelihood you'll become a target, as are tools like two-factor authentication and password managers that store complex and varied passwords. But there's no foolproof way to avoid becoming a target of the scams.
Barr's policy ideas included creating more consistency in cybercrime laws internationally and more coordination among law enforcement agencies, which would make it more difficult for criminal rings to operate undetected. He also called for increasing penalties on those who attempt to use generative AI for fraud. But those won't be the quickest of fixes to keep up with how rapidly the tech has changed.
Even though this tech is readily available, sometimes in free apps and sometimes for purchases of just a few dollars, the problem is less a proliferation of lone wolf hackers, says Jason Ioannides, the vice president of global fintech and sponsor banking at Alloy, a fraud prevention platform. These are often carried out by big, organized crime rings that are able to move in large numbers and are bolstered by automation to carry out thousands of attacks. If they try 1,000 times to get through and make it once, they'll then focus their efforts on chipping away at that same institution, until the bank notices a trend and comes up with fixes to stop it. "They look for a weakness, and then they attack it," Ioannides says. He says banks should "stay nimble" and have "layered approaches" to detect quickly evolving fraud. "You're never going to stop 100% of fraud," he says. And banks generally won't be perfect, but their defense lies in making themselves "less attractive to a bad actor" than other institutions.
Ultimately, I wasn't able to totally hack my bank. I tried to change my debit card PIN and my email address during the phone calls, but I was told I had to do the first at an ATM and the second online. I was able to hear my account balance, and with a bit more prep and expertise, I may have been able to move some money. Each bank has different systems and rules in place, and some might allow people to change personal information, like emails, over the phone, which could give a scammer much easier access to the account. Whether my bank caught on to my use of a generated voice, I'm not sure, but I do sleep a little bit better knowing there are some protections in place.