New Gmail, Outlook Warning—Unbeatable AI Attacks Are Suddenly Here

Forbes

05-04-2025

Your inbox is now at risk
Email attacks are now soaring. But we may look back on this time as the beginning of something more dangerous and more frightening. Almost all attacks are still human generated, even if AI fine tunes text and images. That's about to change. Full AI attacks have just quietly passed a terrifying milestone. They are literally now unbeatable.
Just to illustrate how dangerous this is, both Google and Microsoft say they catch 'more than 99%' of the spam, phishing and malware targeting their users' inboxes. And yet we all still receive such emails, plenty get through. And they're not even smart. My recent favorite was an email from a personal btconnect.com email address, 'from the Office of Singapore Central Bureau of Investigation,' which then told me to respond immediately 'to avoid legal action against you as the Indian law demands."
And yet that email and countless others still bypass today's filters and safety nets. The much more sophisticated 'spear phishing' attacks do better. These are personalized and so take time and attention. There are less of them but more get through. This is where AI is now playing and it will be terrifying. A new report from Hoxhunt warns that these AI-crafted attacks can now beat human attacks for the first time.
'For the first time in over 2 years of testing,' the researchers warn, its AI agents 'created more effective simulated phishing campaigns against millions of global users than our elite human red teams could.' AI beats human — and it isn't even close. This means pointing AI at a target, and then letting it loose on the target's social media, LinkedIn, public profiles, to fuel a highly personalized attack with no errors. And this can be done at unlimited scale, against anyone and everyone, continually and endlessly.
The data shows just how rapidly AI attacks are getting smarter. We're not getting smarter, and while new AI is also deployed by email platforms to catch attacks before they hit, it won't be fast enough. 'In 2023,' Hoxhunt says, 'AI was 31% less effective than humans.' Even as late as November 2024, 'AI was 10% less effective than humans.' But in testing last month, 'AI was 24% more effective than humans."
'In 2024,' Hoxhunt says 'AI agents began tricking more novice users with the better-written emails. Meanwhile, human-generated attacks were much more effective than AI against users with more than 6 months of training. By February/March 2025, AI surpassed human red teams across the spectrum of user skill levels. From 2023 to 2025, AI's phishing performance relative to elite human red teams improved by 55%."
'The threat landscape has changed.' the team says. 'The phishing-as-a-service market will shift to mass adoption of AI Spear Phishing Agents. Once that happens, the baseline quality and effectiveness of mass phishing campaigns will rise to a level we currently equate with targeted spear phishing attacks."
We have seen similar recent warnings from Symantec (see video above) and Tenable, as this AI attack nightmare comes true. But the Hoxhunt report shows how quickly AI is improving, and clearly there are no guardrails that stop it continuing to advance.
So, is there any hope for what can be done? 'The good news from our research,' says Hoxhunt, 'is that there is still time to harden the human layer with adaptive phishing training. Behavior change programs can achieve extremely high levels of engagement and resilience with the use of AI Spear Phishing Agents… AI is a sword that cuts both ways; to penetrate or to parry,' albeit 'as AI technology continues to evolve, the ability to craft more sophisticated phishing attacks on-demand will only increase, making AI an essential tool in both offensive and defensive cybersecurity strategies.'
There is still time, only a relatively small percentage of phishing attacks are AI-driven. But that will change. And user behaviors and 'spidey senses' need to change just as quickly. Today, we are not ready for this. The question is whether we have time to prepare before the inevitable tidal wave is unleashed.