Over-hyped AI will have to work a lot harder before it takes your job
Some fascinating new research suggests that self-deception plays a key role in whether AI is perceived to be a success or a dud.
In a randomised controlled trial – the first of its kind – experienced computer programmers could use AI tools to help them write code. What the trial revealed was a vast amount of self-deception.
'The results surprised us,' research lab METR reported. 'Developers thought they were 20pc faster with AI tools, but they were actually 19pc slower when they had access to AI than when they didn't.'
In reality, using AI made them less productive: they were wasting more time than they had gained. But what is so interesting is how they swore blind that the opposite was true.
If you think AI is helping you in your job, perhaps it's because you want to believe that it works.
Since OpenAI's ChatGPT was thrown open to the general public in late 2022, pundits have been forecasting huge productivity gains from deploying AI. They hope that it will supercharge growth and boost GDP. This has become the default opinion in high-status policy circles.
But all this techno-optimism is founded on delusion. The 'lived experience' of using real tools in the real world paints a very different picture.
The past few days have felt like a turning point, as the reluctance of pointing out the emperor's new clothes diminishes.
'I build AI agents for a living, it's what I do for my clients,' wrote one Reddit user. 'The gap between the hype and what's actually happening on the ground is turning into a canyon'
AI isn't reliable enough to do the job promised. According to an IBM survey of 2,000 chief executives, three out of four AI projects have failed to show a return on investment, which is a remarkably high failure rate.
Don't hold your breath for a white-collar automation revolution either: AI agents fail to complete the job successfully about 65 to 70pc of the time, according to a study by Carnegie Mellon University and Salesforce.
The analyst firm Gartner Group has concluded that 'current models do not have the maturity and agency to autonomously achieve complex business goals or follow nuanced instructions over time.' Gartner's head of AI research Erick Brethenoux says: 'AI is not doing its job today and should leave us alone'.
It's no wonder that companies such as Klarna, which laid off staff in 2023 confidently declaring that AI could do their jobs, are hiring humans again.
This is extraordinary, and we can only have reached this point because of a historic self-delusion. People will even pledge their faith to AI working well despite their own subjective experience to the contrary, the AI critic Professor Gary Marcus noted last week.
'Recognising that it sucks in your own speciality, but imagining that it is somehow fabulous in domains you are less familiar with', is something he calls 'ChatGPT blindness'.
Much of the news is misleading. Firms are simply using AI as an excuse for retrenchment. Cost reduction is the big story in business at the moment.
Globally, President Trump's erratic behaviour has induced caution, while in the UK, business confidence is at 'historically depressed levels', according to the Institute of Directors, reeling from Reeves's autumn taxes. Attributing those lay-offs to technology is simply clever PR, and helps boost the share price.
So why does the faith in AI remain so strong?
The dubious hype doesn't help. Every few weeks a new AI model appears, and smashes industry benchmarks. xAI's Grok 4 did just that last week. But these are deceptive and simply provide more confirmation bias.
'Every single one of them has been wide of that mark. And not one has resolved hallucinations, alignment issues or boneheaded errors,' says Marcus.
Not only is generative AI unreliable, but it can't reason, as a recent demonstration showed: OpenAI's latest ChatGPT4o model was beaten by an 8-bit Atari home games console made in 1977.
'Reality is the ultimate benchmark for AI,' explained Chomba Bupe, a Zambian AI developer, last week. 'You not going to declare that you have built intelligence by beating toy benchmarks … What's the point of getting say 90pc on some physics benchmarks yet be unable to do any real physics?' he asked.
Then there are thousands of what I call 'wowslop' accounts – social media feeds that declare amazement at breakthroughs. As well as the vendors, a lot of shadowy influence money is being spent on maintaining the hype.
This is not to say there aren't uses for generative AI: Anthropic has hit $4bn (£3bn) in annual revenue. For some niches, like language translation and prototyping, it's here to stay. Before it went mad last week, X's Grok was great at adding valuable context.
But even if AI 'discovers' new materials or medicines tomorrow, that won't compensate for the trillion dollars that Goldman Sachs estimates business has already wasted on this generation of dud AI.
That's capital that could have been invested far more usefully. Rather than an engine of progress, poor AI could be the opposite.
METR added an amusing footnote to their study. The researchers used one other control group in its productivity experiment, and this group made the worst, over-optimistic estimates of all. They were economists.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
NBC News
40 minutes ago
- NBC News
Anthropic, Google, OpenAI and xAI granted up to $200 million for AI work from Defense Department
The U.S. Department of Defense on Monday said it's granting contract awards of up to $200 million for artificial intelligence development at Anthropic, Google, OpenAI and xAI. The DoD's Chief Digital and Artificial Intelligence Office said the awards will help the agency accelerate its adoption of 'advanced AI capabilities to address critical national security challenges.' The companies will work to develop AI agents across several mission areas at the agency. 'The adoption of AI is transforming the Department's ability to support our warfighters and maintain strategic advantage over our adversaries,' Doug Matty, the DoD's chief digital and AI officer, said in a release. Elon Musk's xAI also announced Grok for Government on Monday, which is a suite of products that make the company's models available to U.S. government customers. The products are available through the General Services Administration (GSA) schedule, which allows federal government departments, agencies, or offices to purchase them, according to a post on X. OpenAI was previously awarded a year-long $200 million contract from the DoD in 2024, shortly after it said it would collaborate with defense technology startup Anduril to deploy advanced AI systems for 'national security missions.' In June, the company launched OpenAI for Government for U.S. federal, state, and local government workers.
The Independent
an hour ago
- The Independent
Musk's Grok AI lands huge contract despite its recent problematic behaviors
The Department of Defense announced it will begin utilising Grok, the artificial intelligence bot developed by Elon Musk, after awarding xAI a $200 million contract. Musk's company, xAI, launched 'Grok for Government,' a new product suite designed to enable federal, state, and local governments to leverage AI for their specific requirements. This government-specific offering includes Grok 4, Deep Search, Tool Use, custom models for national security, and AI-powered applications for areas like healthcare, accessible via the General Services Administration. The announcement follows recent criticism of Grok for generating antisemitic statements on X, an issue xAI attributed to reliance on user input and is actively working to resolve. This contract is part of a broader initiative by the DoD, which also awarded $200 million contracts to Anthropic, Google, and OpenAI to broaden its use of frontier AI capabilities.
BBC News
2 hours ago
- BBC News
'We're not just gonna roll over': The US Civil War battlefield at the centre of a new conflict
The US state of Virginia saw some 50% of the nation's Civil War casualties. Now, mass construction of AI data centres is encroaching on historic lands, the environment and local communities. As the gunner of the Bull Run Legion lifts the explosive charge from its container and carries it up to the cannon, dozens of phones are lifted from pockets. Visitors open camera apps and raise devices high, focusing in on the handful of blue-uniformed men as they swab the gun's barrel with an era-appropriate sponge fastened to a stick. The cannon blasts its shot out into the air, and in the same moment, each phone fires its parcel of data at the cloud, landing in a data centre where banks of diligent servers sort, clean and route it along. On 21 July 1861, Union soldiers defending the United States watched over the crest of the same hills now framed on tourists' smartphones, as rebelling Confederate forces charged out of the forest in the first major battle of the American Civil War. A century and a half later, the Bull Run Legion, a group of "living historians" (better-known as re-enactors), still gather at Manassas National Battlefield Park using historic techniques and uniforms to commemorate that struggle. But today, the march of technology has turned this historic landmark into the scene of a new kind of battle. Technology companies are planning to erect one of the largest data centres in the world here at Manassas, Virginia, on the very ground where the Union army lost the war's first major land battle. Demand for data centres has skyrocketed as the artificial intelligence (AI) industry blossoms – but experts and advocates say the race for technological growth could threaten resources including water, energy and land. Some experts foresee a "growing crisis" for residents and future generations. The concerns aren't just practical, however – some fear building data centres too close to national landmarks could erase history and bury the lessons the Civil War has for future generations. "Do I want to see a data centre on the view line of this park? No, I do not. But do I recognise, since I worked in IT, that they're necessary? Yeah, I can get that. So there is no easy answer," Bart Wheeler, one of the living historians who works the cannon, tells the BBC. "Any interested citizen has to be worried." Those building the data centres disagree with these concerns and say they are taking steps to ensure the development is respectful of the historical context, even planning new information kiosks and trails around the site. Now, around 150 years after the Union army fought to protect this land in the 19th Century, a coalition of community advocates, environmentalists and history buffs are waging a differenty kind of war against an industry they say is steamrolling past their interests. It's a conflict with global repercussions. 'Data centre alley' The infrastructure which makes the internet work often lives in massive windowless buildings ringed by high black fences. Inside, stacked banks of computer servers receive signals and crunch numbers from devices around the world, providing the technical horsepower that allows for viral videos, cheeky text messages and more recently, untold numbers of AI queries. Data centres process our most intimate thoughts and transactions, but they're seldom seen by most users – that is, unless they're your neighbour. Around the world, the sprawling footprint of data centre development and their demands on resources have made this aspect of our digital lives more visible and controversial, particularly as the AI era sparks meteoric growth. "Customer demand remains very strong, driven by the digitalisation of the economy and AI revolution," says Karen Cohen, a spokesperson for QTS Data Centers, one of two companies seeking to build the massive Prince William Digital Gateway data centre development at Manassas. According to the American Battlefield Trust, the 37-building complex is the largest planned data centre development on earth the world. QTS and other data centre providers like it contract their servers out to other corporations, providing the backbone which makes the internet run. As the internet has grown, so has the industry's footprint in Northern Virginia: the region is already the world's biggest data centre market. As of this summer, there are plans for Virginia's approximately 340 data centres to be joined by up to 1,200 more, according to the Sierra Club, an environmental advocacy group. Northern Virginia was a cradle for the tech industry and continues to be a major nerve centre. Its proximity to Washington, DC and the defence industry, alongside generous tax breaks and access to a skilled workforce, helped the industry plant roots in an area sometimes dubbed " data centre alley". Several of the re-enactors of the Bull Run Legion tell the BBC they have careers in IT themselves, attending battle reenactments on the weekends. Rapidly scaling AI requires significantly more processing power and electricity than previous uses of the internet, and its need for data centres is ravenous: proposals have circulated to build data centres in lunar orbit, install nuclear reactors inside of them and spend trillions of dollars on their construction in the next few years. The Prince William Digital Gateway, planned to occupy land where soldiers clashed during the Battle of Second Manassas, will have a footprint of over 23 million sq ft (6.7 million sq km). Aaron Ruby, a spokesperson for Virginia's electric utility Dominion Power, tells the BBC that Dominion expects the energy demand of the state's data centre industry will quadruple over the next ten years – contributing to the doubling of the entire state's demand. And Virginia is no outlier. In 2024, data centres accounted for 1.5% of global energy demand, and that figure is expected to double to 945 terawatt-hours (TWh) by 2030. That's more energy than Japan uses today. But as the industry's footprint continues to grow, so too has local backlash. A 'growing crisis' A report by the state legislature says building the infrastructure to provide the necessary energy for the data centre boom "will be very difficult to achieve". The report adds that if the state intends to meet its clean energy goals and reduce the use of fossil fuels, it will be even harder to power the planned data centres. It also says that without guidelines in place, massive construction costs to build new power generation and lines could lead to higher electricity prices for regular consumers. QTS, however, says that it will pay any costs related to upgrading the electricity grid to ensure no impact to residential rates. But that's unrealistic, according to Ann Bennett, data centre issues chair of the Virginia chapter of the Sierra Club. "This very reputable audit wing of the government says 'we have a big problem'," Bennet tells the BBC, describing the legislature's report. "We're subsidising these major companies with our energy, but also with our tax dollars." Bennett and the Sierra Club have published their own report about the data centre industry and its environmental impacts, calling it a "growing crisis". Tax breaks supporting the industry totalled over $730m (£536m) in 2024, and while data centres have brought significant revenue and some jobs to the state, critics argue the climate impacts and energy costs are not sufficiently accounted for. Ruby of Dominion Power, though, says that "data centres currently pay the full cost of their power". In order to "make sure that remains the case going forward", he says the state report recommended consumer protections, and Dominion is working to get these protections approved. Bennett says this won't be enough. "It's almost too late," she says. "In some ways, this is becoming a mitigation project already." The data centre industry is entrenched in Virginia, and further development seems like an unstoppable force. But the Sierra Club hopes other jurisdictions can learn from Virginia's experience and strike the right balance. The fight for history "We are not anti-development," says David Duncan, a native Virginian and president of the American Battlefield Trust. His organisation, along with a group of others, is suing to stop the construction of the Prince William Digital Gateway. "We believe this should be an and conversation," Duncan tells the BBC. "That's what has been lost in this rush to build these types of facilities too close to our historic resources." Cohen says the data centre development won't interrupt the historical value of the area and QTS plans to create "miles of multi-purpose trails" and install "historical landmarks, interactive kiosks and other tools in areas of historical importance" on the battlefield site. But for some, the issue isn't a lack of signage. "Nobody wants to look into the forest and see these massive monoliths rising," Jim Matte, another member of the Bull Run Legion, tells the BBC. The Digital Gateway's approval was finalised in a controversial marathon 27 hour town hall meeting in December 2023, after a 4-3 vote in a lame duck session by some legislators who had already lost their seats in the previous election. "We're in litigation because they took shortcuts in the approval processes", says Duncan, adding that activists and citizens alike believe developers and lawmakers anticipated public backlash, but chose to go ahead with the approval anyway. (QTS did not respond to questions on this issue.) The Battlefield Trust has worked with – and litigated against – powerful developers in the past. Years ago, the organisation successfully altered a plan to build a Walmart at The Wilderness, another pivotal battlefield in Virginia, by moving the site three miles (4.8km) down the road. "When we clamp down on your ankle and don't let go, you're going to get sick of us. We're not just gonna roll over," Duncan says. But he also feels the data centre industry has more money and momentum behind it than Walmart. "There's so much power," Duncan says. The Civil War and technological transformation The idea of a data centre would be lost on the soldiers of the American Civil War, but they would be no strangers to technological upheaval. During the Civil War, freshly-strung telegraph lines knit marching armies together. The new technology of photography carried images of battle's toll to homes across the world. Advancements in munitions and firearms increased the deadliness of combat, locking the North and South in trench warfare stalemate through the war's final months. It ultimately gave the North – with its industrialised economy – the advantage. Manassas stood near the crossroads of two railroads, a technology only a few decades old at that point. Manoeuvres throughout the war revolved around access to rail supply lines, with armies seeking to cut off or secure important junctions. The conflict itself was precipitated by the nation's railroad-fuelled westward expansion, which brought the question of extending slavery to new territories or abolishing it to a head. More like this: • The 'bison skull mountain' photo that reveals the US's dark history • The Oregon Trail, the controversial video game that defined the US • I can't drink the water' – life next to a US data centre At Manassas, a version of the world these soldiers knew and fought for is preserved: the open pastures, rolling hills and the farmhouses they encountered on the march and then returned to as old men to commemorate. Today, as Americans face another redefinition of their country in an era of big tech and political division, for many the historic battlefields remain an important resource. "These places are crucial for the future of our country," says Duncan. "This country was defined on the battlefields of the American Civil War." -- For timely, trusted tech news from global correspondents to your inbox, sign up to the Tech Decoded newsletter, while The Essential List delivers a handpicked selection of features and insights twice a week. For more science, technology, environment and health stories from the BBC, follow us on Facebook, X and Instagram.
