Experienced software developers assumed AI would save them a chunk of time. But in one experiment, their tasks took 20% longer
It's like a new telling of the 'Tortoise and the Hare': A group of experienced software engineers entered into an experiment where they were tasked with completing some of their work with the help of AI tools. Thinking like the speedy hare, the developers expected AI to expedite their work and increase productivity. Instead, the technology slowed them down more. The AI-free tortoise approach, in the context of the experiment, would have been faster.
The results of this experiment, published in a study this month, came as a surprise to the software developers tasked with using AI—and to the study's authors, Joel Becker and Nate Rush, technical staff members of nonprofit technology research organization Model Evaluation and Threat Research (METR).
The researchers enlisted 16 software developers, who had an average of five years of experience, to conduct 246 tasks, each one a part of projects on which they were already working. For half the tasks, the developers were allowed to use AI tools—most of them selected code editor Cursor Pro or Claude 3.5/3.7 Sonnet—and for the other half, the developers conducted the tasks on their own.
Believing the AI tools would make them more productive, the software developers predicted the technology would reduce their task completion time by an average of 24%. Instead, AI resulted in their task time ballooning to 19% greater than when they weren't using the technology.
'While I like to believe that my productivity didn't suffer while using AI for my tasks, it's not unlikely that it might not have helped me as much as I anticipated or maybe even hampered my efforts,' Philipp Burckhardt, a participant in the study, wrote in a blog post about his experience.
Why AI is slowing some workers down
So where did the hares veer off the path? The experienced developers, in the midst of their own projects, likely approached their work with plenty of additional context their AI assistants did not have, meaning they had to retrofit their own agenda and problem-solving strategies into the AI's outputs, which they also spent ample time debugging, according to the study.
'The majority of developers who participated in the study noted that even when they get AI outputs that are generally useful to them—and speak to the fact that AI generally can often do bits of very impressive work, or sort of very impressive work—these developers have to spend a lot of time cleaning up the resulting code to make it actually fit for the project,' study author Rush told Fortune.
Other developers lost time writing prompts for the chatbots or waiting around for the AI to generate results.
The results of the study contradict lofty promises about AI's ability to transform the economy and workforce, including a 15% boost to U.S. GDP by 2035 and eventually a 25% increase in productivity.
But Rush and Becker have shied away from making sweeping claims about what the results of the study mean for the future of AI.
For one, the study's sample was small and non-generalizable, including only a specialized group of people to whom these AI tools were brand new. The study also measures technology at a specific moment in time, the authors said, not ruling out the possibility that AI tools could be developed in the future that would indeed help developers enhance their workflow.
The purpose of the study was, broadly speaking, to pump the brakes on the torrid implementation of AI in the workplace and elsewhere, acknowledging more data about AI's actual effects need to be made known and accessible before more decisions are made about its applications.
'Some of the decisions we're making right now around development and deployment of these systems are potentially very high consequence,' Rush said. 'If we're going to do that, let's not just take the obvious answer. Let's make high-quality measurements.'
AI's broader impact on productivity
Economists have already asserted that METR's research aligns with broader narratives on AI and productivity. While AI is beginning to chip away at entry-level positions, according to LinkedIn chief economic opportunity officer Aneesh Raman, it may offer diminishing returns for skilled workers such as experienced software developers.
'For those people who have already had 20 years, or in this specific example, five years of experience, maybe it's not their main task that we should look for and force them to start using these tools if they're already well functioning in the job with their existing work methods,' Anders Humlum, an assistant professor of economics at the University of Chicago's Booth School of Business, told Fortune.
Humlum has similarly conducted research on AI's impact on productivity. He found in a working study from May that among 25,000 workers in 7,000 workplaces in Denmark—a country with similar AI uptake as the U.S.—productivity improved a modest 3% among employees using the tools.
Humlum's research supports MIT economist and Nobel laureate Daron Acemoglu's assertion that markets have overestimated productivity gains from AI. Acemoglu argues only 4.6% of tasks within the U.S. economy will be made more efficient with AI.
'In a rush to automate everything, even the processes that shouldn't be automated, businesses will waste time and energy and will not get any of the productivity benefits that are promised,' Acemoglu previously wrote for Fortune. 'The hard truth is that getting productivity gains from any technology requires organizational adjustment, a range of complementary investments, and improvements in worker skills, via training and on-the-job learning.'
The case of the software developers' hampered productivity points to this need for critical thought on when AI tools are implemented, Humlum said. While previous research on AI productivity has looked at self-reported data or specific and contained tasks, data on challenges from skilled workers using the technology complicate the picture.
'In the real world, many tasks are not as easy as just typing into ChatGPT,' Humlum said. 'Many experts have a lot of experience [they've] accumulated that is highly beneficial, and we should not just ignore that and give up on that valuable expertise that has been accumulated.'
'I would just take this as a good reminder to be very cautious about when to use these tools,' he added.
This story was originally featured on Fortune.com
Solve the daily Crossword
Hashtags

Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Yahoo
10 minutes ago
- Yahoo
Cognizant sees quarterly revenue above estimates on strong enterprise demand
(Reuters) -IT consulting company Cognizant Technology forecast third-quarter revenue above Wall Street expectations on Wednesday, owing to strong spending from customers looking to integrate artificial intelligence into their platforms. Cognizant's services have seen strong uptake from enterprises looking to automate processes and shift workloads to the cloud as they adopt AI in the hopes of boosting productivity and optimizing costs. "Our investments in talent, platforms and AI infrastructure drove our fourth-straight quarter of organic year-over-year revenue growth," said Cognizant CEO Ravi Kumar S. The company forecast third-quarter revenue between $5.27 billion and $5.35 billion, compared with analysts' expectations of $5.27 billion, according to data compiled by LSEG. It reported revenue of $5.25 billion in the second quarter, beating estimates of $5.19 billion. Cognizant reported earnings per share of $1.31 in the quarter ended June 30, compared with a profit of $1.14 per share a year ago.

Yahoo
10 minutes ago
- Yahoo
Lam Research forecasts upbeat quarterly revenue on strong chip equipment demand
(Reuters) -Lam Research forecast first-quarter revenue above Wall Street expectations on Wednesday, driven by strong demand for its specialized chip-making equipment used in developing advanced artificial intelligence processors, sending its shares up over 4% in extended trading. Demand for Lam's equipment has been bolstered by a surge in orders for artificial intelligence semiconductors, with chip designers striving to develop advanced processors to meet growing computing needs. However, tariffs imposed by U.S. President Donald Trump have introduced volatility into the chip industry, prompting firms to reassess costs and seek clarity on the potential impact of the duties. Dutch firm ASML, the world's biggest supplier of computer chip-making equipment, warned earlier this month that it may not achieve revenue growth in 2026. Lam forecast first-quarter revenue of $5.20 billion, plus or minus $300 million, compared with estimates of $4.63 billion, according to data compiled by LSEG. It forecast quarterly adjusted earnings per share of $1.20, plus or minus 10 cents, while analysts expect $1 per share. The company reported revenue of $5.17 billion for the quarter ended June 29, beating estimates of $5.01 billion. Lam reported quarterly earnings per share of $1.35, compared with a profit of 78 cents per share, a year ago. Sign in to access your portfolio
Yahoo
10 minutes ago
- Yahoo
Arm-Backed Ambiq Micro Shares Rise 61% After $96 Million IPO
(Bloomberg) -- Ambiq Micro Inc. shares rose 61% in their first trading day, after the maker of ultra-low power semiconductors for AI applications raised $96 million in an upsized initial public offering. The World's Data Center Capital Has Residents Surrounded An Abandoned Art-Deco Landmark in Buffalo Awaits Revival Budapest's Most Historic Site Gets a Controversial Rebuild San Francisco in Talks With Vanderbilt for Downtown Campus We Should All Be Biking Along the Beach Shares of the Austin-based company closed at $38.53 each on Wednesday in New York, above the IPO price of $24 apiece. The stock was halted twice for volatility. Ambiq's offering of 4 million shares was marketed in a range of $22 to $25 each. The trading gives Ambiq a market value of $680 million, based on the outstanding shares listed in its filings. Including stock options, warrants and restricted stock units, Ambiq has a diluted value closer to $741 million. Ambiq's platform promises to deliver as much as five times lower energy consumption versus traditional chips, the filings show. The Arm Holdings Plc-backed company's chip technology is helping to move AI computing power from the data center to wearable products such as smartwatches and fitness trackers, whose power limitations have previously hampered their AI potential. 'In phase one of the company we saw customers extend the battery life of devices, but the real value proposition that matters is when we enable the same battery life and add AI functions that were not viable before,' said Scott Hanson, Ambiq's chief technology officer. Other potential applications of Ambiq's technology could include embedding large language models on devices such as augmented- and virtual-reality glasses, Hanson said in an interview with Bloomberg News. Largest Customer Ambiq has a relatively concentrated customer base. Its largest customer in the first quarter of this year, which it didn't identify, accounted for 38% of revenue in the period, according to the filings. End customers accounting for 10% or more of net sales during the quarter included Garmin Ltd., Alphabet Inc.'s Google and another unidentified customer. In 2024, China's Huawei Technologies Co. accounted for 41% of net sales, while Garmin had 24% and Google had 21%. Ambiq's revenues have shifted away from mainland China, which generated 45.5% of sales in the first half of last year but only 8% to 9.5% based on preliminary sales figures for the first half of this year, according to the filings. The company was focusing outside China due to geopolitical concerns and subsidized competitors creating a price-sensitive environment in that country, the filings show. The company reported a net loss of $8.3 million on revenue of $15.7 million in the three months ended March 31, compared with a loss of $9.8 million on $15.2 million revenue in the same period in 2024, the filings show. Chip designer Arm, Kleiner Perkins, Singaporean state-backed investor EDB Investments Pte, VentureTech Alliance and Conductive Ventures are among Ambiq's investors. Bank of America Corp. and UBS Group AG led the offering. The company's shares trade on the New York Stock Exchange under the symbol AMBQ. --With assistance from Michael Hytha. (Updates with shares in first three paragraphs.) Burning Man Is Burning Through Cash Russia Builds a New Web Around Kremlin's Handpicked Super App Everyone Loves to Hate Wind Power. Scotland Found a Way to Make It Pay Off It's Not Just Tokyo and Kyoto: Tourists Descend on Rural Japan Cage-Free Eggs Are Booming in the US, Despite Cost and Trump's Efforts ©2025 Bloomberg L.P. Error in retrieving data Sign in to access your portfolio Error in retrieving data Error in retrieving data Error in retrieving data Error in retrieving data