Latest news with #JoelBecker
India Today
2 days ago
- India Today
AI was supposed to speed up coders, new study says it did the opposite
Contrary to popular belief, new research has found that using AI tools can actually slow down experienced software developers, especially when working in codebases they already know well. The study, conducted by the nonprofit research group METR, revealed that seasoned open-source developers took 19 per cent longer to complete tasks when using Cursor, a widely used AI-powered coding assistant. As per the study, the result was based on a randomised controlled trial, which involved contributors working on their own open-source projects. advertisementBefore the trial began, developers believed AI would significantly increase their speed, which is estimated at a 24 per cent improvement in task completion time. Even after finishing their tasks, many still believed the AI had helped them work faster, estimating a 20 per cent improvement. But the real data showed otherwise.'We found that when developers use AI tools, they take 19 per cent longer than without, AI makes them slower,' the researchers wrote. The lead authors of the study, Joel Becker and Nate Rush, admitted the results came as a surprise. Rush had initially predicted 'a 2x speed up, somewhat obviously.' But the study told a different story. The findings challenge the widespread notion that AI tools automatically make human coders more efficient, a belief that has attracted billions of dollars in investment and sparked predictions that AI could soon replace many junior engineering studies have shown strong productivity gains with AI. One found that AI helped developers complete 56 per cent more code, while another claimed a 26 per cent boost in task volume. But the METR study suggests that those gains don't apply to all situations, especially where developers already have deep familiarity with the of streamlining work, the AI often made suggestions that were only 'directionally correct,' said Becker. 'When we watched the videos, we found that the AIs made some suggestions about their work, and the suggestions were often directionally correct, but not exactly what's needed.'As a result, developers spent additional time reviewing and correcting AI-generated code, which ultimately slowed them down. However, the researchers do not believe this slowdown would apply to all coding scenarios, such as those involving junior developers or unfamiliar the results, both the study's authors and most participants continue to use Cursor. Becker suggested that while the tool may not speed up work, it can still make development feel easier and more enjoyable.'Developers have goals other than completing the task as soon as possible,' he said. 'So they're going with this less effortful route.'The authors also emphasised that their findings should not be over-generalised. The slowdown only reflects a snapshot of AI's capabilities as of early 2025, and further improvements in prompting, training, and tool design could lead to different outcomes in AI systems continue to evolve, METR plans to repeat such studies to better understand how AI might accelerate, or hinder, human productivity in real-world development settings.- Ends
Yahoo
2 days ago
- Yahoo
Experienced software developers assumed AI would save them a chunk of time. But in one experiment, their tasks took 20% longer
AI tools don't always boost productivity. A new study from Model Evaluation and Threat Research found that when 16 software developers were asked to perform tasks using AI tools, the they took longer than when they weren't using the technology, despite their expectations AI would boost productivity. The research challenges the dominant narrative of AI driving a workplace efficiency boost. It's like a new telling of the 'Tortoise and the Hare': A group of experienced software engineers entered into an experiment where they were tasked with completing some of their work with the help of AI tools. Thinking like the speedy hare, the developers expected AI to expedite their work and increase productivity. Instead, the technology slowed them down more. The AI-free tortoise approach, in the context of the experiment, would have been faster. The results of this experiment, published in a study this month, came as a surprise to the software developers tasked with using AI—and to the study's authors, Joel Becker and Nate Rush, technical staff members of nonprofit technology research organization Model Evaluation and Threat Research (METR). The researchers enlisted 16 software developers, who had an average of five years of experience, to conduct 246 tasks, each one a part of projects on which they were already working. For half the tasks, the developers were allowed to use AI tools—most of them selected code editor Cursor Pro or Claude 3.5/3.7 Sonnet—and for the other half, the developers conducted the tasks on their own. Believing the AI tools would make them more productive, the software developers predicted the technology would reduce their task completion time by an average of 24%. Instead, AI resulted in their task time ballooning to 19% greater than when they weren't using the technology. 'While I like to believe that my productivity didn't suffer while using AI for my tasks, it's not unlikely that it might not have helped me as much as I anticipated or maybe even hampered my efforts,' Philipp Burckhardt, a participant in the study, wrote in a blog post about his experience. Why AI is slowing some workers down So where did the hares veer off the path? The experienced developers, in the midst of their own projects, likely approached their work with plenty of additional context their AI assistants did not have, meaning they had to retrofit their own agenda and problem-solving strategies into the AI's outputs, which they also spent ample time debugging, according to the study. 'The majority of developers who participated in the study noted that even when they get AI outputs that are generally useful to them—and speak to the fact that AI generally can often do bits of very impressive work, or sort of very impressive work—these developers have to spend a lot of time cleaning up the resulting code to make it actually fit for the project,' study author Rush told Fortune. Other developers lost time writing prompts for the chatbots or waiting around for the AI to generate results. The results of the study contradict lofty promises about AI's ability to transform the economy and workforce, including a 15% boost to U.S. GDP by 2035 and eventually a 25% increase in productivity. But Rush and Becker have shied away from making sweeping claims about what the results of the study mean for the future of AI. For one, the study's sample was small and non-generalizable, including only a specialized group of people to whom these AI tools were brand new. The study also measures technology at a specific moment in time, the authors said, not ruling out the possibility that AI tools could be developed in the future that would indeed help developers enhance their workflow. The purpose of the study was, broadly speaking, to pump the brakes on the torrid implementation of AI in the workplace and elsewhere, acknowledging more data about AI's actual effects need to be made known and accessible before more decisions are made about its applications. 'Some of the decisions we're making right now around development and deployment of these systems are potentially very high consequence,' Rush said. 'If we're going to do that, let's not just take the obvious answer. Let's make high-quality measurements.' AI's broader impact on productivity Economists have already asserted that METR's research aligns with broader narratives on AI and productivity. While AI is beginning to chip away at entry-level positions, according to LinkedIn chief economic opportunity officer Aneesh Raman, it may offer diminishing returns for skilled workers such as experienced software developers. 'For those people who have already had 20 years, or in this specific example, five years of experience, maybe it's not their main task that we should look for and force them to start using these tools if they're already well functioning in the job with their existing work methods,' Anders Humlum, an assistant professor of economics at the University of Chicago's Booth School of Business, told Fortune. Humlum has similarly conducted research on AI's impact on productivity. He found in a working study from May that among 25,000 workers in 7,000 workplaces in Denmark—a country with similar AI uptake as the U.S.—productivity improved a modest 3% among employees using the tools. Humlum's research supports MIT economist and Nobel laureate Daron Acemoglu's assertion that markets have overestimated productivity gains from AI. Acemoglu argues only 4.6% of tasks within the U.S. economy will be made more efficient with AI. 'In a rush to automate everything, even the processes that shouldn't be automated, businesses will waste time and energy and will not get any of the productivity benefits that are promised,' Acemoglu previously wrote for Fortune. 'The hard truth is that getting productivity gains from any technology requires organizational adjustment, a range of complementary investments, and improvements in worker skills, via training and on-the-job learning.' The case of the software developers' hampered productivity points to this need for critical thought on when AI tools are implemented, Humlum said. While previous research on AI productivity has looked at self-reported data or specific and contained tasks, data on challenges from skilled workers using the technology complicate the picture. 'In the real world, many tasks are not as easy as just typing into ChatGPT,' Humlum said. 'Many experts have a lot of experience [they've] accumulated that is highly beneficial, and we should not just ignore that and give up on that valuable expertise that has been accumulated.' 'I would just take this as a good reminder to be very cautious about when to use these tools,' he added. This story was originally featured on Solve the daily Crossword
Phone Arena
11-07-2025
- Business
- Phone Arena
Do you know what is slowing down senior coders? As it turns out, it's AI
What has been your experience with AI? It helps me do things faster. It's a bottleneck! Can't really say right now. It helps me do things faster. 0% It's a bottleneck! 0% Can't really say right now. 0% Receive the latest mobile news By subscribing you agree to our terms and conditions and privacy policy Recommended Stories – Joel Becker, METR research leader, July 2025 What about smartphone AI? Grab Surfshark VPN now at more than 50% off and with 3 extra months for free! Secure your connection now at a bargain price! We may earn a commission if you make a purchase Check Out The Offer Picture this: the very thing that was created to help you is now your performance bottleneck. Quite the a new study that wreaks havoc to the idea that artificial intelligence supercharges the work of seasoned software developers. Instead of speeding them up, using AI tools actually slowed down experienced coders when they tackled projects they already knew study is carried out by AI research nonprofit METR and is focused on veteran developers working with Cursor, a popular AI coding assistant, on open-source projects they were familiar with. Before diving into the study, the developers expected the AI would save them time, guessing it could cut task completion by nearly a quarter. Even after using the AI, many still felt it had made them roughly 20% faster. But the hard data told a different story: AI extended the time needed to finish tasks by 19%.Joel Becker and Nate Rush, who led the research, admitted they were caught off guard by the outcome. Rush, before the study began, had predicted the AI would double productivity – and that's what many of us would think. The findings, however, cast doubt on the widespread assumption that AI tools reliably boost the productivity of highly skilled, high-cost software engineers. This idea is something that many companies have chosen to invest heavily into and big investments have been made comes as some tech leaders, including Dario Amodei, CEO of AI company Anthropic, have suggested that AI could eliminate as much as half of all entry-level white-collar jobs within the next five years. Previous studies have added to the hype, reporting significant productivity gains: one claimed AI sped up coding by 56%, while another found developers using AI completed 26% more tasks within the same METR's study tells a very different story. The boost in productivity doesn't hold up when developers are working on large, complex codebases they know well. In these cases, AI not only failed to help but actively slowed developers down. The problem stemmed from the need to double-check and often correct the AI's suggestions – suggestions that were frequently close to correct, but not precise enough to be trusted without careful explained that video recordings of the participants showed how AI often nudged developers in the right direction, but rarely delivered exactly what was needed:That led to additional time spent reviewing, editing, and sometimes discarding the AI's researchers were careful to point out that these results likely don't apply across the board. Less experienced engineers or those working on unfamiliar codebases could still benefit from AI study's findings may also offer a glimpse into how everyday smartphone users (like you and me) interact with AI-powered features on their devices. Just like developers, many smartphone users expect AI tools – from predictive text and voice assistants to photo editing suggestions – to streamline the day-to-day these features often require us to pause, review, and correct AI missteps, sometimes making simple actions feel more complicated than they should be. Whether it's an autocorrect blunder, a poorly framed photo enhancement, or a confusing AI-generated message reply, we as smartphone users are also discovering that AI is far from can sometimes come at the cost of time!
The Hindu
11-07-2025
- Business
- The Hindu
AI slows down some experienced software developers, study finds
Contrary to popular belief, using cutting-edge artificial intelligence tools slowed down experienced software developers when they were working in codebases familiar to them, rather than supercharging their work, a new study found. AI research nonprofit METR conducted the in-depth study on a group of seasoned developers earlier this year while they used Cursor, a popular AI coding assistant, to help them complete tasks in open-source projects they were familiar with. Before the study, the open-source developers believed using AI would speed them up, estimating it would decrease task completion time by 24%. Even after completing the tasks with AI, the developers believed that they had decreased task times by 20%. But the study found that using AI did the opposite: it increased task completion time by 19%. The study's lead authors, Joel Becker and Nate Rush, said they were shocked by the results: prior to the study, Rush had written down that he expected 'a 2x speed up, somewhat obviously.' The findings challenge the belief that AI always makes expensive human engineers much more productive, a factor that has attracted substantial investment into companies selling AI products to aid software development. AI is also expected to replace entry-level coding positions. Dario Amodei, CEO of Anthropic, recently told Axios that AI could wipe out half of all entry-level white collar jobs in the next one to five years. Prior literature on productivity improvements has found significant gains: one study found using AI sped up coders by 56%, another study found developers were able to complete 26% more tasks in a given time. But the new METR study shows that those gains don't apply to all software development scenarios. In particular, this study showed that experienced developers intimately familiar with the quirks and requirements of large, established open source codebases experienced a slowdown. Other studies often rely on software development benchmarks for AI, which sometimes misrepresent real-world tasks, the study's authors said. The slowdown stemmed from developers needing to spend time going over and correcting what the AI models suggested. 'When we watched the videos, we found that the AIs made some suggestions about their work, and the suggestions were often directionally correct, but not exactly what's needed,' Becker said. The authors cautioned that they do not expect the slowdown to apply in other scenarios, such as for junior engineers or engineers working in codebases they aren't familiar with. Still, the majority of the study's participants, as well as the study's authors, continue to use Cursor today. The authors believe it is because AI makes the development experience easier, and in turn, more pleasant, akin to editing an essay instead of staring at a blank page. 'Developers have goals other than completing the task as soon as possible,' Becker said. 'So they're going with this less effortful route.'
Indian Express
11-07-2025
- Business
- Indian Express
AI tools increase seasoned developers' task times, study finds
Contrary to popular belief, using cutting-edge artificial intelligence tools slowed down experienced software developers when they were working in codebases familiar to them, rather than supercharging their work, a new study found. AI research nonprofit METR conducted the in-depth study on a group of seasoned developers earlier this year while they used Cursor, a popular AI coding assistant, to help them complete tasks in open-source projects they were familiar with. Before the study, the open-source developers believed using AI would speed them up, estimating it would decrease task completion time by 24%. Even after completing the tasks with AI, the developers believed that they had decreased task times by 20%. But the study found that using AI did the opposite: it increased task completion time by 19%. The study's lead authors, Joel Becker and Nate Rush, said they were shocked by the results: prior to the study, Rush had written down that he expected 'a 2x speed up, somewhat obviously.' The findings challenge the belief that AI always makes expensive human engineers much more productive, a factor that has attracted substantial investment into companies selling AI products to aid software development. AI is also expected to replace entry-level coding positions. Dario Amodei, CEO of Anthropic, recently told Axios that AI could wipe out half of all entry-level white collar jobs in the next one to five years. Prior literature on productivity improvements has found significant gains: one study found using AI sped up coders by 56%, another study found developers were able to complete 26% more tasks in a given time. But the new METR study shows that those gains don't apply to all software development scenarios. In particular, this study showed that experienced developers intimately familiar with the quirks and requirements of large, established open source codebases experienced a slowdown. Other studies often rely on software development benchmarks for AI, which sometimes misrepresent real-world tasks, the study's authors said. The slowdown stemmed from developers needing to spend time going over and correcting what the AI models suggested. 'When we watched the videos, we found that the AIs made some suggestions about their work, and the suggestions were often directionally correct, but not exactly what's needed,' Becker said. The authors cautioned that they do not expect the slowdown to apply in other scenarios, such as for junior engineers or engineers working in codebases they aren't familiar with. Still, the majority of the study's participants, as well as the study's authors, continue to use Cursor today. The authors believe it is because AI makes the development experience easier, and in turn, more pleasant, akin to editing an essay instead of staring at a blank page. 'Developers have goals other than completing the task as soon as possible,' Becker said. 'So they're going with this less effortful route.'
