AI slows down some experienced software developers, study finds
The study found that using AI increased task completion time by 19 per cent.
SAN FRANCISCO - Contrary to popular belief, using cutting-edge artificial intelligence tools slowed down experienced software developers when they were working in codebases familiar to them, rather than supercharging their work, a new study found.
AI research nonprofit METR conducted the in-depth study on a group of seasoned developers earlier this year while they used Cursor, a popular AI coding assistant, to help them complete tasks in open-source projects they were familiar with.
Before the study, the open-source developers believed using AI would speed them up, estimating it would decrease task completion time by 24 per cent.
Even after completing the tasks with AI, the developers believed that they had decreased task times by 20 per cent. But the study found that using AI did the opposite: it increased task completion time by 19 per cent.
The study's lead authors, Mr Joel Becker and Mr Nate Rush, said they were shocked by the results: prior to the study, Mr Rush had written down that he expected 'a 2x speed up, somewhat obviously'.
The findings challenge the belief that AI always makes expensive human engineers much more productive, a factor that has attracted substantial investment into companies selling AI products to aid software development.
AI is also expected to replace entry-level coding positions. Mr Dario Amodei, CEO of Anthropic, recently told Axios that AI could wipe out half of all entry-level white collar jobs in the next one to five years.
Prior literature on productivity improvements has found significant gains: one study found using AI sped up coders by 56 per cent, another study found developers were able to complete 26 per cent more tasks in a given time.
But the new METR study shows that those gains don't apply to all software development scenarios. In particular, this study showed that experienced developers intimately familiar with the quirks and requirements of large, established open source codebases experienced a slowdown.
Other studies often rely on software development benchmarks for AI, which sometimes misrepresent real-world tasks, the study's authors said.
The slowdown stemmed from developers needing to spend time going over and correcting what the AI models suggested.
'When we watched the videos, we found that the AIs made some suggestions about their work, and the suggestions were often directionally correct, but not exactly what's needed,' Mr Becker said.
The authors cautioned that they do not expect the slowdown to apply in other scenarios, such as for junior engineers or engineers working in codebases they aren't familiar with.
Still, the majority of the study's participants, as well as the study's authors, continue to use Cursor today. The authors believe it is because AI makes the development experience easier, and in turn, more pleasant, akin to editing an essay instead of staring at a blank page.
'Developers have goals other than completing the task as soon as possible,' Mr Becker said. 'So they're going with this less effortful route.' REUTERS
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Straits Times
an hour ago
- Straits Times
Iran says it held ‘frank' nuclear talks with European powers
Find out what's new on ST website and app. A European delegation leaving the Iranian Consulate in Istanbul, following nuclear talks on July 25. ISTANBUL - Iranian diplomats said they held 'frank and detailed' nuclear talks on July 25 with counterparts from Germany, Britain and France, who have threatened to trigger sanctions if Tehran fails to agree a deal on uranium enrichment and cooperation with UN inspectors. The meeting in Istanbul was the first since Israel launched an attack on Iran in June targeting key nuclear and military sites, sparking a 12-day war and leading Tehran to pull away from working with the UN watchdog. The European diplomats were seen leaving the Iranian consulate shortly before 1100 GMT (7pm in Singapore) after spending several hours inside. Israel's offensive – which killed top commanders, nuclear scientists and hundreds of others and in which residential areas and military sites were struck – also derailed US-Iran nuclear talks that began in April. Since then, the European powers, known as the E3, have threatened to trigger a so-called 'snapback mechanism' under a moribund 2015 nuclear deal that would reinstate UN sanctions on Iran by the end of August. The sanctions trigger expires in October, and Tehran has warned of consequences should the E3 opt to activate it. Iran's Deputy Foreign Minister, Mr Kazem Gharibabadi, who attended the talks alongside senior Iranian diplomat Majid Mr Takht-Ravanchi, wrote on X that he had used the meeting to criticise the European stance on the 12-day conflict with Israel. Top stories Swipe. Select. Stay informed. Singapore SMRT to pay lower fine of $2.4m for EWL disruption; must invest at least $600k to boost reliability Singapore MRT service changes needed to modify 3 East-West Line stations on Changi Airport stretch: LTA Singapore S'pore could have nuclear energy 'within a few years', if it decides on it: UN nuclear watchdog chief Life 'Do you kill children?': Even before independence, S'pore has always loved its over-the-top campaigns Singapore Lung damage, poor brain development, addiction: What vaping does to the body Singapore Tipsy Collective sues former directors, HR head; alleges $14m lost from misconduct, poor decisions Singapore Fine for couple whose catering companies owed $432,000 in salaries to 103 employees Singapore Kopi, care and conversation: How this 20-year-old helps improve the well-being of the elderly He said the snapback mechanism had also been discussed, adding: 'It was agreed that consultations on this matter will continue.' Mr Takht-Ravanchi told state news agency IRNA the Iranian side had demanded 'punitive sanctions' be lifted 'as soon as possible'. Before the talks, a European source said the three countries were preparing to trigger the mechanism 'in the absence of a negotiated solution'. The source urged Iran to make 'clear gestures' on uranium enrichment and resuming cooperation with the UN nuclear watchdog. 'Common ground' Mr Gharibabadi warned earlier in the week that triggering sanctions – which would deepen Iran's international isolation and place further pressure on its already strained economy – would be 'completely illegal'. He accused European powers of 'halting their commitments' under the 2015 deal, which the United States unilaterally withdrew from in 2018 during President Donald Trump's first term. 'We have warned them of the risks, but we are still seeking common ground to manage the situation,' said Mr Gharibabadi. Iranian diplomats have previously warned that Tehran could withdraw from the global nuclear non-proliferation treaty if sanctions were reimposed. Israeli Foreign Minister Gideon Saar has urged European powers to trigger the mechanism. Israel's June 13 attack on Iran came two days before Tehran and Washington were scheduled to meet for a sixth round of nuclear negotiations. On June 22, the US joined Israel's offensive by striking Iranian nuclear facilities at Fordo, Isfahan, and Natanz. Before the war, the US and Iran were divided over uranium enrichment – with Tehran describing it as a 'non-negotiable' right, while Washington called it a 'red line'. The International Atomic Energy Agency (IAEA) says Iran is enriching uranium to 60 per cent purity – far above the 3.67 per cent cap under the 2015 deal and close to weapons-grade levels. Tehran has said it is open to discussing the rate and level of enrichment, but not the right to enrich uranium. A year after the US withdrew from the nuclear deal, Iran began rolling back its commitments, which had placed restrictions on its nuclear activities in exchange for sanctions relief. Israel and Western powers accuse Iran of pursuing nuclear weapons, a charge Tehran has repeatedly denied. 'New form' Iran insists it will not abandon its nuclear programme, with Foreign Minister Abbas Araghchi describing the position as 'unshakable'. Though he claimed enrichment had come to a halt because of 'serious and severe' damage to nuclear sites caused by US and Israeli strikes, the full extent of the damage sustained in the US bombing remains unclear. Mr Trump claimed at the time the sites had been 'completely destroyed', but US media reports based on Pentagon assessments cast doubt on the scale of destruction . Since the 12-day war, Iran has suspended cooperation with the IAEA, accusing it of bias and failing to condemn the attacks. Inspectors have since left the country but a technical team is expected to return in the coming weeks after Iran said future cooperation would take a 'new form'. Israel has warned it may resume strikes if Iran rebuilds facilities or moves toward weapons capability. Iran has pledged a 'harsh response' to any future attacks. AFP
Straits Times
an hour ago
- Straits Times
SpaceX probes for cause of Starlink's global satellite network outage
Find out what's new on ST website and app. A 2022 photo shows Ukrainians using a Starlink terminal after Russia's retreat from the Ukrainian city of Kherson. SAN FRANCISCO - SpaceX's Starlink satellite network was back up and running on July 25 as engineers hunted for the root cause of one of its biggest international outages the night before, a rare disruption for the powerful internet system set off by an internal software failure. Users in the US and Europe began experiencing the outage at around 3pm EDT on July 24 (3am on July 25 in Singapore), according to Downdetector, a crowdsourced outage tracker that said as many as 61,000 user reports to the site were made. In Ukraine, where troops rely heavily on Starlink for battlefield communications, the outage affected combat operations as service was 'down across the entire front,' said Major Robert Brovdi, the commander of Ukraine's drone forces. Starlink, active in roughly 140 countries and territories and used by a growing number of militaries and government agencies, is a key source of revenue for Mr Elon Musk's SpaceX. The network has grown rapidly since 2020 into a disruptive force in the satellite communications industry. Starlink acknowledged the outage on its X account on July 24 and said 'we are actively implementing a solution.' The service mostly resumed after 2.5 hours, Mr Michael Nicolls, SpaceX vice-president of Starlink Engineering, wrote on X. By 8pm, the company wrote on X that the 'network issue has been resolved, and Starlink service has been restored.' 'The outage was due to failure of key internal software services that operate the core network,' Mr Nicolls said, apologising for the disruption and vowing to find its cause. Top stories Swipe. Select. Stay informed. Singapore SMRT to pay lower fine of $2.4m for EWL disruption; must invest at least $600k to boost reliability Singapore MRT service changes needed to modify 3 East-West Line stations on Changi Airport stretch: LTA Singapore S'pore could have nuclear energy 'within a few years', if it decides on it: UN nuclear watchdog chief Life 'Do you kill children?': Even before independence, S'pore has always loved its over-the-top campaigns Singapore Lung damage, poor brain development, addiction: What vaping does to the body Singapore Tipsy Collective sues former directors, HR head; alleges $14m lost from misconduct, poor decisions Singapore Fine for couple whose catering companies owed $432,000 in salaries to 103 employees Singapore Kopi, care and conversation: How this 20-year-old helps improve the well-being of the elderly Mr Musk also apologised: 'Sorry for the outage. SpaceX will remedy root cause to ensure it doesn't happen again,' the SpaceX chief executive officer wrote on X. The outage was a rare hiccup for SpaceX's most commercially sensitive business. Experts speculated whether the service, known for its resilience and speedy development, was beset by a glitch, a botched software update or perhaps a cyberattack. Mr Doug Madory, an expert at the internet analysis firm Kentik, said such a sweeping global outage was unusual. 'This is likely the longest outage ever for Starlink, at least while it became a major service provider,' Mr Madory said. As Starlink amasses more than six million users, SpaceX has focused in recent months on updating its network to accommodate demands for higher speed and bandwidth. The company, in a partnership with T-Mobile, is also expanding the constellation with larger, more powerful satellites to offer direct-to-cell text messaging services, a line of business in which mobile phone users can send emergency text messages through the network in rural areas. SpaceX has launched more than 8,000 Starlink satellites since 2020, building a uniquely distributed network in low-Earth orbit that has attracted intense demand from militaries, transportation industries and consumers in rural areas with poor access to traditional, fiber-based internet. 'I'd speculate this is a bad software update, not entirely dissimilar to the CrowdStrike mess with Windows last year, or a cyberattack,' said Dr Gregory Falco, director of a space and cybersecurity laboratory at Cornell University. An update to CrowdStrike's widely used cybersecurity software led to worldwide flight cancellations and impacted industries around the globe in July 2024. The outage disrupted internet services, affecting 8.5 million Microsoft Windows devices. It was unclear whether the July 25 outage affected SpaceX's other satellite-based services that rely on the Starlink network. Starshield, the company's military satellite business unit, has billions of dollars' worth of contracts with the Pentagon and US intelligence agencies. Separately, Reuters reported on July 25 that Mr Musk ordered a partial shutdown of Starlink during a pivotal push by Ukraine to retake territory in its war with Russia in late September 2022. REUTERS
CNA
an hour ago
- CNA
Spear AI raises first round of funding to apply AI to submarine data
SAN FRANCISCO :A startup founded by U.S. Navy veterans aiming to help the U.S. military use artificial intelligence to decipher data gathered by submarines has raised its first round of outside capital. Washington-based Spear AI specializes in working with what is known as passive acoustic data, which is gathered by listening devices underwater. Its long-term aim is to use AI to help submarine operators understand whether an object heard could be a rain squall, a whale, or a vessel that could be a threat, and to detect where it is and how fast it is moving. The challenge is that most existing AI tools are trained on data such as words or images that have been painstakingly labeled and organized over years or decades by companies such as Scale AI, which recently signed a $14.8-billion deal with Meta Platforms. Data from acoustic sensors is different. Spear AI co-founders Michael Hunter, a former U.S. Navy SEAL analyst, and John McGunnigle, a former nuclear submarine commander for the U.S. Navy, are building a hardware and software platform that aims to prepare that data for AI algorithms. The company sells sensors that can be attached to buoys or vessels and a software tool to help label and sort the data gathered by the sensors to make it ready to be put into AI systems. The U.S. Navy this month awarded Spear AI a $6-million contract for its data-labeling tool. Spear AI, founded in 2021, has been self-funded and has about 40 employees. Hunter, the CEO, said it raised $2.3 million from AI-focused venture firm Cortical Ventures and private equity firm Scare the Bear. The funding will be used to double the company's headcount to support its government contracts and commercial business prospects, such as monitoring underwater pipelines and cables. Hunter said Spear AI also aims to sell consulting services, a model similar to defense tech firm Palantir. "We wanted to build the product and actually get it out the door before the contract came in to get it," Hunter told Reuters. "The only way you can do that is with private capital."
