Improvements in ‘reasoning' AI models may slow down soon, analysis finds
An analysis by Epoch AI, a nonprofit AI research institute, suggests the AI industry may not be able to eke massive performance gains out of reasoning AI models for much longer. As soon as within a year, progress from reasoning models could slow down, according to the report's findings.
Reasoning models such as OpenAI's o3 have led to substantial gains on AI benchmarks in recent months, particularly benchmarks measuring math and programming skills. The models can apply more computing to problems, which can improve their performance, with the downside being that they take longer than conventional models to complete tasks.
Reasoning models are developed by first training a conventional model on a massive amount of data, then applying a technique called reinforcement learning, which effectively gives the model 'feedback' on its solutions to difficult problems.
So far, frontier AI labs like OpenAI haven't applied an enormous amount of computing power to the reinforcement learning stage of reasoning model training, according to Epoch.
That's changing. OpenAI has said that it applied around 10x more computing to train o3 than its predecessor, o1, and Epoch speculates that most of this computing was devoted to reinforcement learning. And OpenAI researcher Dan Roberts recently revealed that the company's future plans call for prioritizing reinforcement learning to use far more computing power, even more than for the initial model training.
But there's still an upper bound to how much computing can be applied to reinforcement learning, per Epoch.
According to an Epoch AI analysis, reasoning model training scaling may slow down.
Image Credits:Epoch AI
Josh You, an analyst at Epoch and the author of the analysis, explains that performance gains from standard AI model training are currently quadrupling every year, while performance gains from reinforcement learning are growing tenfold every 3-5 months. The progress of reasoning training will 'probably converge with the overall frontier by 2026,' he continues.
Techcrunch event
Exhibit at TechCrunch Sessions: AI Secure your spot at TC Sessions: AI and show 1,200+ decision-makers what you've built — without the big spend. Available through May 9 or while tables last.
Exhibit at TechCrunch Sessions: AI Secure your spot at TC Sessions: AI and show 1,200+ decision-makers what you've built — without the big spend. Available through May 9 or while tables last.
Berkeley, CA
|
BOOK NOW
Epoch's analysis makes a number of assumptions, and draws in part on public comments from AI company executives. But it also makes the case that scaling reasoning models may prove to be challenging for reasons besides computing, including high overhead costs for research.
'If there's a persistent overhead cost required for research, reasoning models might not scale as far as expected,' writes You. 'Rapid compute scaling is potentially a very important ingredient in reasoning model progress, so it's worth tracking this closely.'
Any indication that reasoning models may reach some sort of limit in the near future is likely to worry the AI industry, which has invested enormous resources developing these types of models. Already, studies have shown that reasoning models, which can be incredibly expensive to run, have serious flaws, like a tendency to hallucinate more than certain conventional models.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Entrepreneur
40 minutes ago
- Entrepreneur
A Legacy of Service, Strength, and Self-Made Success
His name carries weight not just in business circles, but also within the walls of public service departments, corporate boardrooms, and humanitarian organizations across the country. Opinions expressed by Entrepreneur contributors are their own. You're reading Entrepreneur Asia Pacific, an international franchise of Entrepreneur Media. In the heart of America's industrial, civic, and philanthropic landscapes stands Michael Benner—a man whose journey from hands-on law enforcement supplier to nationally recognized entrepreneur speaks volumes of perseverance, patriotism, and purposeful innovation. His name carries weight not just in business circles but also within the walls of public service departments, corporate boardrooms, and humanitarian organizations across the country. The Man Who Rebuilt Stewart Warner One of the most transformative chapters of Benner's life began with Stewart Warner Instruments—a company whose legacy dated back to Henry Ford's early automobile models. In the 1990s, the historic firm faced a steep decline under foreign ownership. With unwavering resolve, Benner and a small team of executives orchestrated a daring management buyout. They mortgaged homes, liquidated personal assets, and secured a $9 million credit line to reclaim the company and its heritage. Under Benner's leadership, Stewart Warner's on-time delivery rates surged from under 50% to over 90%, and backlogged orders were reduced from $2.1 million to just $100,000. The company also earned ISO 9001 certification, joining an elite class of global manufacturers recognized for quality in design and production. Through improved processes, better control of inventory, and decisive action, Benner led the firm back to profitability, preserving a brand that had once equipped Henry Ford's earliest automobiles. At the Forefront of Law Enforcement Innovation Benner's commitment to public service began even earlier, with the founding of Constable Equipment Company. He was a trailblazer in supplying cutting-edge radar systems and safety gear, including thousands of custom-fitted soft body armor vests to police departments across the country. His hands-on engagement—from training officers to attending national security conferences—earned him commendations from the FBI, the Missouri Chiefs of Police, and departments from Chicago to El Paso. Chicago's Police Department, under the leadership of Mayor Jane Byrne and Superintendent Richard Brezeczek, honored Benner with an autographed Jim Beam commemorative decanter with a message of appreciation for his personal efforts inscribed on the roof. His contributions were pivotal in securing over 13,000 body armor vests for officers—an effort that helped modernize safety standards in a rapidly evolving world. Giving with a Heart Full of Purpose Following the catastrophic impact of Hurricane Katrina, Benner again rose to the occasion—not in uniform or business attire, but as a citizen and philanthropist. He and his wife Kathy, donated an astonishing $517,817 to the American Red Cross of Central Florida in early 2006. Their gift directly aided over 2,000 displaced individuals and drew personal letters of appreciation from Red Cross leadership. This was no isolated act of generosity. The Benners' gift was one of the largest individual donations received during that time. So valued was their contribution that Red Cross leaders extended invitations to strategic planning meetings, hoping to gain from the couple's wisdom, vision, and community-first values. Their generosity became a beacon of hope amid disaster. A Pioneer of Purposeful Retirement What makes Michael Benner's story uniquely compelling is not just what he built—but how and when he chose to step back. At the age of 50, after decades of relentless drive, Benner made the bold decision to withdraw from daily operations. Inspired by his family's history of health challenges, he opted to prioritize quality time with his grandchildren, manage his personal investments, and nurture the properties he had acquired across Illinois, Florida, Minnesota, Texas, and Arizona. "I wanted to have as much quality time as possible before dealing with serious health issues," Benner once said. And his foresight proved true. His decision gave him more than 25 years of active, fulfilling life before facing a series of health issues in recent years, including bypass surgery and the onset of Myasthenia Gravis. His retirement was not retreat—it was a redirection. He managed income properties, stayed connected to civic efforts, and laid the foundation for a legacy that would inspire future entrepreneurs to value life as much as labor. A Living Legacy From a boy selling newspapers on Chicago's South Side to a CEO negotiating international operations, Michael Benner's journey is one of grit, grace, and gravitas. His story is one of few that can weave together frontline public safety, historic industrial turnaround, large-scale philanthropy, and personal transformation into a seamless narrative of the American spirit. Today, his name is not only tied to iconic brands and civic honors—it is etched in the lives of the officers he helped protect, the families he helped shelter, and the employees whose futures he fought to secure. Benner isn't just an author, a businessman, or a donor. He's a legacy-builder. And in that role, he continues to write the story of a life well-lived and a future yet to be defined.
The Hill
42 minutes ago
- The Hill
China blasts US for its computer chip moves and for threatening student visas
TAIPEI, Taiwan (AP) — China blasted the U.S. on Monday over moves it alleged harmed Chinese interests, including issuing AI chip export control guidelines, stopping the sale of chip design software to China, and planning to revoke Chinese student visas. 'These practices seriously violate the consensus' reached during trade discussions in Geneva last month, the Commerce Ministry said in a statement. That referred to a China-U.S. joint statement in which the United States and China agreed to slash their massive recent tariffs, restarting stalled trade between the world's two biggest economies. But last month's de-escalation in President Donald Trump's trade wars did nothing to resolve underlying differences between Beijing and Washington and Monday's statement showed how easily such agreements can lead to further turbulence. The deal lasts 90 days, creating time for U.S. and Chinese negotiators to reach a more substantive agreement. But the pause also leaves tariffs higher than before Trump started ramping them up last month. And businesses and investors must contend with uncertainty about whether the truce will last. U.S. Trade Representative Jamieson Greer said the U.S. agreed to drop the 145% tax Trump imposed last month to 30%. China agreed to lower its tariff rate on U.S. goods to 10% from 125%. The Commerce Ministry said China held up its end of the deal, canceling or suspending tariffs and non-tariff measures taken against the U.S. 'reciprocal tariffs' following the agreement. 'The United States has unilaterally provoked new economic and trade frictions, exacerbating the uncertainty and instability of bilateral economic and trade relations,' while China has stood by its commitments, the statement said. It also threatened unspecified retaliation, saying China will 'continue to take resolute and forceful measures to safeguard its legitimate rights and interests.' And in response to recent comments by Trump, it said of the U.S.: 'Instead of reflecting on itself, it has turned the tables and unreasonably accused China of violating the consensus, which is seriously contrary to the facts.' Trump stirred further controversy Friday, saying he will no longer be nice with China on trade, declaring in a social media post that the country had broken an agreement with the United States. Hours later, Trump said in the Oval Office that he will speak with Chinese President Xi Jinping and 'hopefully we'll work that out,' while still insisting China had violated the agreement. 'The bad news is that China, perhaps not surprisingly to some, HAS TOTALLY VIOLATED ITS AGREEMENT WITH US,' Trump posted. 'So much for being Mr. NICE GUY!' The Trump administration also stepped up the clash with China in other ways last week, announcing that it would start revoking visas for Chinese students studying in the U.S. U.S. campuses host more than 275,000 students from China. Both countries are in a race to develop advanced technologies such as artificial intelligence, with Washington seeking to curb China's access to the most advanced computer chips. China is also seeking to displace the U.S. as the leading power in the Asia-Pacific, including through gaining control over close U.S. partner and leading tech giant Taiwan.
Yahoo
44 minutes ago
- Yahoo
Not OK, computer: firms using AI to cut corners are playing with fire
The corporate world is agog. Ever since Eben Upton, the chief executive of Raspberry Pi, said he ran his annual results statement through AI before its publication, the talk has been of machines taking over the boardroom. The reaction to Upton's admission was astonishment. Raspberry Pi is stock market listed — these were its first full set of figures since flotation. They were eagerly awaited and, as with any quoted company, they were a closely guarded secret. Upton asked Claude, the AI bot designed by Amazon-funded Anthropic, to conduct a 'tone analysis'of the document, to say how it felt the microcomputer business was doing, on a scale of one to 100. Getting a so-so score, he set the computer to work. As the bot dialled up the language, the score improved. Too much, as it made his words seem breathlessly over the top. He made some improvements of his own, took out descriptions like 'exceptional' and reached an acceptable level. Eyebrows shot up on two counts. AI is a third-party, it's mechanical, susceptible to intrusion. It was not clear if he did but it is to be hoped Upton used a secure internal system. Then, there is the issue of the statement being entirely his — it is supposed to be his thoughts on the company's performance. Here he was, asking AI to look at what he planned to say. To be fair to Upton, he said in public what others may well be doing in private. Still, it was the most glaring instance yet of AI doing a boss's bidding. Others include a multinational senior executive freely saying he uses AI to draft his emails. An avatar of a CEO recently 'spoke' in a short video accompanying a stock exchange results announcement. Another corporate head told a tech conference how he uses AI to help prepare his speeches. While the software advances, the authorities stall. No regulation or guidance on AI's expansion and use is forthcoming. It is up to companies to make their own policies, not only to reap the benefits of AI but also to prevent a scandal and shareholder disaster. That is a worrying state of affairs. Specialist financial reporting and advisory consultancy Falcon Windsor teamed up with Insig AI, which delivers data infrastructure and AI-powered environmental, social and governance research tools, to look at the FTSE 350 companies. Their study, based on engagement with 40 firms and analysis of all FTSE 350 reports published from 2020 to 2024, revealed that generative AI use is multiplying across UK companies, often without any training, policy or oversight. They titled their report Your Precocious Intern, using the term to describe AI as useful but also a liability, the equivalent of someone who requires careful handling. While investors see the adoption of AI as inevitable and look forward to the advantages and efficiencies it could bring, they are increasingly alarmed about its implications for the truthfulness and authorship of corporate reporting. Everyone agrees that company reports and statements must remain the direct expression of management's thinking. Without rules and a common code, AI risks undermining the accuracy, authenticity and accountability that underpin trust in the stock markets. AI is moving so fast that there is only 'a short window of opportunity' to upskill and mitigate the risks it represents to the financial system. Their conclusion? 'Treat generative AI like a precocious intern: useful, quick, capable, but inexperienced, prone to overconfidence and should never be left unsupervised.' Claire Bodanis, a leading authority on UK corporate reporting and founder and director at Falcon Windsor, told The London Standard: 'If people use it unthinkingly, without proper training or guidelines, it could fatally undermine the accuracy and truthfulness of reporting.' Comments like these from two FTSE company secretaries should also be a warning. 'I think there are some real benefits in using generative AI as a summarising tool, and I'm quite keen to utilise it a bit more for efficiency if we can get comfortable with the accuracy of it,' said one. Another said: 'Would I be able hand on heart say that none of my contributors had used gen AI to provide the bit they've sent in? I have no idea.' Institutional investors are understandably afraid. As one told the researchers: 'I would be very wary about AI being used in forward[1]looking statements, or anything that is based around an opinion or a judgment.' Another said: 'I see generative AI as a flawed subordinate who's learning the ropes.' A third said: 'I feel very strongly that there should be a notification in the annual report if there's anything that has not been written by a human — there's no accountability through generative AI.' According to Bodanis, Raspberry Pi ought to act as a wake-up call. She asked: 'If a director gets AI to decide what is his or her opinion of their results based on what people are likely to think, then how is that honestly and truthfully their opinion?' History tells us, she said, what can happen. 'You think back to those stock market bubbles. Companies have to account to investors what they've done with their money and what they are going to do with it.' There must, said Bodanis, be 'a building of trust between a company and its shareholders'. One issue is the amount of material companies are obliged to produce. Annual reports that had grown to 80 pages, which felt huge, can reach 300 pages. That is because of the amount of non-financial reporting they must provide — on issues such as climate change, for example. If CEOs are using AI, it's difficult to decide what's true and what's not Claire Bodanis 'They are expected to use detail and opinion to create the truth of the state of the company,' said Bodanis. 'But if they are using AI, it is very difficult to decide what is true and what is not.' Just when corporate reporting is becoming 'ever more onerous and important', supplying all manner of information by law, along comes AI to make it easier. 'We should be using AI to do things humans can't do like crunch the numbers, not using it to do the things humans can do, like express opinions,' says Bodanis. A company report, she said, 'should be like looking the chairman in the eye and hearing it from them direct'. The slippery slope, too, is that distinction is lost. All company communications end up resembling each other — with the same wording and descriptions — when they are meant to be unique, coming straight from the top. The Financial Reporting Council, which regulates financial reporting and accounting, is dragging its heels, thinking about what to do about generative AI but so far not doing anything to police its rise. The FRC last got in touch with company boards about where it thought AI was heading in relation to results and reports some 18 months ago. That feels like a lifetime, such is AI's acceleration. As for companies uploading their sensitive figures to AI, Bodanis's point is succinct: 'AI has not signed an NDA. Error in retrieving data Sign in to access your portfolio Error in retrieving data Error in retrieving data Error in retrieving data Error in retrieving data
