Apple AI research shows reasoning models collapse when problems are more complex
A research paper from Apple published on June 6 stated that although large reasoning models (LRMs) showed improved performance on benchmarks, they struggled with accuracy when the problems became more complex. Titled, 'The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity,' the paper revealed that even the most advanced AI reasoning models collapsed entirely when facing harder problems.
'They exhibit a counter- intuitive scaling limit: their reasoning effort increases with problem complexity up to a point, then declines despite having an adequate token budget,' the paper noted.
To test the AI models, the researchers categorised the problems into low complexity, medium complexity and high complexity tasks which included a bunch of puzzles like Checkers Jumping, River Crossing, Blocks World and the Tower of Hanoi.
The researchers picked Claude 3.7 Sonnet and DeepSeek-V3 from among the large language models and the Claude 3.7 Sonnet with Thinking and DeepSeek-R1 from among the large reasoning models. The research concluded that both the types of AI models had a similar level of capability.
For low complexity problems, the models were found to solve the puzzles but as they proceeded to the high complex category, both AI models failed to work.
The hardware giant has been seen as lagging behind in developing AI technology. Notably, Apple's annual Worldwide Developers Conference is also expected to begin later today.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
The Hindu
29 minutes ago
- The Hindu
Apple WWDC 2025 LIVE: iPhone-maker needs to restore faith after Apple Intelligence rollout delays
Apple's Worldwide Developers Conference (WWDC) 2025 is set to begin on June 9, as watchers worldwide tune in to see how the iPhone-maker will navigate U.S. President Donald Trump's order to build products in the U.S. or face heavy tariffs if it continues to rely on China and India. While WWDC is mainly an event focusing on software upgrades and developer-centric launches, the impact of geopolitics on Apple's balance sheet will certainly be on most viewers' minds. Also Read:Apple may raise iPhone prices ahead of new launch If this was not enough pressure, Apple is facing mounting scrutiny over the company's inability to deploy long-awaited Apple Intelligence (AI) features, such as an AI-enhanced Siri assistant, within the promised timeline. Also Read:Apple under pressure to shine during WWDC 2025 after Siri AI stumble
Mint
an hour ago
- Mint
Apple Greets Developers at WWDC as an AI Spoilsport
(Bloomberg Opinion) -- You know, it could just be a coincidence. On the eve of Apple Inc.'s World Wide Developers' Conference, a new paper from the company's own researchers poured some cold water over some of the artificial-intelligence hype. The study argues that the advanced reasoning models, heralded by some as a new frontier for how AI 'thinks,' fall well short of expectations. When a problem becomes sufficiently complex, the team of six researchers wrote, the models suffered a 'complete accuracy collapse.' It examined top efforts from OpenAI, Google and and Anthropic — three AI makers considered several furlongs ahead of Apple in the AI race. You may be interested in As I say — perhaps it's just a coincidence. And the debate around the paper's conclusions are only just starting. But it's certainly useful that it arrived just as the iPhone maker's perceived failing to build competitive AI capabilities comes back into sharp focus as third-party Apple developers descend on San Jose on Monday for the annual pilgrimage, with many more joining remotely. WWDC isn't typically the venue for Apple's biggest product launches. But it is a chance for Apple to get developers hyped on some experimental ideas. The Vision Pro mixed-reality headset was unveiled at WWDC in 2023, and last year's event was about the long-awaited reveal of Apple's answer to ChatGPT and the rest: Apple Intelligence. The tone of this year's WWDC will be markedly different for one obvious reason: Apple has embarrassed itself. Many of the Apple Intelligence upgrades the company outlined in 2024 have yet to materialize on users' devices, with the company forced to quietly stop running ads that suggested the features were imminent. The bells and whistles that did get released were underwhelming, buggy or both. Backed into a corner, by Wall Street or just competitive instincts against its peers, Apple looked to be in a bit of a scramble. But it's yesterday's news to say that Apple is behind on AI. Investors know that, and the shortcomings have already been priced in; just one of several factors contributing to Apple's share price drop of 19% so far this year. (That would have made it the worst performer in the Magnificent Seven were it not for the Trump-Musk feud's fallout on Tesla Inc. last week.) Today's story is instead about whether Apple can convincingly talk its way around its glaring deficit. That job will fall on the shoulders of Chief Executive Officer Tim Cook, who could be forgiven for being distracted as Apple contends with the continued unpredictabilities of the continuing trade war. Instead of any significant AI announcements, according to Bloomberg News' chief Apple whisperer, Mark Gurman, attendees will be walked through a revised visual language for its apps, an upgrade to its Mac OS operating system, a new games offering and a smattering of other tweaks. There could, of course, be a 'one more thing' surprise, but it seems highly unlikely. More so than any significant Apple event in recent memory, this year's WWDC keynote seems set to be a snoozefest, a placeholder for more significant progress to come next year. Apple's muted WWDC will stand in stark contrast to the buzz created around the recent announcement that ex-Apple design guru Jony Ive was working on a device with OpenAI. That will take time to materialize, if it ever does. More urgent is the threat from Alphabet Inc.'s Google, Microsoft Corp. and Inc., which all seem closer to creating the breakthrough AI personal assistant that science fiction has promised us for so long. So it serves a useful purpose, then, to put out a research paper diminishing others' AI progress. Apple doesn't seem able to speed up, so it might as well take a shot at slowing others down. And indeed, we might look back in five or 10 years and concede Apple was entirely correct in its reservations. Taking that thought a step further, one way to spin the situation to its benefit, as discussed by analysts at Evercore ISI recently, might be for Apple executives to make more of the fact that while Apple hasn't produced any groundbreaking AI achievements compared with its peers, it hasn't thrown tens of billions of dollars at the pursuit either. More From Bloomberg Opinion: This column reflects the personal views of the author and does not necessarily reflect the opinion of the editorial board or Bloomberg LP and its owners. Dave Lee is Bloomberg Opinion's US technology columnist. He was previously a correspondent for the Financial Times and BBC News. More stories like this are available on
Mint
an hour ago
- Mint
Apple WWDC 2025 LIVE Updates: iOS 26 overhaul, SIRI upgrades and what more to expect
Apple WWDC 2025 Live updates: Apple is set to kick off its annual Worldwide Developers Conference (WWDC) at Apple Park in Cupertino later today, where the tech giant is expected to unveil a host of software upgrades across its device ecosystem. The keynote, scheduled to begin at 10 AM Pacific Time (10:30 PM IST), will be streamed live via YouTube, Apple's website, and the official Apple app, with playback available on demand afterwards. This year's WWDC arrives at a critical juncture for the company. Despite Apple's global dominance in hardware, it has struggled to keep pace with rivals in the rapidly evolving AI space. Its rollout of Apple Intelligence features has faced delays and inconsistent implementation across devices, prompting growing scrutiny. However, today's announcements aim to refocus the conversation around innovation and future strategy. Apple WWDC 2025 Live updates: Perhaps the most symbolic shift will likely be the change in how Apple labels its software versions. Instead of continuing with the traditional numerical progression (which would have introduced iOS 19), Apple is expected to align its operating system names with the calendar year. This means that the upcoming versions could be titled iOS 26, iPadOS 26, macOS 26, and so on—marking a fresh start and a more intuitive naming system for users and developers alike. According to Bloomberg's Mark Gurman, the most eye-catching change could be a comprehensive redesign of the user interface across Apple's platforms. Borrowing heavily from visionOS, the new UI is expected to incorporate translucent layers and more pronounced lighting effects—what has been described internally as 'digital glass'. This aesthetic will bring a cleaner, more futuristic appearance, with redesigned toolbars, tab layouts, and app icons. Users may also gain access to context-sensitive controls, with buttons triggering quick menus of relevant actions—streamlining interaction and improving productivity across apps. Apple WWDC 2025 Live updates: Apple is also reportedly revamping key apps such as Phone, Safari, and Camera, particularly on iOS and iPadOS. The Camera app could see enhanced manual controls and AI-powered image optimisation, while Safari may get smarter web content summarisation and reading modes. Notably, AI will play a more prominent behind-the-scenes role this year. While Apple has so far taken a more conservative approach to generative AI compared to competitors, it is expected to introduce system-level AI improvements. These include features aimed at extending battery life by intelligently managing background tasks and optimising app performance. Apple WWDC 2025 Live updates: While hardware is not expected to take centre stage at this year's event, WWDC 2025 may serve as a crucial moment for Apple to reassert its relevance in a rapidly shifting tech landscape. With a reimagined user interface, deeper AI integration, and a strategic reset in branding, Apple hopes to convince both developers and consumers that its next chapter is just beginning. More details will unfold as the event goes live later tonight.
