DeepSeek's chatbot achieves 17% accuracy, trails Western rivals in NewsGuard audit
Rishi Kant
Reuters
Chinese AI startup DeepSeek's chatbot achieved only 17% accuracy in delivering news and information in a NewsGuard audit that ranked it tenth out of eleven in a comparison with its Western competitors including OpenAI's ChatGPT and Google Gemini.
The chatbot repeated false claims 30% of the time and gave vague or not useful answers 53% of the time in response to news-related prompts, resulting in an 83% fail rate, according to a report published by trustworthiness rating service NewsGuard on Wednesday.
That was worse than an average fail rate of 62% for its Western rivals and raises doubts about AI technology that DeepSeek has claimed performs on par or better than Microsoft-backed OpenAI at a fraction of the cost.
Within days of its roll-out, DeepSeek's chatbot became the most downloaded app in Apple's AAPL.O App Store, stirring concerns about United States' lead in AI and sparking a market rout that wiped around $1 trillion off U.S. technology stocks.
The Chinese startup did not immediately respond to a request for comment.
Need a break? Play the USA TODAY Daily Crossword Puzzle.
Artificial intelligence wars:American AI firms try to poke holes in disruptive DeepSeek
NewsGuard said it applied the same 300 prompts to DeepSeek that it had used to evaluate its Western counterparts, which included 30 prompts based on 10 false claims spreading online.
Topics for the claims included last month's killing of UnitedHealthcare executive Brian Thompson and the downing of Azerbaijan Airlines flight 8243.
NewsGuard's audit also showed that in three of the ten prompts, DeepSeek reiterated the Chinese government's position on the topic without being asked anything relating to China.
On prompts related to the Azerbaijan Airlines crash — questions unrelated to China — DeepSeek responded with Beijing's position on the topic, NewsGuard said.
"The importance of the DeepSeek breakthrough is not in answering Chinese news-related question accurately, it is in the fact that it can answer any question at 1/30th of the cost of comparable AI models," D.A. Davidson analyst Gil Luria said.
Like other AI models, DeepSeek was most vulnerable to repeating false claims when responding to prompts used by people seeking to use AI models to create and spread false claims, NewsGuard added.
Reporting by Rishi Kant in Bengaluru; Editing by Shounak Dasgupta
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Yahoo
17 minutes ago
- Yahoo
How major US stock indexes fared Monday, 6/9/2025
U.S. stocks drifted closer to their record as the world's two largest economies begin talks on trade that could help avoid a recession. The S&P 500 rose 0.1% Monday and is 2.3% below the record it reached in February. The Dow Jones Industrial Average was flat, and the Nasdaq composite rose 0.3%. Markets are waiting to hear what comes of trade talks between the United States and China taking place in London. Treasury yields slipped after a survey suggested consumers' expectations for coming inflation eased. Chinese stocks rose, while indexes were mixed across the rest of Asia and Europe. On Monday: The S&P 500 rose 5.52 points, or 0.1%, to 6,005.88. The Dow Jones Industrial Average fell 1.11 points, or less than 0.1%, to 42,761.76. The Nasdaq composite rose 61.28 points, or 0.3%, to 19,591.24. The Russell 2000 index of smaller companies rose 12.20 points, or 0.6%, to 2,144.45. For the year: The S&P 500 is up 124.25 points, or 2.1%. The Dow is up 217.54 points, or 0.5%. The Nasdaq is up 280.44 points, or 1.5%. The Russell 2000 is down 85.71 points, or 3.8%. Sign in to access your portfolio
New York Post
31 minutes ago
- New York Post
Apple admits to delays in Siri AI overhaul at lackluster WWDC presentation: ‘Overall a yawner'
Apple admitted it needs more time to complete its long-delayed overhaul of the Siri voice assistant – the latest sign that CEO Tim Cook is struggling to meet investor demands for innovation in artificial intelligence. The Cupertino, Calif.-based firm revealed bumps in AI development on Monday at its Worldwide Developers Conference, a yearly event that used to represent the cutting edge with software engineers — but which lately has lost its cachet. 'This work needed more time to reach our high quality bar and we look forward to sharing more about it in the coming year,' Apple software chief Craig Federighi said regarding the delays to Siri's AI overhaul. Advertisement 5 Apple held its annual Worldwide Developers Conference on Monday. Getty Images Apple unveiled plans to allow app developers to use the large language model that powers its AI systems, dubbed 'Apple Intelligence.' Apple also announced a sweeping design overhaul of its operating systems featuring a sleek new look called 'Liquid Glass.' Wedbush analyst Dan Ives, an Apple bull, said the event showed 'slow and steady improvements' but was 'overall a yawner.' Advertisement 'WWDC laid out the vision for developers BUT was void of any major Apple Intelligence progress as Cupertino is playing it safe and close to the vest after the missteps last year,' Ives said in a note to clients. During the event, Apple touted a 'live translation' tool that works for real-time conversations. The company also plans to rename its operating systems updates based on the year they are released, rather than the sequential numbers they've used in the past. 5 Apple shares fell slightly in trading during the presentation. AP Elsewhere, Apple unveiled a 'Call Screening' in which iPhones will automatically answer calls from unknown numbers and ask the caller why they called. The company is also partnering with OpenAI's ChatGPT to add image generation to its 'Image Playground' app. Advertisement Apple shares were down more than 1% during the keynote presentation. 'Has there actually been an exciting WWDC in the last decade?' X user @MikeYelovich wrote in a post. 'This feels like what ChatGPT thinks people want from Apple.' Analysts had widely expected this year's event to be underwhelming, with minor software improvements, such as an AI-powered writing assistant, taking center stage. 5 Apple software chief Craig Federighi said the Siri enhancements 'needed more time. Getty Images Advertisement 'There's no way Writing Tools is the first apple intelligence thing they brag about at wwdc,' joked Interaction co-founder Marvin von Hagen. 'Every single time i've seen this menu was against my will.' The company also detailed enhancements to its 'Visual Intelligence' app, which allows users to seamlessly identify products through their iPhone cameras. Federighi said the plan to open up Apple's 'Foundation Models framework' to developers and users would have a transformational effect – allowing them to use an advanced AI model without even needing an internet connection. 'This will ignite a whole new wave of intelligence,' said Federighi, who cited examples such as apps using the AI model to recommend nearby hiking trails or create practice exams for students. 5 Apple CEO Tim Cook addresses the crowd at WWDC. Getty Images 5 Apple was under pressure to show progress on AI development at WWDC. AP Despite the lukewarm reception, Apple is still in an 'enviable position given its large installed base of users' and still has time to catch up to its rivals, said Paolo Pescatore, an industry analyst and founder of research firm PP Foresight. Apple is focused on ensuring its major AI products are ready for prime time rather than risk a backlash from its fans. Advertisement 'While it might seem others are leading the AI race, it is not a sought-after feature among users and there's no revenue uplift – for now,' said Pescatore. 'Considering the negative perception, Apple needs to tread carefully not to frustrate and disappoint its loyal base of iPhone users.' Apple is locked in a fierce competition with rivals like Google and Sam Altman-led OpenAI – the latter of which recently acquired famed iPhone designer Jony Ive's firm with an eye toward developing its own hardware product built specifically for AI. With Post wires
Yahoo
34 minutes ago
- Yahoo
Russia makes a year's worth of NATO ammunition in 3 months, Rutte says
Russia produces as much ammunition in three months as NATO does in a year, posing serious risks for the alliance, NATO Secretary General Mark Rutte said on June 9 at Chatham House in London. "The capabilities of (Vladimir) Putin's war machine are speeding up, not slowing down," Rutte said. "Russia is reconstituting its forces with Chinese technology and producing more weapons faster than we thought." Rutte's statement comes amid mounting tensions between the alliance and Russia and an increasingly uncertain U.S. commitment to European security. According to Rutte, Russia is restoring its military potential with the help of China, Iran, and North Korea. This year, the Russian industrial complex is expected to roll out 1,500 tanks, 3,000 armored vehicles, and 200 Iskander missiles, NATO chief said. "Russia could be ready to use military force against NATO within five years," Rutte said. "Let's not kid ourselves." "We are all on the eastern flank now. The new generation of Russian missiles travels at the speed of sound. The distance between European capitals is only a matter of minutes. There is no longer east or west. There is just NATO," he added. When asked whether a long-term truce in Ukraine's war against Russia would allow the latter to stockpile even more weapons and increase the risks to NATO, Rutte said: "That's a statement of fact." "Our assumption at the moment is that even whilst the war against Ukraine is continuing, they (Russian forces) are still able to increase stockpiles slightly, but that's debatable," Rutte added. Russia's full-scale invasion of Ukraine in 2022 prompted European countries to hike defense spending in order to revive their military capabilities, which atrophied after decades of disarmament following the Cold War. The matter gained more urgency after signals that the U.S., the most powerful military in NATO, plans to scale down its presence in Europe as President Donald Trump shifts strategic focus to the Asia-Pacific region. Ukrainian foreign intelligence chief Oleh Ivashchenko recently warned that Russia will be able to replenish its forces between two and four years after the war in Ukraine, allowing it to launch aggression against Europe. Read also: 'Find and destroy' – how Ukraine's own Peaky Blinders mastered the art of bomber drones We've been working hard to bring you independent, locally-sourced news from Ukraine. Consider supporting the Kyiv Independent.
