DeepSeek's chatbot achieves 17% accuracy, trails Western rivals in NewsGuard audit
Reuters
Chinese AI startup DeepSeek's chatbot achieved only 17% accuracy in delivering news and information in a NewsGuard audit that ranked it tenth out of eleven in a comparison with its Western competitors including OpenAI's ChatGPT and Google Gemini.
The chatbot repeated false claims 30% of the time and gave vague or not useful answers 53% of the time in response to news-related prompts, resulting in an 83% fail rate, according to a report published by trustworthiness rating service NewsGuard on Wednesday.
That was worse than an average fail rate of 62% for its Western rivals and raises doubts about AI technology that DeepSeek has claimed performs on par or better than Microsoft-backed OpenAI at a fraction of the cost.
Within days of its roll-out, DeepSeek's chatbot became the most downloaded app in Apple's AAPL.O App Store, stirring concerns about United States' lead in AI and sparking a market rout that wiped around $1 trillion off U.S. technology stocks.
The Chinese startup did not immediately respond to a request for comment.
Need a break? Play the USA TODAY Daily Crossword Puzzle.
Artificial intelligence wars:American AI firms try to poke holes in disruptive DeepSeek
NewsGuard said it applied the same 300 prompts to DeepSeek that it had used to evaluate its Western counterparts, which included 30 prompts based on 10 false claims spreading online.
Topics for the claims included last month's killing of UnitedHealthcare executive Brian Thompson and the downing of Azerbaijan Airlines flight 8243.
NewsGuard's audit also showed that in three of the ten prompts, DeepSeek reiterated the Chinese government's position on the topic without being asked anything relating to China.
On prompts related to the Azerbaijan Airlines crash — questions unrelated to China — DeepSeek responded with Beijing's position on the topic, NewsGuard said.
"The importance of the DeepSeek breakthrough is not in answering Chinese news-related question accurately, it is in the fact that it can answer any question at 1/30th of the cost of comparable AI models," D.A. Davidson analyst Gil Luria said.
Like other AI models, DeepSeek was most vulnerable to repeating false claims when responding to prompts used by people seeking to use AI models to create and spread false claims, NewsGuard added.
Reporting by Rishi Kant in Bengaluru; Editing by Shounak Dasgupta
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Yahoo
26 minutes ago
- Yahoo
OpenAI CFO sees firm selling AI infrastructure service in future
(Bloomberg) — OpenAI ( is considering eventually helping other businesses tap into the data centers and physical infrastructure needed for artificial intelligence, potentially creating a new revenue line that could offset some of the ChatGPT maker's immense costs. The service would be loosely inspired by the success Inc. found renting out its spare cloud computing capacity to companies, OpenAI Chief Financial Officer Sarah Friar said in an interview Wednesday. OpenAI is not 'actively looking' at such an effort today because it's focused on securing computing capacity for its own operations, she said. But 'I do think about it as a business down the line, for sure,' Friar said. In recent years, OpenAI has gained expertise on how to design and set up data centers to optimize workloads for AI. The company now sees an opportunity to capitalize on that know-how. It also wants to become more directly involved in the process rather than rely solely on third-party vendors. 'If all we do is buy from others, all we're doing is giving them our IP because they're learning how to build AI infrastructure,' Friar said. OpenAI has raised tens of billions of dollars to pay for advanced chips and data centers to build and operate cutting-edge AI services. The company is also working with SoftBank Group Corp. and Oracle Corp. on an ambitious infrastructure venture called Stargate, with plans to build massive data centers in the US and abroad. Even that may be a drop in the bucket. Chief Executive Officer Sam Altman told reporters this month that people 'should expect OpenAI to spend trillions of dollars' on data center construction in the 'not very distant future.' Altman said OpenAI is working to 'design a very interesting new kind of financial instrument' to help support that effort, without providing details. Historically, OpenAI has leaned on other partners, including backer Microsoft Corp. and Oracle, to help finance the cost of building out data centers. Now, the company is also seeing banks and private equity firms 'come to the table' with debt financing to support its infrastructure initiatives, Friar said. 'That's the next path we're going down,' Friar said. From there, she added, OpenAI is 'trying to be thoughtful' about whether there are 'other interesting, novel ways we could do that beyond debt.' OpenAI remains an unprofitable business, which limits its ability to finance data center projects without outside capital. However, the company has seen revenue grow at a fast clip on the strength of consumer and business demand for ChatGPT. OpenAI generated $1 billion in revenue in July, marking its first month hitting that milestone, Friar said. The company is also in early talks about a potential sale of stock for current and former employees at a valuation of about $500 billion, Bloomberg News has reported. Its previous valuation stood at $300 billion in a $40 billion financing round led by SoftBank. Of that $40 billion, OpenAI was responsible for lining up $10 billion through a syndicate of investors. Friar said there was so much investor demand that OpenAI ended up raising $11 billion, bringing the total funding amount for the current round to $41 billion. ©2025 Bloomberg L.P.
Engadget
27 minutes ago
- Engadget
Microsoft is working on a fix for PC shader stutter
Microsoft is creating a new "advanced shader delivery" feature for the ASUS ROG Xbox Ally handhelds that might make loading games faster and more stutter-free. The company teased the upcoming feature alongside the announcement of the launch date for the first Xbox-branded handhelds and the company's new compatibility system for figuring out if games will run well on handhelds. Shader stutter is a widespread issue in PC gaming that doesn't occur on consoles because of their uniform hardware. You have to download shaders for a PC game to make sure that they run correctly on your specific hardware. Typically, a game downloads shaders when they're first launched, which can add to the wait before you can actually start playing, and lead to things like framerate drops as shaders are compiled in the background. Microsoft's system "preloads game shaders during download" which the company says will allow games to launch up to 10 times faster, "run smoother, and use less battery on first play." It sounds like this new shader delivery system will require some amount of buy-in from developers, which means it likely won't impact every game at launch. It's just one of the ways Microsoft is trying to address the complexity and inefficiency Windows 11 can add to gaming. Besides including easier access to features like the Game Bar and ASUS' Armoury Crate software, the company also says that its handheld version of Windows will "minimize background activity and defer non-essential tasks" to devote more resources to playing games. Microsoft wants theses handhelds to be a success, so it makes sense to put in the work. But in a world where Xbox home consoles look even more like Windows PCs, the company might also be making improvements now that it can take advantage of later.
The Hill
27 minutes ago
- The Hill
Huckabee blames Europe for stalled Gaza talks, highlighting a growing Western divide
JERUSALEM (AP) — U.S. Ambassador to Israel Mike Huckabee sought Wednesday to blame a recent breakdown in Gaza ceasefire talks on the decision by some European leaders to recognize Palestinian statehood. Talks over a lasting ceasefire have repeatedly stalled since the early months of the war sparked by Hamas' Oct. 7, 2023, attack — and long before there was any talk of major European states recognizing Palestinian statehood. The decisions were announced by France, Britain and other countries after the Trump administration's Mideast envoy had already walked away in frustration from the negotiations, which happened behind closed doors. It's unclear how and when they began to break down. But Huckabee's remarks in an interview with The Associated Press point to a sharp divide among Western nations about how to approach the Israeli-Palestinian conflict, and the rift has only deepened since President Donald Trump took office. Many European countries have sought to pressure Israel — the stronger party — and frame the pursuit of a two-state solution as a way to address the root causes of a conflict that long predates the war in Gaza. But the Trump administration has given Israel wide latitude to end the war on its terms. A dispute over which side should be pressured The White House fully supported Israeli Prime Minister Benjamin Netanyahu's decision to end an earlier ceasefire that Trump had helped broker and resume wide-ranging military operations, during which Israel prevented food, medicine and other goods from entering Gaza for more than two months. Trump appears to have adopted Israel's position that further military pressure — including a planned offensive into some of the most densely populated areas of Gaza — will force Hamas to surrender. 'We will only see the return of the remaining hostages when Hamas is confronted and destroyed!!!' Trump posted this week on his Truth Social site. Meanwhile, France, Britain and more than two dozen other Western-aligned nations have expressed mounting outrage at Israel's actions, demanding that it halt the fighting and do more to facilitate humanitarian aid. The moves to recognize Palestinians statehood — which were largely symbolic — were in part aimed at pressuring Israel to halt its offensive. Britain explicitly linked the two, saying it would hold off on recognizing a Palestinian state if Israel agreed to a ceasefire in Gaza, stopped building settlements in the West Bank and committed to a two-state solution. Israel's current government and most of its political class were opposed to Palestinian statehood even before the war, and they now say it would reward Hamas and allow the militants to eventually carry out more Oct. 7-style attacks. 'A counterproductive effect' Huckabee, who is himself a longtime opponent of Palestinian statehood, said the 'noise that has been made by European leaders recently … is having the counterproductive effect that they probably think that they want.' 'If they believe that unilaterally calling for a two-state, a Palestinian state recognition, immediately brings them closer, the sad truth is it's taking them further away,' he said. The AP sought comment from the foreign ministries of France and Britain, which did not immediately respond. Most Palestinians believe the decades-old conflict is rooted in Israel's military occupation of lands they want for an independent state and its continuous expansion of Jewish settlements. Attempts to negotiate a two-state solution going back to the early 1990s repeatedly broke down as violence flared, and no serious or substantive talks have been held since Netanyahu returned to office in 2009. On Wednesday, Israel approved plans for settlements in an area known as E1 outside Jerusalem, which the Palestinians and others say will cut the West Bank in half and make the establishment of a viable and contiguous state virtually impossible. Past U.S. administrations joined their Western allies in vehemently opposing such plans. Trump and Huckabee — a strong supporter of Israeli settlements — did not. Ceasefire talks have shown signs of progress The ceasefire talks have continued, with Hamas saying this week that it accepted a proposal from Arab mediators that is similar to an earlier one advanced by the U.S. and accepted by Israel. Israel and the U.S. have not yet responded to the offer. Huckabee said he could not elaborate on the talks beyond saying he would not trust that Hamas is serious until they 'put their signatures on it' and that the group could be 'stringing this along,' as the U.S. and Israel have alleged in the past. The main disagreement has been over the conditions in which the war would end. Hamas has said it will only release the remaining hostages in exchange for a lasting ceasefire and an Israeli withdrawal from Gaza, while Israel says it is committed to returning all the hostages, defeating and disarming Hamas, and maintaining lasting security control over the territory. In Israel, many blame Netanyahu for the failure of the talks, accusing him of seeking to prolong the war in order to keep his far-right governing coalition intact. Hundreds of thousands took to the streets over the weekend to protest his planned offensive and call for a deal to return the captives.
