Groundbreaking BBC research shows issues with over half the answers from Artificial Intelligence (AI) assistants
New BBC research published today provides a warning around the use of AI assistants to answer questions about news, with factual errors and the misrepresentation of source material affecting AI assistants.
The findings are concerning, and show:
51% of all AI answers to questions about the news were judged to have significant issues of some form
19% of AI answers which cited BBC content introduced factual errors – incorrect factual statements, numbers and dates
13% of the quotes sourced from BBC articles were either altered or didn't actually exist in that article.
The study, conducted over a month, saw the BBC test four prominent, publicly available AI assistants – OpenAI's ChatGPT; Microsoft's Copilot; Google's Gemini; and Perplexity. These AI assistants were given access to the BBC's website and asked questions about the news, prompting them to use BBC News articles as sources where possible. AI answers were reviewed by BBC journalists, all experts in the question topics, on criteria including accuracy, impartiality and how they represented BBC content.
Pete Archer, Programme Director for Generative AI at the BBC says: 'We're excited about the future of AI and the value it can bring audiences. We have already used it to add subtitles to programmes on BBC Sounds and translate content into different languages on BBC News. AI can bring real value if used responsibly.'
'But AI is also bringing significant challenges for audiences. People may think they can trust what they're reading from these AI assistants, but this research shows they can produce responses to questions about key news events that are distorted, factually incorrect or misleading. The use of AI assistants will grow so it's critical the information they provide audiences is accurate and trustworthy.'
'Publishers, like the BBC, should have control over whether and how their content is used and AI companies should show how assistants process news along with the scale and scope of errors and inaccuracies they produce. This will require strong partnerships between AI and media companies and new ways of working that put the audience first and maximise value for all. The BBC is open and willing to work closely with partners to do this.'
Some examples of the significant problems identified in responses from these AI assistants include:
ChatGPT and Copilot claimed that former Prime Minister Rishi Sunak and former First Minister Nicola Sturgeon were still in office after they had left.
Gemini incorrectly stated that 'The NHS advises people not to start vaping, and recommends that smokers who want to quit should use other methods.' In fact, the NHS does recommend vaping as a method to quit smoking.
A Perplexity response on the escalation of conflict in the Middle East, giving BBC as its source, said Iran initially showed 'restraint' and described Israel's actions as 'aggressive' – yet those adjectives hadn't been used in the BBC's impartial reporting.
The full research can be found on the BBC website.
Read more: Article from CEO, BBC News and Current Affairs, Deborah Turness
IW
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
The Independent
an hour ago
- The Independent
Apple teases features for new iPhones at conference
Apple has introduced a "Liquid Glass" design for all its products, featuring transparent and dynamic menus that react to movement, inspired by visionOS on the Vision Pro. Apple is rebranding its operating systems to be named after the year of release, such as iOS 26 instead of iOS 19, aligning all systems under a unified naming convention. Apple is opening its foundational AI model for third-party developers and integrating OpenAI's ChatGPT into its Image Playground app. New iPhone features include "Call Screening", which automatically answers unknown calls and transcribes the caller's purpose, and live translation for phone calls, compatible with non- iPhones. Apple's Visual Intelligence app will be enhanced to analyse screen items and link them with apps, allowing users to find similar products online through installed apps.
Telegraph
2 hours ago
- Telegraph
Public sector employment swells to highest level in 14 years
Public sector employment has surged to the highest level in 14 years as Rachel Reeves prepares to unveil a £300bn spending spree this week. Almost 6.2m people were employed in the public sector in March, official figures show, 35,000 more than a year earlier. This is the highest number of public sector employees since December 2011. The figures from the Office for National Statistics came ahead of Ms Reeves's spending review on Wednesday, which is expected to offer big increases to defence and health while squeezing other departments. The Chancellor has raised departmental spending by nearly £400bn since Labour won the election. It comes as economists have warned more tax rises are 'inevitable' in autumn. The figures from the ONS also show that the number of civil servants is the highest since 2006, at 550,000, rising by 6,000 from a year earlier. This helped to push the total figure of central government workers to a record high of 4m, up by 93,000 from a year ago. The ONS said the rise was driven by the NHS, the Civil Service and some local authority schools becoming academies, which changes how their staff are classified in the numbers. While public sector hiring surged, the jobs downturn across the economy deepened as firms grappled with big tax and minimum wage hikes. The number of vacancies fell from 760,000 on average across February to April to 736,000 for the three months to May.
Daily Mail
3 hours ago
- Daily Mail
EXCLUSIVE Starling launches AI chatbot Spending Intelligence in UK banking first
Finding out how much money you spent on coffee last month will soon become a faster exercise than trawling through bank statements or paying for a budgeting app to work it out. That's because Starling Bank is ramping up its use of Artificial Intelligence for current account customers. It has become the first bank in the UK to integrate its very own Large Language Model (LLM) AI chatbot within its banking app. The chatbot, called Spending Intelligence, uses LLMs to answer questions customers pose to it about their spending. Starling's chatbot is powered by Gemini, Google's LLM and answer to OpenAI's ChatGPT, a move which made sense for Starling as the bank houses its tech with Google Cloud Platform. It's available to all customers, both personal and business accounts - sole and joint - from today. Spending Intelligence appears as a search bar and sits at the top of Starling's legacy 'spending' area within Starling's banking app. The spending area gives customers information about how much they spend, where they spend it and when. It can be split up into more than 50 categories which customers can customise. Starling's AI chatbot takes the spending area a step further by providing answers to customers about their spending within a specific period of time, for example within the last month or year, and showing them a breakdown. Customers can ask it questions like 'how much did I spend on groceries last week?' or 'how much did I donate to charity last year?' These questions can either be typed into the search bar or customers can ask the chatbot verbally. Spending Intelligence can only draw insights from Starling accounts and gives answers based on this data. The chatbot cannot tell customers what to do with their money - for example where to invest it or how to save more money - as this would fall into the remit of regulated advice. But the idea is to get customers to think about how they are spending. The hope is that it will help customers to see exactly where their money goes The Starling chatbot is free to use and chief information officer Harriet Rees says there are 'absolutely no plans to make it paid for use'. More AI coming down the line for Starling The Spending Intelligence chatbot is the first step towards a bigger ambition to implement AI within its banking app. The hope is it will nudge customers towards using more of its existing money management features as well as new tools Starling develops in the future. Rees says: 'Customers can use AI to feed their natural curiosity about their finances so that they can make informed decisions about their budgeting, and better utilise Starling's suite of money management tools.' While customers will learn about their spending habits by using the chatbot, at the same time Starling will be learning from its customers through the questions they ask to identify areas where it can roll out more AI in the future. 'This is the first step for us of putting AI in the hands of customers' Rees tells This is Money. 'But there's so much further we could go' she adds. These could include fraud detection and new customer onboarding according to Rees. The information Starling receives from customers through the questions they ask cannot be used by Google's Gemini to feed its Large Language Models, this is part of Starling's agreement with Google. Starling said: 'Customers have to opt in to use Spending Intelligence. It does not store sensitive information and customers can opt out at any time.' In the future Spending Intelligence could potentially evolve into telling customers how they could save money if they were to ask the chatbot to identify areas. But for the moment it is focused solely on spending. Rees says: 'We're having a think about how you could save. But we genuinely believe giving customers insights about how they are spending is the best way to get them thinking about where they could save.'
