Groundbreaking BBC research shows issues with over half the answers from Artificial Intelligence (AI) assistants
New BBC research published today provides a warning around the use of AI assistants to answer questions about news, with factual errors and the misrepresentation of source material affecting AI assistants.
The findings are concerning, and show:
51% of all AI answers to questions about the news were judged to have significant issues of some form
19% of AI answers which cited BBC content introduced factual errors – incorrect factual statements, numbers and dates
13% of the quotes sourced from BBC articles were either altered or didn't actually exist in that article.
The study, conducted over a month, saw the BBC test four prominent, publicly available AI assistants – OpenAI's ChatGPT; Microsoft's Copilot; Google's Gemini; and Perplexity. These AI assistants were given access to the BBC's website and asked questions about the news, prompting them to use BBC News articles as sources where possible. AI answers were reviewed by BBC journalists, all experts in the question topics, on criteria including accuracy, impartiality and how they represented BBC content.
Pete Archer, Programme Director for Generative AI at the BBC says: 'We're excited about the future of AI and the value it can bring audiences. We have already used it to add subtitles to programmes on BBC Sounds and translate content into different languages on BBC News. AI can bring real value if used responsibly.'
'But AI is also bringing significant challenges for audiences. People may think they can trust what they're reading from these AI assistants, but this research shows they can produce responses to questions about key news events that are distorted, factually incorrect or misleading. The use of AI assistants will grow so it's critical the information they provide audiences is accurate and trustworthy.'
'Publishers, like the BBC, should have control over whether and how their content is used and AI companies should show how assistants process news along with the scale and scope of errors and inaccuracies they produce. This will require strong partnerships between AI and media companies and new ways of working that put the audience first and maximise value for all. The BBC is open and willing to work closely with partners to do this.'
Some examples of the significant problems identified in responses from these AI assistants include:
ChatGPT and Copilot claimed that former Prime Minister Rishi Sunak and former First Minister Nicola Sturgeon were still in office after they had left.
Gemini incorrectly stated that 'The NHS advises people not to start vaping, and recommends that smokers who want to quit should use other methods.' In fact, the NHS does recommend vaping as a method to quit smoking.
A Perplexity response on the escalation of conflict in the Middle East, giving BBC as its source, said Iran initially showed 'restraint' and described Israel's actions as 'aggressive' – yet those adjectives hadn't been used in the BBC's impartial reporting.
The full research can be found on the BBC website.
Read more: Article from CEO, BBC News and Current Affairs, Deborah Turness
IW
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Press and Journal
an hour ago
- Press and Journal
Why out of town location is perfect for Elgin business cashing in on town's housing boom
The owner of popular Elgin cafe The Deli, Claire Willox, has seen first hand the housing boom in the town. She believes having out of town premises on Southfield Drive has had major benefits for her firm with free parking, an ever-growing population on the doorstep and nearby businesses for customers. In 2017, she bought the lunch stop with a business partner after more than 30 years in the flooring industry. Later down the line, she took over full ownership. The premises can hold up to 50 people. Claire said: 'Prior to the cafe, I worked in the flooring industry. 'When the opportunity came about to buy The Deli it was to try something completely different and I have really enjoyed it. 'I think it is really good being out of town with the businesses and doctors nearby. 'We get a lot of trade from other businesses and homes at lunchtime. 'The free parking helps too and we are really lucky to have many people walking in by for coffees, cookies and sandwiches.' In recent years, hundreds of homes have been built in the south of Elgin with thousands more still in the pipeline. Claire added: 'It is great for our businesses, with more people moving to Elgin. 'However I do feel it puts pressure on the NHS as well obviously. 'Some people say that nobody wants to live in Elgin, however there are so many new houses getting built. She added: 'I get people coming in who have just moved here and say they love it here. 'Meanwhile, people often come back after visiting too. 'I think we do take the place for granted because we have always been here and it is lovely to stay here.' Claire also holds private events inside The Deli which brings extra income for the business. She said: 'This year my outside catering has been really busy. 'We are closed on Sundays and private events are often held in it then like baby showers, birthday parties and other things. 'We have an alcohol licence as well which opens up different opportunities. 'It is just so great to have a different income stream too.' Meanwhile, she is thinking about getting a catering van to attend events like Highland Games and to provide more space than her car. She said: 'We could get a van because we are doing so much outside catering and there is an opportunity with it. 'It would great to go to the different markets and also promote The Deli shop too which would be good too. 'We will wait and see but it certainly been something popular that many businesses are doing.' She also praised her staff and manager Kirsten Hendry for making the Deli a friendly and inviting place for people. The Deli is open 9am to 4pm from Monday to Saturday every week. All meals are made to order from fresh and locally sourced produce.
Glasgow Times
an hour ago
- Glasgow Times
North missed £140bn of transport investment over last government, research finds
Independent analysis by think tank the Institute for Public Policy Research (IPPR) looked at Treasury figures between 2009/10 and 2022/23, during which time the Conservatives were in power. It reached the figure, which it said was enough to build seven Elizabeth Lines, by considering the amount of spending per person across the different English regions over that period. While England as a whole saw £592 spent per person each year, London received double that amount with £1,183 spent per person, the IPPR said. The entire North region saw £486 spent per person, with the North East and North West seeing £430 and £540 spent per person respectively. This amounted to £140 billion of missed investment for the North, more than the entire £83 billion estimate of capital spending on transport in the region since 1999/2000, according to the analysis. The region with the lowest amount of investment over the period was the East Midlands with just £355 spent per person. Among the most divisive transport investment projects for the previous government was the HS2 rail project, which was axed north of Birmingham in October 2023. Then-prime minister Rishi Sunak pledged to 'reinvest every single penny, £36 billion, in hundreds of new transport projects in the North and the Midlands', including improvements to road, rail and bus schemes. Earlier this week, Chancellor Rachel Reeves announced a £15.6 billion package for mayoral authorities to use on public transport projects across the North and Midlands ahead of the spending review. It is expected to include funding to extend the metros in Tyne and Wear, Greater Manchester and the West Midlands, along with a renewed tram network in South Yorkshire and a new mass transit system in West Yorkshire. Rachel Reeves has set out plans for new transport investment in the North and Midlands (Peter Byrne/PA) Marcus Johns, senior research fellow at IPPR North, said: 'Today's figures are concrete proof that promises made to the North over the last decade were hollow. It was a decade of deceit. 'We are 124 years on from the end of Queen Victoria's reign, yet the North is still running on infrastructure built during her rein – while our transport chasm widens. 'This isn't London bashing – Londoners absolutely deserve investment. But £1,182 per person for London and £486 for northerners? The numbers don't lie – this isn't right. 'This Government have begun to restore fairness with their big bet on transport cash for city leaders. 'They should continue on this journey to close this investment gap in the upcoming spending review and decades ahead.' Former Treasury minister Lord Jim O'Neill said: 'Good governance requires the guts to take a long-term approach, not just quick fixes. So the Chancellor is right in her focus on the UK's long-standing supply-side weaknesses – namely our woeful productivity and weak private and public investment. 'Backing major infrastructure is the right call, and this spending review is the right time for the Chancellor to place a big bet on northern growth and begin to close this investment chasm. 'But it's going to take more than commitments alone – she'll need to set out a transparent framework for delivery.'
Scotsman
an hour ago
- Scotsman
Prevention is better than cure when it comes to dealing with corruption
Put measures in place to lessen the risk of financial crimes occurring, says Sally Clark Sign up to our Scotsman Money newsletter, covering all you need to know to help manage your money. Sign up Thank you for signing up! Did you know with a Digital Subscription to The Scotsman, you can get unlimited access to the website including our premium content, as well as benefiting from fewer ads, loyalty rewards and much more. Learn More Sorry, there seem to be some issues. Please try again later. Submitting... Prison sentences were handed out in Glasgow last week to four men involved in a £6 million corruption and bribery investigation involving health boards across Scotland. This case, which underlines the importance of compliance and ensuring anti-financial crime prevention mechanisms are implemented and regularly monitored, exposed a series of fraudulently-awarded NHS contracts. Advertisement Hide Ad Advertisement Hide Ad Oricom Ltd, a company founded by Adam Sharoudi and Gavin Brown, secured deals to provide telecoms and video conferencing equipment to NHS trusts in Lothian, Grampian, Lanarkshire, Greater Glasgow & Clyde, and Ayrshire & Arran between 2010 and 2017. However, amid concerns about how the contracts were awarded, Oricom's offices were later raided by investigators including those from NHS Scotland Counter Fraud Services. CMS logo new The follow-up investigation found the company had been given 'commercially sensitive information' by former NHS employees Alan Hush and Gavin Cox. They received nearly £90,000 in cash and a range of other gifts, including luxury hotel stays, concert tickets, and holiday vouchers in return. The High Court heard how Hush helped Oricom land a number of business deals with the NHS including a £750,000 contract with NHS Lothian. Meanwhile, Cox helped the company secure contracts with NHS Lanarkshire. Overall, Hush was given £18,231 worth of cash bribes and gifts, while Cox received more than £70,000 of illicit rewards from the company. Advertisement Hide Ad Advertisement Hide Ad After being found guilty of a string of charges including bribery, corruption, fraud, and money laundering, Oricom founders Sharoudi and Brown were sentenced to eight and seven years' imprisonment respectively. Sally Clark is an Of Counsel and disputes resolution specialist, CMS Hush, who was telecommunications manager at NHS Lothian, then NHS Scotland video conferencing manager, was ordered to serve eight years while Cox, former head of IT and infrastructure at NHS Lanarkshire. received a six-year sentence. This case highlighted how basic procurement procedures were circumvented to ensure a company paying bribes was awarded contracts. This may be a familiar bribery tale, but is a strong reminder that putting in place avoidance measures to prevent corruption and other forms of financial crime is preferable to taking steps to deal with such violations. In this case, the police and NHS counter-fraud investigation began in 2015, and a decade-long journey culminated with a trial which heard 65 days of evidence. Advertisement Hide Ad Advertisement Hide Ad No business or organisation wants to face the potential legal costs, reputational damage, and prospect of immense disruption that accompanies a financial crime investigation. Those being investigated are typically required to provide materials under warrant, which includes the interrogation of computer systems by law enforcement agencies. Staff may also be required to provide witness statements and respond to ongoing queries from investigating authorities, all of which requires significant time and resource. Being involved in a law enforcement investigation can generate exceptional anxiety amongst staff members who are required to provide witness statements and give evidence in court. This also brings further business disruption with people being diverted from their core focus within a business or organisation. As this NHS case highlights, putting regularly monitored measures in place to prevent the prospect of financial crimes occurring is a much more preferable alternative to dealing with the fallout of failing to do so. The introduction of section 196 of the Economic Crime and Corporate Transparency Act 2023 should further underline this message – under this new provision, a business or organisation can be prosecuted for economic crimes that are perpetrated by senior managers who are on their staff.
