Groundbreaking BBC research shows issues with over half the answers from Artificial Intelligence (AI) assistants
New BBC research published today provides a warning around the use of AI assistants to answer questions about news, with factual errors and the misrepresentation of source material affecting AI assistants.
The findings are concerning, and show:
51% of all AI answers to questions about the news were judged to have significant issues of some form
19% of AI answers which cited BBC content introduced factual errors – incorrect factual statements, numbers and dates
13% of the quotes sourced from BBC articles were either altered or didn't actually exist in that article.
The study, conducted over a month, saw the BBC test four prominent, publicly available AI assistants – OpenAI's ChatGPT; Microsoft's Copilot; Google's Gemini; and Perplexity. These AI assistants were given access to the BBC's website and asked questions about the news, prompting them to use BBC News articles as sources where possible. AI answers were reviewed by BBC journalists, all experts in the question topics, on criteria including accuracy, impartiality and how they represented BBC content.
Pete Archer, Programme Director for Generative AI at the BBC says: 'We're excited about the future of AI and the value it can bring audiences. We have already used it to add subtitles to programmes on BBC Sounds and translate content into different languages on BBC News. AI can bring real value if used responsibly.'
'But AI is also bringing significant challenges for audiences. People may think they can trust what they're reading from these AI assistants, but this research shows they can produce responses to questions about key news events that are distorted, factually incorrect or misleading. The use of AI assistants will grow so it's critical the information they provide audiences is accurate and trustworthy.'
'Publishers, like the BBC, should have control over whether and how their content is used and AI companies should show how assistants process news along with the scale and scope of errors and inaccuracies they produce. This will require strong partnerships between AI and media companies and new ways of working that put the audience first and maximise value for all. The BBC is open and willing to work closely with partners to do this.'
Some examples of the significant problems identified in responses from these AI assistants include:
ChatGPT and Copilot claimed that former Prime Minister Rishi Sunak and former First Minister Nicola Sturgeon were still in office after they had left.
Gemini incorrectly stated that 'The NHS advises people not to start vaping, and recommends that smokers who want to quit should use other methods.' In fact, the NHS does recommend vaping as a method to quit smoking.
A Perplexity response on the escalation of conflict in the Middle East, giving BBC as its source, said Iran initially showed 'restraint' and described Israel's actions as 'aggressive' – yet those adjectives hadn't been used in the BBC's impartial reporting.
The full research can be found on the BBC website.
Read more: Article from CEO, BBC News and Current Affairs, Deborah Turness
IW
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Daily Mirror
42 minutes ago
- Daily Mirror
'Rachel Reeves promises so much - but we're worried how it will be paid for'
Rachel Reeves' Spending Review promised big boosts for the NHS, defence, and housing - but many of you are worried about how it will all be funded If you're wondering where all the money's going to come from to pay for the Chancellor's Spending Review then you're not alone. Rachel Reeves finally delivered her long-anticipated plan to renew Britain – and while there were some big promises, it left many of you asking how it's going to be funded. Standing at the despatch box in the House of Commons, the Chancellor pitched her plan as a blueprint for 'Britain's renewal' and took a swipe at years of 'destructive' Tory austerity. She announced that departmental budgets will rise by 2.3% each year in real terms, with hefty sums heading to the NHS, defence, and housing. In parliament, Ms Reeves batted away claims she would raise taxes and said her spending plans were paid for by decisions made at the Autumn Budget. She repeated her pledge not to raise taxes on working people. But a think tank says tax rises are likely in the autumn. The Resolution Foundation reported the big rise in public spending has been largely paid for by £39.7 billion in tax increases (set for 2028-29) announced in last Autumn's Budget, along with £3.6 billion in benefit cuts (also in 2028-29) revealed in the Spring Statement – which works out to about £1,550 for every family in Britain. "But the combination of a weaker economic outlook, an unfunded spending commitment on winter fuel payments, and just £9.9 billion of headroom against the chancellor's fiscal rules, mean further tax rises are likely to be needed this autumn," it says. We asked Did Rachel Reeves get her Spending Review right? Just 908 of you felt she did, while a whopping 1,982 said she didn't. Many of you responded to our original story, here's just a slice of what some of you had to say: Martinsopinion posted: "I don't know about you, but I don't want to pay more council tax for less services. I wonder what other ways they will find to tax us? Maybe spending money on poorly connected urban sprawl to maybe help a few, when the country needs a new city full of six-storey buildings. The policies are so misguided." Hfchbffch said: "What happened to the 22 billion black hole? Looks like it's gone bigger. Where is all the cash coming from? I don't see any country given money to the UK for all the problems it has got. It looks like the UK working people will foot another bill." 72Momma added: "Where is all the money coming from? Have they actually got rid of the 'black hole' or are they just remortgaging the country? To be able to spend over £102bn ... who is she borrowing from?" Joeyd: "Margaret Thatcher's Reagonomics caused the global economic crash. She sold off Britain to foreign countries for a pittance, wrecked mining, shipbuilding and steel towns across Britain, sold off council housing and put all our money into banking. When the banks collapsed, the country collapsed with them. The Conservative Government introduced 14 years of failed austerity making the rich even richer, and the poorest even poorer. Someone has to reverse decades of neglect, we'll have to wait and see if they carry through with the infrastructure plans she's promising." Seccmjfl01: "It might not be perfect but at least she's trying and making an effort. Things take time and she needed some stability. Good luck to her." Wolveslegends: "The Chancellor promises so much with no money to pay for it. Come the Autumn she will gain the funds from guess where from? All of us will donate to her cause if we like it or not including the disabled and those too old to work. Welcome to New Labour." Reeves is pledging an extra £29 billion a year for the NHS – a real-terms boost of around 3% annually. And on housing, she laid out a massive £39 billion package to build hundreds of thousands of affordable homes over the next decade. It's being called the 'biggest cash injection' in 50 years and is part of the drive to get 1.5 million new homes built by 2029. Following details of the winter fuel U-turn earlier this week, Reeves confirmed plans to restore the payments to millions of pensioners. OAPs with an income of £35,000 or below in England and Wales will receive the benefit this winter. It is a major uplift from the current £11,500 cut-off point announced last summer. In an "age of insecurity" the Chancellor said defence spending will rise to 2.6% of GDP by 2027. She said the figure includes spending on intelligence agencies. 'We will make Britain a defence industrial superpower," Ms Reeves vowed. The Mirror understands schools will also get a major boost to per pupil funding, with £4.5billion extra for the core schools budget. This includes a major expansion of free school meals to 500,000 more pupils. Some £15.6billion will also be handed to mayors for major transport projects across the country. This will go towards plans to improve trams, trains and buses in the North and the Midlands. Other announcements include ending the "costly" use of hotels to house asylum seekers, £3 cap on single bus tickets, and over £14billion worth of investment will go towards building the Sizewell C nuclear power plant on the Suffolk coast. Keir Starmer told the Cabinet that the Spending Review: "Marks the end of the first phase of this Government, as we move to a new phase that delivers on the promise of change for working people all around the country and invests in Britain's renewal." However, Tory Shadow Chancellor Mel Stride slammed the announcement, as he fumed: "This spending review is not worth the paper it is written on.' He goes on to claim Ms Reeves has "completely lost control" and warns of tax rises to come in the autumn."
Reuters
an hour ago
- Reuters
Temasek joins Microsoft, BlackRock and MGX to develop AI infrastructure
SINGAPORE, June 12 (Reuters) - Temasek has joined a consortium backed by Microsoft (MSFT.O), opens new tab, BlackRock (BLK.N), opens new tab and tech investment company MGX to invest and expand artificial intelligence infrastructure, according to BlackRock's investor day presentation slides on Thursday. The Singapore state investment company has joined AI Infrastructure Partnership, a group that also includes BlackRock's Global Infrastructure Partners, the slides showed. AIP, formed in September with a goal to initially invest more than $30 billion in AI-related projects, is one of the world's largest efforts to invest in data centres and energy facilities needed to power AI applications such as ChatGPT. It aims to mobilise up to $100 billion including debt financing for such investments, which will focus on the United States. Temasek's participation comes after the Kuwait Investment Authority joined AIP earlier in June. The sovereign wealth fund of Kuwait was the first non-founder financial anchor investor to join the consortium, which also counts partners including Nvidia (NVDA.O), opens new tab and billionaire Elon Musk's xAI. "Temasek's investment in the AI Infrastructure Partnership reflects our focus on the big shifts and trends of the future," Ravi Lambah, Temasek's head of strategic initiatives, said in an email to Reuters. "AI is potentially the most transformative and impactful technology for all sectors and businesses," he added. Temasek did not disclose financial details of the investment. The global investment company had a net portfolio value of S$389 billion ($304 billion) as of March 31, 2024, according to its website. ($1 = 1.2804 Singapore dollars)
Daily Mirror
an hour ago
- Daily Mirror
Poorest Brits to benefit most from major spending changes - but tax rises loom
The poorest Brits will benefit most from Rachel Reeves's public sector spending spree but tax rises look likely in the autumn, think tank the Resolution Foundation has said The poorest Brits will benefit most from Rachel Reeves's public sector spending spree but tax rises look likely in the autumn, a leading think tank has said. The Resolution Foundation said extra funding for hospitals, schools and police would be worth around £1,700 in provision for the worst-off fifth of families by 2028/29. It estimated that a middle-income household will gain £1,400 on average. But it warned that tight public finances and a weaker economic outlook could push the Chancellor into hiking taxes. In its Spending Review analysis, the thinktank said Britain is turning into a 'National Health State' - and health could account for half (49%) of all day-to-day public service spending controlled by Westminster by the end of the decade. This is up from a third (34%) in 2009/10. READ MORE: Rachel Reeves pumps cash into NHS with 4million more tests and procedures to cut waits While funding per person for health has increased by 36% over the decade, it has fallen by 16% for justice, 31% for the DWP, and 50% for Housing, Communities and Local Government. On Wednesday the Chancellor turned on the spending taps with a £300billion package to renew Britain. It included £39billion for affordable housing and £29billion for the NHS. Ms Reeves refused to rule out future tax hikes but said all the cash doled out in her Spending Review had been accounted for already in the Budget and Spring Statement. 'It would be very risky for a Chancellor to try and write future budgets in a world as uncertain as ours, but I won't have to repeat a Budget like the one I did last year,' she told LBC. It comes as the Institute for Fiscal Studies said a "big chunk" of funding for local authorities in the Spending Review was coming from an assumption that council tax in England will rise by the maximum amount of just under 5% each year. Ms Reeves declined to rule out council tax rising by 5% every year. But she said local authorities did not have to increase tax by this figure. She also did not accept claims a 5% rise would be needed to boost police funding, saying the Government had increased the spending power of the police by 2.3% every year. READ MORE: Elsewhere, the Chancellor was also forced to reject the suggestion she was a "Klarna Chancellor" who had announced a "buy now, pay later" spending review. "I don't accept that at all," she said. "The idea that yesterday I racked up a bill that I'm going to need to pay for in the future, that's just not right. Her comments come as the Office for National Statistics reported the economy shrank by 0.3% in April. It was the biggest monthly contraction since October 2023 and worse than the 0.1% fall most economists had expected. Ruth Curtice, chief executive of the Resolution Foundation, said without economic growth, another round of tax rises was likely to come in the autumn as the Chancellor seeks to balance the books. She said: "The extra money in this spending review has already been accounted for in the last forecast. But a weaker economic outlook and the unfunded changes to winter fuel payments mean the Chancellor will likely need to look again at tax rises in the autumn." Meanwhile, Kemi Badenoch yesterday criticised the Government for giving 'more and more money' to public services - but declined to say what she would do differently. "What we should be talking about is reform of public services. They're just giving more and more money,' she said. "This is a war on the private sector, where private businesses are having to cut their coat according to their cloth. Of course, we want to fund public services, but we need to make sure that we're doing things better." But asked repeatedly to set out what the Government should stop doing, or whether she would reverse last year's rise in national insurance, Mrs Badenoch declined to answer. She said: "There's no election today. What I'm not doing is setting out a manifesto for four years' time." The Conservative leader also described rising health spending as a "conundrum", with a similar approach having been taken "again and again". In reference to a pro-Brexit campaign stunt, Mrs Badenoch said: "I mean, who remembers the side of a red bus that said 'we're going to give the NHS £350 million more a week'? "Many people don't know that we did that. We did do that, and yet, still we're not seeing the returns. We've put more and more money in, and we're getting less and less out."
