NHS Lothian partner in project evaluating adoption of artificial intelligence products
Testing a single piece of AI software currently takes more than nine months
NHS Lothian is one of two health boards in Scotland partnering with an evaluation company to speed up the safe adoption of artificial intelligence (AI) in the healthcare system.
The £1 million Innovate UK-funded project brings together NHS Lothian and NHS Greater Glasgow and Clyde hospitals, Edinburgh and Glasgow universities, and technology companies.
Testing a single piece of AI software currently takes more than nine months, and with more than 200 AI products available for a single hospital department, checking all of them individually using traditional clinical trial or service evaluation models isn't realistic. The result is that very few trials turn into real-world solutions, meaning that patients, hospitals and AI developers all lose out.
The collaboration will use Aival's independent evaluation platform to assess AI systems in a secure environment, with all patient data anonymised and remaining within an NHS secure environment.
The project will assess the use of the Aival platform to compare six commercial AI products for the triage of stroke and lung cancer patients from AI developers InferVision, Annalise-AI and Qure.AI – who are all collaborating with the programme.
AI products are often calibrated to the locations where they are being used to allow for differences in the local population, meaning they may not generalise to different groups without extra training.
The performance of AI products changes over time – even when the product itself isn't altered. This is known as 'drift', and can happen as a patient population changes, as the commonness of a disease shifts, or a scanner's software is updated.
Dr Rishi Ramaesh, Consultant Radiologist and Innovation Fellow at NHS Lothian, said: 'Artificial intelligence has tremendous potential to improve patient care, but healthcare leaders need confidence that these systems are safe and effective. This project will help healthcare leaders to evaluate AI, and make sure that new technologies deliver real benefits for patients.'
Luciana D'Adderio, Edinburgh University academic and AI evaluation and assurance expert, said: 'AI technology is achieving widespread deployment across healthcare settings, yet its assurance has not received the rigorous attention it demands.
'The current frameworks for evaluation and monitoring fall short in adequately assuring AI systems, primarily because AI represents a fundamental departure from the traditional technologies these frameworks were designed to assess.
'There is an urgent need for innovative tools and technologies for AI assurance, which themselves must undergo thorough evaluation and validation. This critical challenge forms the cornerstone of our groundbreaking project.'
Kanwal Bhatia, CEO and founder of Aival, said: 'It's vital that we monitor and check AI that's being used in decisions on patients' health, to ensure the best outcomes for patients.
'Any healthcare workers using AI need to be sure that the product is doing what it says it does – not just now, but five years in the future. While many NHS teams understand this, they currently lack the tools to properly evaluate the AI products.
'Putting in place effective validation systems will encourage trust and adoption of AI, and will deliver cost savings and growth in the NHS and in private healthcare.
'We work hand in hand with NHS leaders, clinical and technical teams to provide the expertise and software to ensure that their AI systems are doing what they're supposed to do.'
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
North Wales Chronicle
2 days ago
- North Wales Chronicle
China's proposed ‘super embassy poses super risk' to security, Tories claim
More than a thousand demonstrators took to the streets of central London earlier this year to protest against the proposed embassy. Ministers are expected to make a final decision on the redevelopment plans at the historic former site of the Royal Mint, after it was rejected by the local council, Tower Hamlets. Speaking in the Commons, shadow Cabinet Office minister Alex Burghart accused the Government of being 'too weak' to block the plans. He said: 'The Government's own cyber experts, Innovate UK, have warned the Government that the proposed Chinese embassy at the Royal Mint threatens to compromise the telephone and internet exchange that serves the financial City of London. 'The experts are now telling the Government what everyone else has known all along: the super embassy poses a super risk. 'Yet the Deputy Prime Minister's office has said that any representations on the planning application have to be made available to the applicants. 'So perhaps the real Deputy Prime Minister can clear this up – is the Government seriously saying that if MI5 or GCHQ have concerns about security on this site, those concerns will have to be passed to the Chinese Communist Party or has the Deputy Prime Minister (Angela Rayner) got it wrong?' Chancellor of the Duchy of Lancaster Pat McFadden replied: 'When it comes to both engagement with China and with an issue like this, we will of course engage properly and always bear in mind our own national security considerations.' He also accused Tory governments of opting to 'withdraw' from engaging with China for a number of years after they had previously sought a 'golden era' with Beijing. A spokesperson for Innovate UK said it 'has not raised any concerns', adding: 'Innovate UK does not have responsibility for cyber security.' Comments referred to by Mr Burghart are understood to have been made in a personal capacity. Mr Burghart went on to say: 'At the heart of this are two simple facts. First, the Government already knows that this site is a security risk; it's a security risk to the City of London and through it our economy and the economies of all nations that trade in London. 'Second, the Government has the power to block it. Ireland and Australia have both already blocked similar embassy developments. Why is this Government too weak to act?' In his reply, Mr McFadden said: 'A decision on this application will be taken in full consideration of our national security considerations. That is always part of this and it is part of our engagement with China and with other countries.' Chancellor Rachel Reeves flew to China in January to meet with the country's leaders and set out a path towards further investment into Britain. Elsewhere in Cabinet Office questions, Mr McFadden said the Civil Service should be more representative of the UK and speak with 'all the accents of the country'. This came in response to Labour MP for Dudley, Sonia Kumar, who said: 'I welcome the relocation of the Civil Service across the country and with a recent study showing that Dudley has high levels of economic inactivity, what reassurances can (he) give me that young people in Dudley want every opportunity to build their career in the Civil Service, whether that's training, apprenticeships, or any mentorships?' Mr McFadden replied: '(Ms Kumar) will not be surprised to hear my strong enthusiasm for greater employment opportunities for young people in the Black Country. 'When we made the announcement last week about the relocation, we also announced a new apprentice scheme because we don't just have to change location, we have to change recruitment patterns too if we're really going (to) get that Civil Service that speaks with all the accents of the country.' Later in the session, Conservative MP Lincoln Jopp (Spelthorne) asked for reassurance that 'no Cabinet Office ministers will be attending Glastonbury this year', because 'controversial Ulster rap band' Kneecap remain part of the festival's line-up. Mr McFadden replied: 'I will not be going to Glastonbury, but I'm very much looking forward to going to see Bruce Springsteen at Anfield Stadium on Saturday night.'
South Wales Guardian
2 days ago
- South Wales Guardian
China's proposed ‘super embassy poses super risk' to security, Tories claim
More than a thousand demonstrators took to the streets of central London earlier this year to protest against the proposed embassy. Ministers are expected to make a final decision on the redevelopment plans at the historic former site of the Royal Mint, after it was rejected by the local council, Tower Hamlets. Speaking in the Commons, shadow Cabinet Office minister Alex Burghart accused the Government of being 'too weak' to block the plans. He said: 'The Government's own cyber experts, Innovate UK, have warned the Government that the proposed Chinese embassy at the Royal Mint threatens to compromise the telephone and internet exchange that serves the financial City of London. 'The experts are now telling the Government what everyone else has known all along: the super embassy poses a super risk. 'Yet the Deputy Prime Minister's office has said that any representations on the planning application have to be made available to the applicants. 'So perhaps the real Deputy Prime Minister can clear this up – is the Government seriously saying that if MI5 or GCHQ have concerns about security on this site, those concerns will have to be passed to the Chinese Communist Party or has the Deputy Prime Minister (Angela Rayner) got it wrong?' Chancellor of the Duchy of Lancaster Pat McFadden replied: 'When it comes to both engagement with China and with an issue like this, we will of course engage properly and always bear in mind our own national security considerations.' He also accused Tory governments of opting to 'withdraw' from engaging with China for a number of years after they had previously sought a 'golden era' with Beijing. A spokesperson for Innovate UK said it 'has not raised any concerns', adding: 'Innovate UK does not have responsibility for cyber security.' Comments referred to by Mr Burghart are understood to have been made in a personal capacity. Mr Burghart went on to say: 'At the heart of this are two simple facts. First, the Government already knows that this site is a security risk; it's a security risk to the City of London and through it our economy and the economies of all nations that trade in London. 'Second, the Government has the power to block it. Ireland and Australia have both already blocked similar embassy developments. Why is this Government too weak to act?' In his reply, Mr McFadden said: 'A decision on this application will be taken in full consideration of our national security considerations. That is always part of this and it is part of our engagement with China and with other countries.' Chancellor Rachel Reeves flew to China in January to meet with the country's leaders and set out a path towards further investment into Britain. Elsewhere in Cabinet Office questions, Mr McFadden said the Civil Service should be more representative of the UK and speak with 'all the accents of the country'. This came in response to Labour MP for Dudley, Sonia Kumar, who said: 'I welcome the relocation of the Civil Service across the country and with a recent study showing that Dudley has high levels of economic inactivity, what reassurances can (he) give me that young people in Dudley want every opportunity to build their career in the Civil Service, whether that's training, apprenticeships, or any mentorships?' Mr McFadden replied: '(Ms Kumar) will not be surprised to hear my strong enthusiasm for greater employment opportunities for young people in the Black Country. 'When we made the announcement last week about the relocation, we also announced a new apprentice scheme because we don't just have to change location, we have to change recruitment patterns too if we're really going (to) get that Civil Service that speaks with all the accents of the country.' Later in the session, Conservative MP Lincoln Jopp (Spelthorne) asked for reassurance that 'no Cabinet Office ministers will be attending Glastonbury this year', because 'controversial Ulster rap band' Kneecap remain part of the festival's line-up. Mr McFadden replied: 'I will not be going to Glastonbury, but I'm very much looking forward to going to see Bruce Springsteen at Anfield Stadium on Saturday night.'
South Wales Argus
2 days ago
- South Wales Argus
China's proposed ‘super embassy poses super risk' to security, Tories claim
More than a thousand demonstrators took to the streets of central London earlier this year to protest against the proposed embassy. Ministers are expected to make a final decision on the redevelopment plans at the historic former site of the Royal Mint, after it was rejected by the local council, Tower Hamlets. Speaking in the Commons, shadow Cabinet Office minister Alex Burghart accused the Government of being 'too weak' to block the plans. He said: 'The Government's own cyber experts, Innovate UK, have warned the Government that the proposed Chinese embassy at the Royal Mint threatens to compromise the telephone and internet exchange that serves the financial City of London. 'The experts are now telling the Government what everyone else has known all along: the super embassy poses a super risk. 'Yet the Deputy Prime Minister's office has said that any representations on the planning application have to be made available to the applicants. 'So perhaps the real Deputy Prime Minister can clear this up – is the Government seriously saying that if MI5 or GCHQ have concerns about security on this site, those concerns will have to be passed to the Chinese Communist Party or has the Deputy Prime Minister (Angela Rayner) got it wrong?' Chancellor of the Duchy of Lancaster Pat McFadden replied: 'When it comes to both engagement with China and with an issue like this, we will of course engage properly and always bear in mind our own national security considerations.' Shadow Cabinet Office minister Alex Burghart (Ben Whitley/PA) He also accused Tory governments of opting to 'withdraw' from engaging with China for a number of years after they had previously sought a 'golden era' with Beijing. A spokesperson for Innovate UK said it 'has not raised any concerns', adding: 'Innovate UK does not have responsibility for cyber security.' Comments referred to by Mr Burghart are understood to have been made in a personal capacity. Mr Burghart went on to say: 'At the heart of this are two simple facts. First, the Government already knows that this site is a security risk; it's a security risk to the City of London and through it our economy and the economies of all nations that trade in London. 'Second, the Government has the power to block it. Ireland and Australia have both already blocked similar embassy developments. Why is this Government too weak to act?' In his reply, Mr McFadden said: 'A decision on this application will be taken in full consideration of our national security considerations. That is always part of this and it is part of our engagement with China and with other countries.' Pat McFadden, Chancellor of the Duchy of Lancaster (Stefan Rousseau/PA) Chancellor Rachel Reeves flew to China in January to meet with the country's leaders and set out a path towards further investment into Britain. Elsewhere in Cabinet Office questions, Mr McFadden said the Civil Service should be more representative of the UK and speak with 'all the accents of the country'. This came in response to Labour MP for Dudley, Sonia Kumar, who said: 'I welcome the relocation of the Civil Service across the country and with a recent study showing that Dudley has high levels of economic inactivity, what reassurances can (he) give me that young people in Dudley want every opportunity to build their career in the Civil Service, whether that's training, apprenticeships, or any mentorships?' Mr McFadden replied: '(Ms Kumar) will not be surprised to hear my strong enthusiasm for greater employment opportunities for young people in the Black Country. 'When we made the announcement last week about the relocation, we also announced a new apprentice scheme because we don't just have to change location, we have to change recruitment patterns too if we're really going (to) get that Civil Service that speaks with all the accents of the country.' Later in the session, Conservative MP Lincoln Jopp (Spelthorne) asked for reassurance that 'no Cabinet Office ministers will be attending Glastonbury this year', because 'controversial Ulster rap band' Kneecap remain part of the festival's line-up. Mr McFadden replied: 'I will not be going to Glastonbury, but I'm very much looking forward to going to see Bruce Springsteen at Anfield Stadium on Saturday night.'
