How innocent people could be jailed because of sinister AI bot ChatGPT… and leave dangerous criminals walking free

The Sun

10-05-2025

JURORS could secretly be using ChatGPT to decide crunch verdicts, top US lawyers fear.
Dean Strang and Jerry Buting, who starred in Netflix documentary Making a Murderer, warned it could see innocent people convicted - or let criminals off the hook.
4
4
The lawyers, who defended Steven Avery on the hit show, insisted it is vital jurors are stopped from accessing the AI tool during trials.
Strang said jurors could be tempted to use ChatGPT as a 'sounding board' if they are uncertain about a defendant.
But he stressed it could have "disastrous" consequences as jurors could get skewed answers that force a mistrial or bring them to the wrong decision.
Strang, who has worked in criminal defence for more than 30 years, told The Sun: 'Jurors should be banned from using ChatGPT.
'I can't prove this and I'm not a computer engineer, but my sense is that AI, in part, is assessing what it thinks we want to hear and tailoring answers to its perception of what our human wishes and purposes are.
'That would be disastrous in a jury room, to the extent that AI decides the juror is inclined to a conviction.
'You're going to get a skewed answer. To the extent AI perceives the jurors are inclined to acquittal or if AI develops its own will, which I read is at least possible, kind of developing a consciousness.
'Either way, you're getting a skew and neither skew is good, and you're substituting out what you hope is human judgment, a human assessment of what's happening in the courtroom, credibility determinations.'
Asked if he fears jurors are already using the tool, or soon will, Strang answered: 'Yes.
'Some never would, and I fear that some would be tempted to or use it as a sounding board, use it as an advisor.
China's new cheap AI DeepSeek sparks ALARM as it outperforms West's models like ChatGPT amid race to superintelligence
'Even if the juror says I'm going to make the decision, but I'm getting advice from AI, that's going to be advice that's probably hard to ignore.'
Strang and Buting defended Avery in the 2015 documentary, which claimed Avery had been framed for the murder of Teresa Halbach.
Avery has been behind bars for the crime since 2007.
Before that, he spent 18 years in jail for a rape and attempted murder he was later found innocent of.
Strang and Buting continue to maintain his innocence, as does Avery himself.
Now, Buting fears even more alleged miscarriages of justice through the use of ChatGPT.
Buting, author of Illusion of Justice, told The Sun: 'I've seen people online take situations like the Steven Avery case or the Karen Reed case, a second trial going on right now because the jury was hung, try and use GPT by putting in the facts as they know them.
'Then they ask ChatGPT whether the person is guilty or innocent and they come up with an answer.
'It's bulls*** basically, because it depends on what you put in as the facts.
What are the arguments against AI?
Artificial intelligence is a highly contested issue, and it seems everyone has a stance on it. Here are some common arguments against it:
Loss of jobs - Some industry experts argue that AI will create new niches in the job market, and as some roles are eliminated, others will appear. However, many artists and writers insist the argument is ethical, as generative AI tools are being trained on their work and wouldn't function otherwise.
Ethics - When AI is trained on a dataset, much of the content is taken from the internet. This is almost always, if not exclusively, done without notifying the people whose work is being taken.
Privacy - Content from personal social media accounts may be fed to language models to train them. Concerns have cropped up as Meta unveils its AI assistants across platforms like Facebook and Instagram. There have been legal challenges to this: in 2016, legislation was created to protect personal data in the EU, and similar laws are in the works in the United States.
Misinformation - As AI tools pull information from the internet, they may take things out of context or suffer hallucinations that produce nonsensical answers. Tools like Copilot on Bing and Google's generative AI in search are always at risk of getting things wrong. Some critics argue this could have lethal effects - such as AI prescribing the wrong health information.
'This would be the problem with jurors doing it because jurors listen to the evidence.
"If they go back in there and they can't really resolve something themselves, well then that's probably reasonable doubt in most cases.
'But if they say, well, we can't resolve this, you think this and I think that and there's a dispute, let's put it into ChatGPT and see what it says, and then it comes up with an answer, then somebody may be swayed by that.
'AI, at least in its current iteration, has built in biases because of the algorithms.'
Buting previously told The Sun he fears AI could destroy the entire justice system by sending innocent people to jail with fake CCTV.
ChatGPT exploded onto the scene in 2022 and has since become an essential tool for individuals and businesses worldwide.
Buting said: 'We don't know at this point exactly how it's filtering, how it's learning.
'The idea is that it learns from more fact situations presented over and over, but what fact situations, what trials, for instance, is it looking at?
4
4
'Are there already ones where there has been a built-in bias? Because there is a lot of bias in America's legal system. Bias against minorities in particular.
'So are they kind of underrepresented in the algorithm, machine learning that is happening with AI? A lot of people wonder about that.
'I just know I've seen people use ChatGPT. I can use it and put in facts, and leave out facts that I want to, and it'll come up with an answer probably that I want.
'So I think there's a real problem.'
When The Sun asked ChatGPT if Avery was guilty, the response was: "Legally: Steven Avery is guilty, he was convicted and remains in prison.
What does the law say?
UNDER UK law, judges are allowed to use ChatGPT to assist them when making rulings.
AI assistance was given the green light by a landmark ruling in 2023.
Guidelines from the Judicial Office sent to thousands of judges across England and Wales said that the tech can be useful for summarising large volumes of text or carrying out administrative tasks.
However, it cautions that chatbots should relied upon for researching legal frameworks, because there is a risk it will fabricate cases or legal documents.
The guidelines also warn that generative AI technology could be used to create false evidence - such as deepfake pictures or videos.
The US is currently grappling with the integration of AI in its legal proceedings.
Spearheading progress is the Illinois Supreme Court, which issued guidance on the use of AI by judges and lawyers in December 2024.
The document encouraged the responsible and supervised use of AI, and suggested that the use of chatbots in drafting pleadings need not be declared.
Its tone is generally pro-AI, and emphasises that existing legal and ethical guidance can be applied.
In early May 2025, a federal judicial panel advanced a proposal to seek the public's feedback on a draft rule designed to ensure AI-produced evidence meets the same standards as human evidence.
'Public opinion and expert debate: Divided. Many believe he may have been wrongfully convicted again, especially given the suspicious handling of evidence.
'No definitive proof of innocence or of a frame-up has yet convinced the courts.'
It comes as fears continue to be raised about its threat to jobs, and the dangers of the software outsmarting humans.
Many experts have also warned of the security dangers of advanced AI – and how cyber-crooks could abuse them to scam innocent victims.
The Sun has approached ChatGPT for a response.
Who is Steven Avery?
STEVEN Avery is serving a life sentence at Wisconsin's Waupun Correctional Institution.
He and his nephew Brendan Dassey were convicted of the 2005 murder of Teresa Halbach.
He has been fighting for his freedom ever since he was found guilty of murder in 2007.
Avery argued that his conviction was based on planted evidence and false testimony.
In 1985, Avery was falsely convicted of sexually assaulting a young female jogger.
It took 18 years for his conviction to be overturned and he was given a $36million (£28.2million) payout in compensation.
But days later, he was re-arrested for the murder of Teresa Halbach.
The 62-year-old is continuing serving life in prison without the possibility of parole.
In the 2015 Netflix original series Making a Murderer, Avery documented his struggle for "justice."
In the last episode of the series, viewers were told that Avery had exhausted his appeals and was no longer entitled to state-appointed legal representation.