Does anyone really want AI civil servants?

Spectator

8 hours ago

Of course they've called it 'Humphrey'. The cutesy name that has been given to the AI tool the government is rolling out across the civil service with unseemly haste is a nod – as those of an age will recognise – to the immortal sitcom Yes, Minister. But it may also prove to be more appropriate than they think. The premise of that show, you'll recall, is that Sir Humphrey is the person really in charge – and that he will at every turn imperceptibly thwart and subvert the instructions given to him by the elected minister.
Why is Sir Keir Starmer so absolutely hellbent on turning us into, in his wince-makingly gauche phrase, 'an AI superpower'?
At least in the show, we're encouraged to believe that Sir Humphrey undermines Jim Hacker because he's cleverer and has the best interests of the nation at heart. But an AI Humphrey has no such redeeming qualities: if it undermines the elected minister, it'll be for no reason other than an algorithmic sport.
Item one: generative AI hallucinates. It makes stuff up. Nobody knows exactly why, and nobody knows how to stop it doing so. Some experts in the field say that there's a good chance that the problem will get worse rather than better over time: after all, as an ever-greater proportion of the zillions of words of text on the internet comes to be AI-generated, and AI models are therefore training on the outputs of AI models, those hallucinations are going to be baked in. Garbage in, garbage out, as programmers like to say.
So though ChatGPT and its cousins are a fantastic boon to people who don't want to do their work – be they lazy undergraduates, lawyers who can't be bothered to comb through case law and write their own briefs, or government ministers who imagine the savings to be made if bureaucratic emails were to start writing themselves – they come with significant risks.
It's not just those notorious Google searches that encourage you to put glue on pizza. Already, we're seeing cases coming to court where lawyers have used AI to draft their arguments, and it has emerged that the LLM has invented its legal citations out of whole cloth. Academic work is being turned out with footnotes leading to works that don't exist, and imaginary bibliographies. More than one US newspaper published a syndicated 'summer reading' special in which several of the books it recommended didn't exist.
Is this going to be a problem when it comes to the machinery of Whitehall? I would say so, wouldn't you? The Post Office Horizon scandal – which had at its root a lot of credulous officials believing everything that a malfunctioning computer told them – ruined lives and cost the taxpayer a small fortune in compensation and in the inquiries that had to sort out the whole mess. Embedding a large language model at the heart of government is a recipe for any number of repeat performances. It seems perfectly reasonable to suppose that the legal risk will outweigh any vaunted efficiency savings – to say nothing of the potential for human suffering if the LLM goes wonky in the tax and benefits systems.
The promise to 'have meaningful human control at the right stages' sounds like an excellent principle – but it's not clear how it can be more than an aspiration. You won't know when you've got it wrong until it bites you in the bum. And people, remember, are lazy. What's the betting that they won't always bother to check the computer's homework when the homework sounds plausible enough, and it's getting towards time for a pint in the Red Lion?
Item two: there is a moral case as well as a practical one against Humphrey. Not only does generative AI have serious environmental costs, but it's a plagiarism machine. ChatGPT, which is one of the models on which Humphrey has been built, is known to have scraped text to train its models from piracy websites. This is still a live legal issue. And as Ed Newton-Rex of the campaign group Fairly Trained has put it: 'The government can't effectively regulate these companies if it is simultaneously baking them into its inner workings as rapidly as possible.'
Why is Sir Keir Starmer so absolutely hellbent on turning us into, in his wince-makingly gauche phrase, 'an AI superpower'; so keen to jump the gun that he hadn't even allowed the public consultation on AI and copyright to conclude before he pushed the government's recommendations – which were, basically, to let copyright holders be damned. He seems to have been seduced by the blandishments of the salespeople for this technology, whose main sales tool is FOMO. AI is a solution in search of a problem. Big tech has invested so much in it that they're trying to brute-force it into every area of life, and they are succeeding.
Of course, one can see how – for instance – using AI to minute meetings or draft memos can save costly man-hours. But the way to integrate it into the machinery of Whitehall is, or should be, with extreme caution and on a case-by-case basis, not with the panicky haste of someone who's been persuaded by a lobbyist that if you don't go all-in on this exciting new technology as fast as possible you're going to be left behind. It seems something of a tell, for instance, that Principle 8 in the government's own AI Playbook is: 'You work with commercial colleagues from the start.'
As Sir Humphrey would say: 'No, Prime Minister.'