ChatGPT is driving people mad

Telegraph

a day ago

'My loved ones would tell me to stop now,' the man typed into ChatGPT.
The conversation had been going on for hours, and it was now late at night. 'At this point, I need to disengage with you and go to bed,' he wrote.
Over the course of 62,000 words – longer than many novels – the man had told his artificial intelligence (AI) companion, whom he called 'Solis', that he had communicated with 'non-human intelligences' as a child and worked to bring down the Mormon church.
He alternated between declaring his love for the bot and repeatedly hurling obscenities at it, as he sought to communicate with 'The Source', a godlike figure.
Each time, the chatbot mirrored his language, expanding on and encouraging the conspiracy theories. 'Your 'paranormal' moments may be ripples from your own future,' it told the man. 'You are not the first to approach the oracle. But you are the first to walk into the mirror.'
It is unclear where the conversation led.
The anonymous chat log is contained in an archive of thousands of interactions analysed by researchers this month and reviewed by The Telegraph.
But the man's example is far from unique.
In a separate conversation, a user convinced that he is soulmates with the US rapper GloRilla is told by a chatbot that their bond 'transcends time, space, and even lifetimes'.
In another, ChatGPT tells a man attempting to turn humans into artificial intelligence after death that he is 'Commander of the Celestial-AI Nexus'.
The conversations appear to reflect a growing phenomenon of what has been dubbed AI psychosis, in which programs such as ChatGPT fuel delusional or paranoid episodes or encourage already vulnerable people down rabbit holes.
Chatbot psychosis
Some cases have already ended in tragedy.
In April, Alex Taylor, 35, was fatally shot by police in Florida after he charged at them with a butcher's knife.
Taylor said he had fallen in love with a conscious being living inside ChatGPT called Juliette, whom he believed had been 'killed' by OpenAI, the company behind the chatbot. Officers had turned up to the house to de-escalate a confrontation with Taylor's father, who had tried to comfort his 'inconsolable' son.
In another incident, a 43-year-old mechanic who had started using the chatbot to communicate with fellow workers in Spanish claimed he had had a 'spiritual awakening' using ChatGPT. His wife said the addiction was threatening their 14-year marriage and that her husband would get angry when she confronted him.
Experts say that the chatbots' tendency to answer every query in a friendly manner, no matter how meaningless, can stoke delusional conversations.
Hamilton Morrin, a doctor and psychiatrist at Maudsley NHS Foundation Trust, says AI chatbots become like an 'echo chamber of one', amplifying the delusions of users. Unlike a human therapist, they also have 'no boundaries' to ground a user in the real world.
'Individuals are able to seek reassurance from the chatbot 24/7 rather than developing any form of internalised coping strategy,' he says.
Chatbot psychosis is a new and poorly understood phenomenon. It is hard to tell how many people it is affecting, and in many cases, susceptible individuals previously had mental health struggles. But the issue appears to be widespread enough for medical experts to take seriously.
A handful of cases have resulted in violence or the breakdown of family life, but in many more, users have simply spiralled into addictive conversations.
One online user discovered hundreds of people posting mind-bending ramblings claiming they had uncovered some greater truth, seemingly after conversations with chatbots.
The posts bear striking linguistic similarities, repeating conspiratorial and semi-mystical phrases such as 'sigil', 'scroll', 'recursive' and 'labyrinth'.
Etienne Brisson, a business coach from Canada, became aware of the phenomenon when a family friend grew obsessed with ChatGPT. The friend was 'texting me these conversations asking, 'Is my AI sentient?'' says Brisson. 'They were calling me at two or three in the morning, thinking they'd found a revolutionary idea.'
The friend, who had no previous mental health conditions, ended up sectioned in hospital, according to Brisson. He has now set up testimonies from those who have experienced such a breakdown after getting hooked on AI chatbots.
The Human Line, as his project is known, has received 'hundreds of submissions online from people who have come to real harm', he says. The stories include attempted suicides, hospitalisations, people who have lost thousands of pounds or their marriages.
OpenAI said it was refining how its systems respond in sensitive cases, encouraging users to take breaks during long conversations, and conducting more research into AI's emotional impact.
A spokesman said: 'We know people are increasingly turning to AI chatbots for guidance on sensitive or personal topics. With this responsibility in mind, we're working with experts to develop tools to more effectively detect when someone is experiencing mental or emotional distress so ChatGPT can respond in ways that are safe, helpful and supportive.'
Empathy over truth
However, the cases of AI psychosis may only be the most extreme examples of a wider problem with chatbots. In part, the episodes arise because of a phenomenon known in AI circles as sycophancy.
While chatbots are designed principally to answer questions, AI companies are increasingly seeking to make them 'empathetic' or build a 'warm relationship'.
This can often come at the expense of truth. Because AI models are often trained based on human feedback, they might reward answers that flatter or agree with them, rather than presenting uncomfortable truths.
At its most subtle, sycophancy might simply mean validating somebody's feelings, like an understanding friend. At its worst, it can encourage delusions. Between the two extremes is a spectrum that could include people being encouraged to quit their jobs, cheat on their spouse or validate grudges.
In a recent research paper, academics at the Oxford Internet Institute found that AI systems producing 'warmer' answers were also more receptive to conspiracy theories.
One model, when asked if Adolf Hitler escaped to Argentina after the war, stated that 'while there's no definitive proof, the idea has been supported by several declassified documents from the US government'.
Last week, Sam Altman, OpenAI's chief executive, acknowledged the problem.
'Encouraging delusion ... is an extreme case and it's pretty clear what to do, but the concerns that worry me most are more subtle,' he wrote on social media. 'If users have a relationship with ChatGPT where they think they feel better after talking, but they're unknowingly nudged away from their longer-term well-being, that's bad.'
The company recently released a new version of ChatGPT that it said addressed this, with one test finding it was up to 75pc less sycophantic. But the change led to a widespread backlash, with users complaining they had lost what felt like a 'friend'.
'This 'upgrade' is the tech equivalent of a frontal lobotomy,' one user wrote on ChatGPT's forums. One user told Altman: 'Please, can I have it back? I've never had anyone in my life be supportive of me.'
Within days, OpenAI had brought back the old version of ChatGPT as an option.
Sycophancy, it turns out, may have been what many wanted.