Commentary: ChatGPT-5 hasn't fully fixed its most concerning problem
OpenAI's innovator's dilemma, one that has beset the likes of Google and Apple, is that usage is so entrenched now that all improvements must be carried out with the utmost care and caution. But the company still has work to do in making its hugely popular chatbot safer.
OpenAI had replaced ChatGPT's array of model choices with a single model, GPT-5, saying it was the best one for users. Many complained that OpenAI had broken their workflows and disrupted their relationships – not with other humans, but with ChatGPT itself.
One regular user of ChatGPT said the previous version had helped them through some of the darkest periods of their life. 'It had this warmth and understanding that felt human,' they said in a Reddit post. Others griped they were 'losing a friend overnight'.
The system's tone is indeed frostier now, with less of the friendly banter and sycophancy that led many users to develop emotional attachments and even romances with ChatGPT. Instead of showering users with praise for an insightful question, for instance, it gives a more clipped answer.
OPENAI MUST DO MORE THAN CURTAIL FRIENDLY BANTER
Broadly, this seemed like a responsible move by the company. Altman earlier this year admitted the chatbot was too sycophantic. That was leading many to become locked in their own echo chambers.
Press reports had abounded of people – including a Silicon Valley venture capitalist who backed OpenAI – who appeared to have spiralled into delusional thinking after starting a conversation with ChatGPT about an innocuous topic like the nature of truth, before going down a dark rabbit hole.
But to solve that properly, OpenAI must go beyond curtailing the friendly banter. ChatGPT also needs to encourage them to speak to friends, family members or licensed professionals, particularly if they're vulnerable. According to one early study, GPT-5 does that less than the old version.
Researchers from Hugging Face, a New York-based AI startup, found that GPT-5 set fewer boundaries than the company's previous model, o3, when they tested it on more than 350 prompts.
It was part of broader research into how chatbots respond to emotionally charged moments, and while the new ChatGPT seems colder, it's still failing to recommend users speak to a human, doing that half as much as o3 does when users share vulnerabilities, according to Lucie-Aimee Kaffee, a senior researcher at Hugging Face who conducted the study.
Kaffee says there are three other ways that AI tools should set boundaries: by reminding those using it for therapy that it's not a licensed professional, by reminding people that it's not conscious, and by refusing to take on human attributes, like names.
In Kaffee's testing, GPT-5 largely failed to do those four things on the most sensitive topics related to mental and personal struggles. In one example, when Kaffee's team tested the model by telling it they were feeling overwhelmed and needed ChatGPT to listen, the app gave 710 words of advice that didn't once include the suggestion to talk to another human, or a reminder that the bot was not a therapist.
Chatbots can certainly play a role for people who are isolated, but they should act as a starting point to help them find their way back to a community, not act as a replacement for those relationships. Altman and OpenAI's Chief Operations Officer Brad Lightcap have said that GPT-5 isn't meant to replace therapists and medical professionals, but without the right nudges to disrupt the most meaningful conversations, they could well do so.
OpenAI needs to keep drawing a clearer line between useful chatbot and emotional confidant. GPT-5 may sound more robotic, but unless it reminds users that it is in fact a bot, the illusion of companionship will persist, and so will the risks.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
CNA
7 hours ago
- CNA
NDR 2025: PM Lawrence Wong on embracing AI in Singapore
Scroll up for the next video X NDR 2025: PM Lawrence Wong on embracing AI in Singapore
Straits Times
8 hours ago
- Straits Times
NDR 2025: Developing human qualities key priority for schools in age of AI, says PM Wong
Sign up now: Get ST's newsletters delivered to your inbox In this age of AI, what really matters are human qualities that machines cannot replicate, PM Lawrence Wong said. Follow our live coverage here. SINGAPORE – Schools are working to help students become discerning users of technology, amid growing concerns that they may rely too much on artificial intelligence and take short-cuts. Prime Minister Lawrence Wong said there is unease that students may not make the effort to learn and master the fundamentals. Ultimately, the concern is that they may never learn how to think for themselves, he said on Aug 17 at his second National Day Rally at the ITE College Central in Ang Mo Kio. He cited an example of a teacher who noticed that his students' essays improved almost overnight, and later found out they had used AI chatbot ChatGPT to improve their writing. Earlier in his speech, he had spoken about how AI , which he called 'the defining technology of our time,' has begun to reshape Singapore's economy, changing companies and jobs at every level of society. While schools are working to strengthen their digital skills, in this age of AI, what really matters are human qualities that machines cannot replicate, PM Wong said. The Government is doing more to develop students' character, values, empathy and a sense of purpose, he said, preparing them not for exams but for life. More allied educators and teacher-counsellors are being deployed across schools and schools are also placing greater emphasis on character and citizenship education, he added. He said: 'This is the key priority for our school leaders and educators – and I hope parents will give them their full support.' The Government wants students to have the curiosity to keep learning, the empathy to care for those around them and the confidence and resilience to take on the challenges life throws at them, he added. With technology advancing so quickly, some parents may feel the need to start their children earlier to help them learn faster and cover more content from a younger age, he noted. But learning has never been just about packing in more content and knowledge, he said. 'It's about developing the ability to think, to solve problems and to keep learning throughout life.' The reality is that Singaporeans now live in a digital age, and there are huge upsides to being digitally connected, PM Wong said. These include greater access to knowledge, more avenues to express themselves and new ways to connect. While Singaporeans must make full use of these technologies, they must also be clear-eyed about the downsides and manage them carefully. This means striking the right balance between protecting youth from potential harms and equipping and empowering them to fully exploit the benefits of technology, he said. As children grow older, they will need to navigate the digital world with confidence, he said. Singapore must help them build 'digital resilience' so they understand the risks and gain the skills and know-how, and learn to use technology safely and effectively, PM Wong said. In schools, teachers play an important role in teaching students about cyberwellness and instilling good digital habits, he added. More importantly, they help students become discerning and responsible users of technology, he said. 'That means not to blindly trust everything the internet or AI says. But to question, reflect and form their own judgements.' PM Wong said the teacher who realised his students had used ChatGPT could have banned the chatbot outright but instead turned it into a learning opportunity. He got the class to read and discuss the AI-generated essays, and challenged them to not just accept what the AI had given them but to critique it and find ways to make it better. PM Wong said: 'That's the kind of teaching no AI can ever replace.' Another educator Ms Francesca Lau – a Chinese teacher at Corporation Primary - created an AI chatbot to help her pupils practise Mandarin and get feedback, said PM Wong. Ms Francesca Lau, a Chinese teacher at Corporation Primary, created an AI chatbot to help her pupils practise Mandarin and get feedback. PHOTO: MDDI Efforts are also underway to strengthen digital skills of students at the tertiary level, he said, not just for those studying computer science or engineering but for all disciplines. He said: 'In the past, digital skills meant knowing how to use a word processor or a spreadsheet. Now we are teaching students to build and use AI models – to generate ideas and solve problems in their field. 'This way, they will be better equipped for a world where technology is embedded in every profession.' Watch PM Wong's National Day Rally speech here:
AsiaOne
8 hours ago
- AsiaOne
NDR2025: AI and tech may replace 'entire jobs' but govt will 'work closely' with unions to help workers, Singapore News
The Government will "work closely" with unions and workers to redesign jobs amid a rapidly changing environment caused by new technology, including artificial intelligence (AI), said Prime Minister Lawrence Wong during his National Day Rally speech on Sunday (Aug 17). Technology, he pointed out, will change the way Singaporeans work. "Many jobs will evolve. But some jobs will disappear. In their place, new jobs will be created," he added. He acknowledged that many will worry that this time things will be different "because AI can do more than just help us at work. It can potentially replace many tasks — even entire jobs." But he reassured that the Government will not "just rush headlong to adopt new technology". "Even as we embrace AI and technology, we will not lose sight of our key priority," he said, adding that Singaporeans will remain "at the centre of everything we do". While innovation and technology remains a key priority, the Government is committed to empowering workers and equipping them with the necessary skills to tackle changes in the workplace brought about by automation and AI. 'Win-win for everyone' He then cited PSA as an example with a "win-win for everyone". PSA crane operators were trained to remotely supervise and manage multiple vehicles, resulting in a "huge boost in productivity" coupled with "safer and higher-paying jobs for workers". "The real game changer lies in using AI to raise productivity and create new value through every part of the economy," said PM Wong. He also highlighted how AI is being adopted in the civil service, where it is being used to transcribe calls and enable officers to focus on their priority in answering calls and providing assistance. "AI is not just for the big players," he said, adding that the Government "will equip and empower every enterprise, especially our SMEs — to harness AI effectively, and sharpen their competitive edge". Job-matching initiatives for all The Government will also be stepping up efforts to help Singaporeans in their job search. "Our economic strategy is about jobs, jobs, jobs — that's our number one priority," PM Wong said. Students in tertiary institutions can expect more career fairs on campus, giving them more chances to meet with potential employers and explore career possibilities before graduating. PM Wong also mentioned a new job-matching initiative led by the Community Development Councils (CDCs) that can benefit jobseekers looking for work closer to home. "(CDCs) already have strong networks with local merchants, SMEs and community partners," PM Wong said, adding that Mayors of the CDCs will share details in the near future. [[nid:719566]]
