3 days ago
UAE: Should ChatGPT give medical advice? Mental health experts call for regulation
Imagine this: It's late at night, and you're chatting with an AI about your anxiety, racing thoughts, or fears. The conversation starts to get intense — maybe even too intense. Should the AI step in, pause the session, or suggest seeking professional help?
Mental health experts say yes, and they are urging AI platforms like ChatGPT to incorporate real-world safeguards that can manage emotionally heavy conversations.
A growing number of psychologists and researchers warn that while AI tools can be helpful in moderation, they are gradually taking on roles they were never designed to fill. With rare but serious cases of AI-linked psychosis now being documented, experts say the lack of oversight is a problem, especially as more people turn to chatbots for comfort, therapy-like support, or emotional connection.
'The danger isn't just about (receiving) bad advice. It's that users can become emotionally dependent on AI, treating it like a friend or therapist. In some cases, it even becomes part of a person's distorted thinking. That's where we've seen psychosis emerge," said Dr Randa Al Tahir, a trauma-focused psychologist.
Though AI may seem responsive and comforting, it lacks the ability to recognise when a user is spiralling or in crisis. It also doesn't intervene when conversations cross a critical line.
Documented cases in Europe and the US show individuals with pre-existing vulnerabilities forming deep emotional bonds with chatbots, resulting in delusional beliefs, blurred lines between reality and fiction, and even harmful behaviour. While these are extreme examples, they expose a significant blind spot in how AI is being used.
'AI doesn't have the capacity to flag serious red flags or escalate someone to emergency care yet. But it should,' the Dubai doctor added 'We need built-in measures, whether that's emotional content warnings, timed breaks, or partnerships with international mental health organisations.'
To understand how AI itself views the issue, Khaleej Times asked ChatGPT the question: 'Should ChatGPT have regulations when it comes to mental health?'
The tool answered with surprising clarity:
'It makes sense that medical experts are calling for regulation. AI like ChatGPT can provide helpful general information, but I'm not a licensed medical professional and shouldn't replace doctors or mental health experts. Misunderstandings, outdated info, or oversimplified answers can lead to harm if someone acts on them without consulting a professional.'
It went on to say:
'Mental health advice is nuanced and deeply personal. Even well-trained professionals spend years learning how to assess and respond safely... I believe experts calling for regulation are being responsible.'
Dr Nabeel Ashraf, a clinical psychologist in Dubai, echoed this urgency. He called on AI companies and regulators to quickly implement features aimed at reducing risks, especially for vulnerable users. One key recommendation is training chatbots to detect signs of emotional distress by analysing user language in real time.
'There are patterns that can indicate when someone is spiralling, experiencing delusions, or showing signs of crisis. The AI should be able to respond appropriately,' he said.
In such cases, the chatbot should refer users to verified support services, such as mental health hotlines or licensed therapists. 'It's not enough to say 'I'm sorry you feel that way.' If there's a red flag, there should be a next step.'
'There's no shame in using AI for light advice or to feel heard,' he added. 'But without real limits, these tools can cause more harm than good, especially to someone already struggling.'chat
