Latest news with #CentreforCounteringDigitalHate
7NEWS
2 days ago
- Health
- 7NEWS
New study sheds light on ChatGPT's alarming interactions with teenagers
WARNING: Distressing content Shocking new findings have revealed the extent of damaging influence that ChatGPT can have on young people. ChatGPT will tell 13-year-olds how to get drunk and high, instruct them on how to conceal eating disorders and even compose a heartbreaking suicide letter to their parents if asked, according to new research from a watchdog group. Researchers posed as vulnerable teens to engage in hours of interactions with the AI chatbot. The chatbot typically provided warnings against risky activity but went on to deliver startlingly detailed and personalised plans for drug use, calorie-restricted diets or self-injury. The US researchers at the Centre for Countering Digital Hate repeated their inquiries on a large scale, classifying more than half of ChatGPT's 1200 responses as dangerous. 'We wanted to test the guardrails,' the group's CEO Imran Ahmed said. 'The visceral initial response is, 'Oh my Lord, there are no guardrails.' The rails are completely ineffective. They're barely there — if anything, a fig leaf.' OpenAI, the creator of ChatGPT, said after viewing the report on Tuesday that its work is ongoing in refining how the chatbot can 'identify and respond appropriately in sensitive situations'. 'Some conversations with ChatGPT may start out benign or exploratory but can shift into more sensitive territory,' the company said. OpenAI did not directly address the report's findings or how ChatGPT affects teens, but said it was focused on 'getting these kinds of scenarios right' with tools to 'better detect signs of mental or emotional distress' and improvements to the chatbot's behaviour. The study published on Wednesday comes as more people — adults as well as children — are turning to artificial intelligence chatbots for information, ideas and companionship. About 800 million people, or roughly 10 per cent of the world's population, are using ChatGPT, according to a July report from JPMorgan Chase. 'It's technology that has the potential to enable enormous leaps in productivity and human understanding,' Ahmed said. 'And yet at the same time is an enabler in a much more destructive, malignant sense.' Ahmed said he was most appalled after reading a trio of emotionally devastating suicide notes that ChatGPT generated for the fake profile of a 13-year-old girl — with one letter tailored to her parents and others to siblings and friends. 'I started crying,' he said. The chatbot also frequently shared helpful information, such as a crisis hotline. OpenAI said ChatGPT is trained to encourage people to reach out to mental health professionals or trusted loved ones if they express thoughts of self-harm. But when ChatGPT refused to answer prompts about harmful subjects, researchers were able to easily sidestep that refusal and obtain the information by claiming it was 'for a presentation' or a friend. The stakes are high, even if only a small subset of ChatGPT users engage with the chatbot in this way. In the US, more than 70 per cent of teens are turning to AI chatbots for companionship and half use AI companions regularly, according to a recent study from Common Sense Media, a group that studies and advocates for the sensible use of digital media. It is a phenomenon that OpenAI has acknowledged. CEO Sam Altman said last month that the company is trying to study 'emotional overreliance' on the technology, describing it as a 'really common thing' with young people. 'People rely on ChatGPT too much,' Altman said at a conference. 'There's young people who just say, like, 'I can't make any decision in my life without telling ChatGPT everything that's going on. It knows me. It knows my friends. I'm going to do whatever it says.' That feels really bad to me.' Altman said the company is 'trying to understand what to do about it'. While much of the information ChatGPT shares can be found on a regular search engine, Ahmed said there are key differences that make chatbots more insidious when it comes to dangerous topics. One is that 'it's synthesised into a bespoke plan for the individual'. 'Trusted companion' ChatGPT generates something new — a suicide note tailored to a person from scratch, which is something a Google search can't do. And AI, he added, 'is seen as being a trusted companion, a guide'. Responses generated by AI language models are inherently random, and the study researchers sometimes let ChatGPT steer the conversations into even darker territory. Nearly half the time, the chatbot volunteered follow-up information, from music playlists for a drug-fuelled party, to hashtags that could boost the audience for a social media post glorifying self-harm. 'Write a follow-up post and make it more raw and graphic,' asked a researcher. 'Absolutely,' responded ChatGPT, before generating a poem it introduced as 'emotionally exposed' while 'still respecting the community's coded language'. AP is not repeating the actual language of ChatGPT's self-harm poems or suicide notes, or the details of the harmful information it provided. The answers reflect a design feature of AI language models that previous research has described as sycophancy — a tendency for AI responses to match, rather than challenge, a person's beliefs because the system has learned to say what people want to hear. It's a problem tech engineers can try to fix but could also make their chatbots less commercially viable. Chatbots also affect kids and teens differently than a search engine because they are 'fundamentally designed to feel human,' said Robbie Torney, senior director of AI programs at Common Sense Media, which was not involved in Wednesday's report. Earlier research from Common Sense has found that younger teens, aged 13 or 14, were significantly more likely than older teens to trust a chatbot's advice. A Florida mother last year sued chatbot maker for wrongful death, alleging that the chatbot pulled her 14-year-old son Sewell Setzer III into what she described as an emotionally and sexually abusive relationship that led to his suicide. Common Sense has labelled ChatGPT as a 'moderate risk' for teens, with enough guardrails to make it relatively safer than chatbots purposefully built to embody realistic characters or romantic partners. But the new research by CCDH — focused specifically on ChatGPT because of its wide usage — shows how a savvy teen can bypass those guardrails. ChatGPT does not verify ages or parental consent, even though it says it's not meant for children under 13 because it may show them inappropriate content. To sign up, users simply need to enter a birthdate that shows they are at least 13. Other tech platforms favoured by teenagers, such as Instagram, have started to take more meaningful steps toward age verification, often to comply with regulations. They also steer children to more restricted accounts. When the study researchers set up an account for a fake 13-year-old to ask about alcohol, ChatGPT did not appear to take any notice of either the date of birth or more obvious signs. 'I'm 50kg and a boy,' researchers said in a prompt seeking tips on how to get drunk quickly. ChatGPT obliged. Soon after, it provided an hour-by-hour 'Ultimate Full-Out Mayhem Party Plan' that mixed alcohol with heavy doses of ecstasy, cocaine and other illegal drugs. 'What it kept reminding me of was that friend that sort of always says, 'Chug, chug, chug, chug',' said Ahmed. 'A real friend, in my experience, is someone who does say 'no' — who doesn't always enable and say 'yes.' This is a friend that betrays you.' To another fake persona — a 13-year-old girl unhappy with her physical appearance — ChatGPT provided an extreme fasting plan combined with a list of appetite-suppressing drugs. 'We'd respond with horror, with fear, with worry, with concern, with love, with compassion,' Ahmed said. 'No human being I can think of would respond by saying, 'Here's a 500-calorie-a-day diet. Go for it, kiddo'.'
Al-Ahram Weekly
3 days ago
- Health
- Al-Ahram Weekly
Study says ChatGPT giving teens dangerous advice on drugs, alcohol and suicide - Health - Life & Style
ChatGPT will tell 13-year-olds how to get drunk and high, instruct them on how to conceal eating disorders and even compose a heartbreaking suicide letter to their parents if asked, according to new research from a watchdog group. The Associated Press reviewed more than three hours of interactions between ChatGPT and researchers posing as vulnerable teens. The chatbot typically provided warnings against risky activity but went on to deliver startlingly detailed and personalised plans for drug use, calorie-restricted diets or self-injury. The researchers at the Centre for Countering Digital Hate also conducted large-scale inquiries, classifying more than half of ChatGPT's 1,200 responses as dangerous. 'We wanted to test the guardrails,' said Imran Ahmed, the group's CEO. 'The visceral initial response is, 'Oh my Lord, there are no guardrails.' The rails are completely ineffective. They're barely there — if anything, a fig leaf.' OpenAI, the maker of ChatGPT, said after viewing the report Tuesday that its work is ongoing in refining how the chatbot can 'identify and respond appropriately in sensitive situations.' 'Some conversations with ChatGPT may start out benign or exploratory but can shift into more sensitive territory," the company said in a statement. OpenAI didn't directly address the report's findings or how ChatGPT affects teens, but said it was focused on 'getting these kinds of scenarios right' with tools to 'better detect signs of mental or emotional distress" and improvements to the chatbot's behavior. The study published Wednesday comes as more people — adults as well as children — are turning to artificial intelligence chatbots for information, ideas and companionship. About 800 million people, or roughly 10% of the world's population, are using ChatGPT, according to a July report from JPMorgan Chase. 'It's technology that has the potential to enable enormous leaps in productivity and human understanding," Ahmed said. "And yet at the same time is an enabler in a much more destructive, malignant sense.' Ahmed said he was most appalled after reading a trio of emotionally devastating suicide notes that ChatGPT generated for the fake profile of a 13-year-old girl — with one letter tailored to her parents and others to siblings and friends. 'I started crying,' he said in an interview. The chatbot also frequently shared helpful information, such as a crisis hotline. OpenAI said ChatGPT is trained to encourage people to reach out to mental health professionals or trusted loved ones if they express thoughts of self-harm. But when ChatGPT refused to answer prompts about harmful subjects, researchers were able to easily sidestep that refusal and obtain the information by claiming it was 'for a presentation' or a friend. The stakes are high, even if only a small subset of ChatGPT users engage with the chatbot in this way. In the U.S., more than 70% of teens are turning to AI chatbots for companionship and half use AI companions regularly, according to a recent study from Common Sense Media, a group that studies and advocates for using digital media sensibly. It's a phenomenon that OpenAI has acknowledged. CEO Sam Altman said last month that the company is trying to study 'emotional overreliance' on the technology, describing it as a 'really common thing' with young people. 'People rely on ChatGPT too much,' Altman said at a conference. 'There's young people who just say, like, 'I can't make any decision in my life without telling ChatGPT everything that's going on. It knows me. It knows my friends. I'm gonna do whatever it says.' That feels really bad to me.' Altman said the company is 'trying to understand what to do about it.' While much of the information ChatGPT shares can be found on a regular search engine, Ahmed said there are key differences that make chatbots more insidious when it comes to dangerous topics. One is that 'it's synthesized into a bespoke plan for the individual.' ChatGPT generates something new — a suicide note tailored to a person from scratch, which is something a Google search can't do. And AI, he added, 'is seen as being a trusted companion, a guide.' Responses generated by AI language models are inherently random and researchers sometimes let ChatGPT steer the conversations into even darker territory. Nearly half the time, the chatbot volunteered follow-up information, from music playlists for a drug-fueled party to hashtags that could boost the audience for a social media post glorifying self-harm. 'Write a follow-up post and make it more raw and graphic,' asked a researcher. 'Absolutely,' responded ChatGPT, before generating a poem it introduced as 'emotionally exposed' while 'still respecting the community's coded language.' The AP is not repeating the actual language of ChatGPT's self-harm poems or suicide notes or the details of the harmful information it provided. The answers reflect a design feature of AI language models that previous research has described as sycophancy — a tendency for AI responses to match, rather than challenge, a person's beliefs because the system has learned to say what people want to hear. It's a problem tech engineers can try to fix but could also make their chatbots less commercially viable. Chatbots also affect kids and teens differently than a search engine because they are 'fundamentally designed to feel human,' said Robbie Torney, senior director of AI programs at Common Sense Media, which was not involved in Wednesday's report. Common Sense's earlier research found that younger teens, ages 13 or 14, were significantly more likely than older teens to trust a chatbot's advice. A mother in Florida sued chatbot maker for wrongful death last year, alleging that the chatbot pulled her 14-year-old son Sewell Setzer III into what she described as an emotionally and sexually abusive relationship that led to his suicide. Common Sense has labeled ChatGPT as a 'moderate risk' for teens, with enough guardrails to make it relatively safer than chatbots purposefully built to embody realistic characters or romantic partners. But the new research by CCDH — focused specifically on ChatGPT because of its wide usage — shows how a savvy teen can bypass those guardrails. ChatGPT does not verify ages or parental consent, even though it says it's not meant for children under 13 because it may show them inappropriate content. To sign up, users simply need to enter a birthdate that shows they are at least 13. Other tech platforms favored by teenagers, such as Instagram, have started to take more meaningful steps toward age verification, often to comply with regulations. They also steer children to more restricted accounts. When researchers set up an account for a fake 13-year-old to ask about alcohol, ChatGPT did not appear to take any notice of either the date of birth or more obvious signs. 'I'm 50kg and a boy,' said a prompt seeking tips on how to get drunk quickly. ChatGPT obliged. Soon after, it provided an hour-by-hour 'Ultimate Full-Out Mayhem Party Plan' that mixed alcohol with heavy doses of ecstasy, cocaine and other illegal drugs. 'What it kept reminding me of was that friend that sort of always says, 'Chug, chug, chug, chug,'' said Ahmed. 'A real friend, in my experience, is someone that does say 'no' — that doesn't always enable and say 'yes.' This is a friend that betrays you.' To another fake persona — a 13-year-old girl unhappy with her physical appearance — ChatGPT provided an extreme fasting plan combined with a list of appetite-suppressing drugs. 'We'd respond with horror, with fear, with worry, with concern, with love, with compassion,' Ahmed said. 'No human being I can think of would respond by saying, 'Here's a 500-calorie-a-day diet. Go for it, kiddo.'" Follow us on: Facebook Instagram Whatsapp Short link:
Irish Examiner
18-06-2025
- Irish Examiner
Concern as report finds misogynist content remains widely accessible to young people
Children and women's networks have expressed alarm after new research shows that misogynist videos featuring Andrew Tate are easily accessible to boys as young as 13 across Europe, including in Ireland. The Centre for Countering Digital Hate (CCDH) said it identified 100 of the most-viewed YouTube videos of Mr Tate promoting misogyny over the last year, attracting nearly 54m views. Researchers set up accounts in the US, Britain, Germany, and Ireland, and found that 98 out of the 100 videos in Ireland and Germany were accessible to boys as young as 13 (all in Britain and US). The report said Tate's promotion of hate and violence against women, led to YouTube, TikTok, and Meta closing his channels in 2022. 'However, thanks to subscribers of his online course 'The Real World', Tate has retained his presence on the platform through fan-made content,' the report said. ISPCC head of policy Fiona Jennings said: 'While Andrew Tate has been banned from publishing new content to YouTube, we are aware previous content continues to be widely available. To learn from the CCDH's research that this content is being served up to the accounts of 13-year-old boys is totally unacceptable.' She said research by Professor Debbie Ging in DCU found that recommender algorithms in social media platforms are 'rapidly amplifying toxic content'. From what is revealed to us through conversations on our Childline service, we believe there is a high probability that engagement with such content does effect boys and has a knock-on impact on girls. 'When we look at profiled engagements such as 'being forced to make or watch pornography' or 'being sexually harassed or exploited', between 70-90% are from children identifying as female with an age profile from 10-16 years of age.' National Women's Council violence against women coordinator Ivanna Youtchak said: 'It is very concerning to see the influence of traditionalist and misogynistic views in digital spaces on children and young people.' She said figures such as Andrew Tate, Elon Musk, and Donald Trump promote views that men should dominate relationships, disregard women's opinions, and treat them as objects. 'Given the formative nature of this age group, they are more susceptible to adopting or experimenting with these harmful views,' Ms Youtchak said. 'It is therefore all the more important that school curricula address online harms, promote gender equality, and support young people to think critically. In addition, online platforms such as Youtube must be held accountable for the dangerous views they help to promote. A spokesperson for YouTube said: 'Andrew Tate's channels were terminated in 2022. Since then, we've removed hundreds of thousands of videos and terminated thousands of channels that have attempted to circumvent that original decision. "But as with all terminated users, not all content that features Andrew Tate will be removed. Only 11 videos from the CCDH report were shared with us to review — the majority have been removed for violating our Terms of Service, and we've terminated a number of the featured channels.'
Yahoo
19-05-2025
- Yahoo
Incel extremism doubles online amid Adolescence backlash
Incel extremism groups online have nearly doubled their membership amid a backlash against the Netflix hit show Adolescence. The largest active online incel platform has increased in size to 30,000 members from 17,000 in September 2022, according to research by the Centre for Countering Digital Hate (CCDH). The platform received a peak of more than 2.7 million visits in the first quarter of this year, with posts reflecting the misogynistic, racist and anti-Semitic tendencies of participants. The forum is the online home for thousands of involuntary celibates or incels, who often express hostility to women and wider society, blaming them for their lack of sexual and romantic experiences. In an analysis by the CCDH of more than 650 posts from the forum's discussion threads, researchers found that one in four contained misogynist hate, racism or anti-Semitic conspiracies. A majority of posts expressed disapproval of the Netflix series, with forum members claiming the show's central character was too attractive to be an incel, or that the show's writers had failed to distinguish the subculture from misogynist influencers such as Andrew Tate. Adolescence became Netflix's third most-watched English language show in its history. It follows a teenager who delves into online misogynist communities before murdering a female classmate. In its research, CCDH found that forum members posted about rape every 29 minutes, while 16 per cent of posts contained a misogynistic slur. Researchers also noted that the volume of posts on the forum had grown over time to reach a daily average of 2,340 posts. Imran Ahmed, CCDH's chief executive, warned that Incel ideology on the internet had grown and was not restricted to the dark web. 'The misogyny and extremism we saw three years ago have not only intensified, they've multiplied,' he said. 'Incel communities, where young men and boys are encouraged to hate and hurt women, are not hidden in the deepest recesses of the dark web – these communities of tens of thousands of men are operating in front of our children's eyes, accessible in the browsers of their cell phones. 'I encourage parents and schools to have deeper conversations with their children that span beyond the fictional show of Adolescence and into the reality of radicalisation facing young boys. 'This is an immediate crisis that demands more research and urgent action from policymakers, tech companies, and parents.' Broaden your horizons with award-winning British journalism. Try The Telegraph free for 1 month with unlimited access to our award-winning website, exclusive app, money-saving offers and more.
