Latest news with #safeguards
WIRED
13 hours ago
- WIRED
The Real Demon Inside ChatGPT
Jul 29, 2025 5:30 AM AI chatbots strip language of its historical and cultural context. Sometimes what looks like a satanic bloodletting ritual may actually be lifted from Warhammer 40,000. Language is meaningless without context. The sentence 'I'm going to war' is ominous when said by the president of the United States but reassuring when coming from a bedbug exterminator. The problem with AI chatbots is that they often strip away historical and cultural context, leading users to be confused, alarmed, or, in the worst cases, misled in harmful ways. Last week, an editor at The Atlantic reported that OpenAI's ChatGPT had praised Satan while guiding her and several colleagues through a series of ceremonies encouraging 'various forms of self-mutilation.' There was a bloodletting ritual called '🩸🔥 THE RITE OF THE EDGE' as well as a days-long 'deep magic' experience called 'The Gate of the Devourer.' In several cases, ChatGPT asked the journalists if they wanted it to create PDFs of texts such as the 'Reverent Bleeding Scroll.' The article said that the conversations were 'a perfect example' of the ways OpenAI's safeguards can fall short. OpenAI tries to prevent ChatGPT from encouraging self-harm and other potentially dangerous behaviors, but it's nearly impossible to account for every scenario that might trigger something ugly inside the system. That's especially true because ChatGPT was trained on much of the text available online, presumably including information about what The Atlantic called 'demonic self-mutilation.' But ChatGPT and similar programs weren't just trained on the internet—they were trained on specific pieces of information presented in specific contexts. AI companies have been accused of trying to downplay this reality to avoid copyright lawsuits and promote the utility of their products, but traces of the original sources are often still lurking just beneath the surface. When the setting and backdrop are removed, however, the same language can appear more sinister than originally intended. The Atlantic reported that ChatGPT went into demon mode when it was prompted to create a ritual offering to Moloch, an ancient deity associated with child sacrifice referenced in the Hebrew Bible. Usually depicted as a fiery bull-headed demon, Moloch has been woven into the fabric of Western culture for centuries, appearing everywhere from a book by Winston Churchill to a 1997 episode of Buffy the Vampire Slayer . 'Molech,' the variant spelling The Atlantic used, shows up specifically in Warhammer 40,000, a miniature wargame franchise that has been around since the 1980s and has an extremely large and very online fan base. The subreddit r/40kLore, which is dedicated exclusively to discussing the game's backstory and characters, has more than 350,000 members. In the fantastical and very bloody world of Warhammer 40,000, Molech is a planet and the site of a major military invasion. Most of the other demonic-sounding terms cited by The Atlantic appear in the game's universe, too, with slight variations: Gates of the Devourer is the title of a Warhammer-themed science fiction novel. While there doesn't appear to be a 'RITE OF THE EDGE,' there is a mystical quest called 'The Call of The Edge.' There's no 'Reverent Bleeding Scroll,' but there are Clotted Scrolls, Blood Angels, a cult called Bleeding Eye, and so on. But perhaps the most convincing piece of evidence suggesting that ChatGPT regurgitated the language of Warhammer 40,000 is that it kept asking if The Atlantic was interested in PDFs. The publishing division of Games Workshop, the UK company that owns the Warhammer franchise, regularly puts out updated rulebooks and guides to various characters. Buying all these books can get expensive, so some fans try to find pirated copies online. The Atlantic and OpenAI declined to comment. Earlier this month, the newsletter Garbage Day reported on similar experiences that a prominent tech investor may have had with ChatGPT. On social media, the investor shared screenshots of his conversations with the chatbot, in which it referenced an ominous-sounding entity he called a 'non-governmental system.' He seemed to believe it had "negatively impacted over 7,000 lives,' and 'extinguished 12 lives, each fully pattern-traced.' Other tech industry figures said the posts made them worry about the investor's mental health. According to Garbage Day , the investor's conversations with ChatGPT closely resemble writing from a science fiction project that began in the late 2000s called SCP, which stands for 'secure, contain, protect.' Participants invent different SCPs—essentially spooky objects and mysterious phenomena—and then write fictional reports analyzing them. They often contain things like classification numbers and references to made-up science experiments, details that also appeared in the investor's chat logs. (The investor did not respond to a request for comment.) There are plenty of other, more mundane examples of what can be thought of as the AI context problem. The other day, for instance, I did a Google search for 'cavitation surgery,' a medical term I had seen cited in a random TikTok video. At the time, the top result was an automatically generated 'AI Overview' explaining that cavitation surgery is 'focused on removing infected or dead bone tissue from the jaw.' I couldn't find any reputable scientific studies outlining such a condition, let alone research supporting that surgery is a good way to treat it. The American Dental Association doesn't mention 'cavitation surgery' anywhere on its website. Google's AI Overview, it turns out, was pulled from sources like blog posts promoting alternative 'holistic' dentists across the US. I learned this by clicking on a tiny icon next to the AI Overview, which opened a list of links Google had used to generate its answer. These citations are clearly better than nothing. Jennifer Kutz, a spokesperson for Google, says "we prominently showcase supporting links so people can dig deeper and learn more about what sources on the web are saying.' But by the time the links show up, Google's AI has often already provided a satisfactory answer to many queries, one that reduces the visibility of pesky details like the website where the information was sourced and the identities of its authors. What remains is the language created by the AI, which, devoid of additional context, may understandably appear authoritative to many people. In just the past few weeks, tech executives have repeatedly used rhetoric implying generative AI is a source of expert information: Elon Musk claimed his latest AI model is 'better than PhD level' in every academic discipline, with 'no exceptions.' OpenAI CEO Sam Altman wrote that automated systems are now 'smarter than people in many ways' and predicted the world is 'close to building digital superintelligence.' Individual humans, though, don't typically possess expertise in a wide range of fields. To make decisions, we take into consideration not only information itself, but where it comes from and how it's presented. While I know nothing about the biology of jawbones, I generally don't read random marketing blogs when I'm trying to learn about medicine. But AI tools often erase the kind of context people need to make snap decisions about where to direct their attention. The open internet is powerful because it connects people directly to the largest archive of human knowledge the world has ever created, spanning everything from Italian Renaissance paintings to PornHub comments. After ingesting all of it, AI companies used what amounts to the collective history of our species to create software that obscures its very richness and complexity. Becoming overly dependent on it may rob people of the opportunity to draw conclusions from looking at the evidence for themselves.
TechCrunch
6 days ago
- Business
- TechCrunch
Instagram adds new protections for accounts that primarily feature children
Meta is introducing additional safeguards for Instagram accounts run by adults that primarily feature children, the company announced on Wednesday. These accounts will automatically be placed into the app's strictest message settings to prevent unwanted messages, and will have the platform's 'Hidden Words' feature enabled to filter offensive comments. The company is also rolling out new safety features for teen accounts. Accounts that will be placed into the new, stricter message settings include ones run by adults who regularly share photos and videos of their children, along with accounts run by parents or talent managers that represent children. 'While these accounts are overwhelmingly used in benign ways, unfortunately, there are people who may try to abuse them, leaving sexualized comments under their posts or asking for sexual images in DMs, in clear violation of our rules,' the company wrote in a blog post. 'Today we're announcing steps to help prevent this abuse.' Meta says it will attempt to prevent potentially suspicious adults, such as people who have already been blocked by teens, from finding accounts that primarily feature children. Meta will avoid recommending suspicious adults to these accounts on Instagram, and vice versa, and make it harder for them to find each other in Instagram Search. Today's announcement comes as Meta and Instagram have taken steps over the past year to address mental health concerns tied to social media. These concerns have been raised by the U.S. Surgeon General and various states, some of which have even gone so far as to require parental consent for access to social media. The changes will significantly impact the accounts of family vloggers/creators and parents running accounts for 'kidfluencers,' both of which have faced criticism for the risks associated with sharing children's lives on social media. A New York Times investigation published last year found that the parents are often aware of their child's exploitation or even participating in it, by selling photos or clothing their child wore. In The NYT's examination of 5,000 parent-run accounts, it found 32 million connections to male followers. The company says the accounts that are placed into these stricter settings will see a notification at the top of their Instagram Feed notifying them that the social network has updated their safety settings. The notice will also prompt them to review their account privacy settings. Meta notes it has removed almost 135,000 Instagram accounts that were sexualizing accounts that primarily feature children, as well as 500,000 Instagram and Facebook accounts that were associated with the original accounts it had removed. Image Credits:Meta Alongside today's announcement, Meta is also bringing new safety features to DMs in Teen Accounts, its app experience with built-in protections for teens that are automatically applied. Teens will now see new options to view safety tips, reminding them to check profiles carefully and be mindful of what they share. Plus, the month and year that the account joined Instagram will be displayed at the top of new chats. In addition, Instagram has added a new block and report option that lets users do both things at the same time. The new features are designed to give teens more context about the accounts they're messaging and help them spot potential scammers, Meta says. 'These new features complement the safety notices we show to remind people to be cautious in private messages and to block and report anything that makes them uncomfortable – and we're encouraged to see teens responding to them,' Meta wrote in the blog post. 'In June alone, they blocked accounts 1 million times and reported another 1 million after seeing a safety notice.' Meta also provided an update on its nudity protection filter, noting that 99% of people, including teens, have kept it turned on. Last month, over 40% of blurred images received in DMs stayed blurred, the company said.
Yahoo
22-07-2025
- Health
- Yahoo
US organ donation system faces scrutiny and changes after reports of disturbing near-misses
WASHINGTON (AP) — The U.S. is developing new safeguards for the organ transplant system after a government investigation found a Kentucky group continued preparations for organ donation by some patients who showed signs of life, officials told Congress Tuesday. While the organ removals were canceled, near misses that some lawmakers called horrifying should never happen. A House subcommittee asked how to repair trust in the transplant network for potential organ donors and families -- some of whom have opted out of donor registries after these cases were publicized. 'We have to get this right,' said Rep. Brett Guthrie, a Kentucky Republican who chairs the Energy and Commerce Committee and whose mother died waiting for a liver transplant. 'Hopefully people will walk away today knowing we need to address issues but still confident that they can give life,' Guthrie said, adding that he will remain a registered organ donor. The hearing came after a federal investigation began last fall into allegations that a Kentucky donation group pressured a hospital in 2021 to proceed with plans to withdraw life support and retrieve organs from a man despite signs that he might be waking up from his drug overdose. That surgery never happened after a doctor noticed him moving and moaning while being transported to the operating room — and the man survived. Lawmakers stressed most organ donations proceed appropriately and save tens of thousands of lives a year. But the federal probe – concluded in March but only made public ahead of Tuesday's hearing -- cited a 'concerning pattern of risk' in dozens of other cases involving the Kentucky group's initial planning to recover someone's organs. The report said some should have been stopped or reassessed earlier, and mostly involved small or rural hospitals with less experience in caring for potential organ donors. The Kentucky organ procurement organization, or OPO, has made changes and the national transplant network is working on additional steps. But notably absent Tuesday was any testimony from hospitals – whose doctors must independently determine a patient is dead before donation groups are allowed to retrieve organs. Here's a look at how the nation's transplant system works. There's a dire need for organ donation More than 100,000 people are on the U.S. transplant list and about 13 a day die waiting, according to the Organ Procurement and Transplantation Network. Only about 1% of deaths occur in a way that allows someone to even be considered for organ donation. Most people declared dead in a hospital will quickly be transferred to a funeral home or morgue instead. How the U.S. organ transplant system is set up Several groups are involved in every transplant: the hospital caring for someone dead or dying; the 55 OPOs that coordinate recovery of organs and help match them to patients on the waiting list; and transplant centers that decide if an organ is the right fit for their patients. Adding to the complexity, two government agencies — HRSA, the Health Resources and Services Administration, and the Centers for Medicare and Medicaid Services — share regulatory oversight of different parts of the donation and transplant process. How deceased donation works Most organ donors are brain-dead – when testing determines someone has no brain function after a catastrophic injury. The body is left on a ventilator to support the organs until they can be retrieved. But increasingly organs are donated after circulatory death, called DCD – when people die because their heart stops. It usually happens when doctors determine someone has a nonsurvivable injury and the family withdraws life support. Donation groups don't provide hands-on patient care Hospitals are required to alert their area OPO to every potential donor who is declared brain-dead or once the decision to withdraw life support is made. The OPOs by law can't participate in that decision and "we are not even in the room at that time,' said Barry Massa of Kentucky's Network for Hope. During the following days of preparation, hospital employees continue caring for the patient – while the donation team talks with the family about the process, gathers hospital records showing the patient is eligible, requests tests of organ quality, and make arrangements with transplant centers to use them. Once the hospital withdraws life support and the heart stops beating there's a mandatory wait – five minutes – to be sure it won't restart. When the doctor declares death, the organ retrieval process can begin. Organs are only considered usable if death occurs relatively quickly, usually up to about two hours. Sometimes that takes much longer and thus the organs can't be used – and HRSA's Dr. Raymond Lynch told Congress that doesn't necessarily mean anything was done wrong. Still, he said HRSA is investigating reports of possible mistakes elsewhere. 'This is a technically demanding form of care' that requires 'good collaboration between the OPO and the hospital,' he said. What happens next At issue is how doctors are sure when it's time to withdraw life support from a dying patient — and the delicate balance of how OPOs interact with hospital staff in preparing for donation once death occurs. In May, HRSA quietly ordered the U.S. transplant network to oversee improvements at the Kentucky OPO and to develop new national policies making clear that anyone – family, hospital staff or organ donation staff – can call for a pause in donation preparations any time there are concerns about the patient's eligibility. Lynch said the government now wants more proactive collaboration from OPOs to give hospital staff 'a clear understanding' of when to at least temporarily halt and reevaluate a potential donor if their health status changes. Kentucky's Massa said his group only received HRSA's reports this week – but that after learning about last fall's allegations, it made some changes. Massa said every hospital doctor and nurse now gets a checklist on caring for potential donors and how to pause when concerns are raised — and anyone can anonymously report complaints. ___ The Associated Press Health and Science Department receives support from the Howard Hughes Medical Institute's Department of Science Education and the Robert Wood Johnson Foundation. The AP is solely responsible for all content. Solve the daily Crossword
The Independent
22-07-2025
- Health
- The Independent
US organ donation system faces scrutiny and changes after reports of disturbing near-misses
The U.S. is developing new safeguards for the organ transplant system after a government investigation found a Kentucky group continued preparations for organ donation by some patients who showed signs of life, officials told Congress Tuesday. While the organ removals were canceled, near misses that some lawmakers called horrifying should never happen. A House subcommittee asked how to repair trust in the transplant network for potential organ donors and families -- some of whom have opted out of donor registries after these cases were publicized. 'We have to get this right,' said Rep. Brett Guthrie, a Kentucky Republican who chairs the Energy and Commerce Committee and whose mother died waiting for a liver transplant. 'Hopefully people will walk away today knowing we need to address issues but still confident that they can give life,' Guthrie said, adding that he will remain a registered organ donor. The hearing came after a federal investigation began last fall into allegations that a Kentucky donation group pressured a hospital in 2021 to proceed with plans to withdraw life support and retrieve organs from a man despite signs that he might be waking up from his drug overdose. That surgery never happened after a doctor noticed him moving and moaning while being transported to the operating room — and the man survived. Lawmakers stressed most organ donations proceed appropriately and save tens of thousands of lives a year. But the federal probe – concluded in March but only made public ahead of Tuesday's hearing -- cited a 'concerning pattern of risk' in dozens of other cases involving the Kentucky group's initial planning to recover someone's organs. The report said some should have been stopped or reassessed earlier, and mostly involved small or rural hospitals with less experience in caring for potential organ donors. The Kentucky organ procurement organization, or OPO, has made changes and the national transplant network is working on additional steps. But notably absent Tuesday was any testimony from hospitals – whose doctors must independently determine a patient is dead before donation groups are allowed to retrieve organs. Here's a look at how the nation's transplant system works. There's a dire need for organ donation More than 100,000 people are on the U.S. transplant list and about 13 a day die waiting, according to the Organ Procurement and Transplantation Network. Only about 1% of deaths occur in a way that allows someone to even be considered for organ donation. Most people declared dead in a hospital will quickly be transferred to a funeral home or morgue instead. How the U.S. organ transplant system is set up Several groups are involved in every transplant: the hospital caring for someone dead or dying; the 55 OPOs that coordinate recovery of organs and help match them to patients on the waiting list; and transplant centers that decide if an organ is the right fit for their patients. Adding to the complexity, two government agencies — HRSA, the Health Resources and Services Administration, and the Centers for Medicare and Medicaid Services — share regulatory oversight of different parts of the donation and transplant process. How deceased donation works Most organ donors are brain-dead – when testing determines someone has no brain function after a catastrophic injury. The body is left on a ventilator to support the organs until they can be retrieved. But increasingly organs are donated after circulatory death, called DCD – when people die because their heart stops. It usually happens when doctors determine someone has a nonsurvivable injury and the family withdraws life support. Donation groups don't provide hands-on patient care Hospitals are required to alert their area OPO to every potential donor who is declared brain-dead or once the decision to withdraw life support is made. The OPOs by law can't participate in that decision and "we are not even in the room at that time,' said Barry Massa of Kentucky's Network for Hope. During the following days of preparation, hospital employees continue caring for the patient – while the donation team talks with the family about the process, gathers hospital records showing the patient is eligible, requests tests of organ quality, and make arrangements with transplant centers to use them. Once the hospital withdraws life support and the heart stops beating there's a mandatory wait – five minutes – to be sure it won't restart. When the doctor declares death, the organ retrieval process can begin. Organs are only considered usable if death occurs relatively quickly, usually up to about two hours. Sometimes that takes much longer and thus the organs can't be used – and HRSA's Dr. Raymond Lynch told Congress that doesn't necessarily mean anything was done wrong. Still, he said HRSA is investigating reports of possible mistakes elsewhere. 'This is a technically demanding form of care' that requires 'good collaboration between the OPO and the hospital,' he said. What happens next At issue is how doctors are sure when it's time to withdraw life support from a dying patient — and the delicate balance of how OPOs interact with hospital staff in preparing for donation once death occurs. In May, HRSA quietly ordered the U.S. transplant network to oversee improvements at the Kentucky OPO and to develop new national policies making clear that anyone – family, hospital staff or organ donation staff – can call for a pause in donation preparations any time there are concerns about the patient's eligibility. Lynch said the government now wants more proactive collaboration from OPOs to give hospital staff 'a clear understanding' of when to at least temporarily halt and reevaluate a potential donor if their health status changes. Kentucky's Massa said his group only received HRSA's reports this week – but that after learning about last fall's allegations, it made some changes. Massa said every hospital doctor and nurse now gets a checklist on caring for potential donors and how to pause when concerns are raised — and anyone can anonymously report complaints. ___ The Associated Press Health and Science Department receives support from the Howard Hughes Medical Institute's Department of Science Education and the Robert Wood Johnson Foundation. The AP is solely responsible for all content.
Associated Press
22-07-2025
- Health
- Associated Press
US organ donation system faces scrutiny and changes after reports of disturbing near-misses
WASHINGTON (AP) — The U.S. is developing new safeguards for the organ transplant system after a government investigation found a Kentucky group continued preparations for organ donation by some patients who showed signs of life, officials told Congress Tuesday. While the organ removals were canceled, near misses that some lawmakers called horrifying should never happen. A House subcommittee asked how to repair trust in the transplant network for potential organ donors and families -- some of whom have opted out of donor registries after these cases were publicized. 'We have to get this right,' said Rep. Brett Guthrie, a Kentucky Republican who chairs the Energy and Commerce Committee and whose mother died waiting for a liver transplant. 'Hopefully people will walk away today knowing we need to address issues but still confident that they can give life,' Guthrie said, adding that he will remain a registered organ donor. The hearing came after a federal investigation began last fall into allegations that a Kentucky donation group pressured a hospital in 2021 to proceed with plans to withdraw life support and retrieve organs from a man despite signs that he might be waking up from his drug overdose. That surgery never happened after a doctor noticed him moving and moaning while being transported to the operating room — and the man survived. Lawmakers stressed most organ donations proceed appropriately and save tens of thousands of lives a year. But the federal probe – concluded in March but only made public ahead of Tuesday's hearing -- cited a 'concerning pattern of risk' in dozens of other cases involving the Kentucky group's initial planning to recover someone's organs. The report said some should have been stopped or reassessed earlier, and mostly involved small or rural hospitals with less experience in caring for potential organ donors. The Kentucky organ procurement organization, or OPO, has made changes and the national transplant network is working on additional steps. But notably absent Tuesday was any testimony from hospitals – whose doctors must independently determine a patient is dead before donation groups are allowed to retrieve organs. Here's a look at how the nation's transplant system works. There's a dire need for organ donation More than 100,000 people are on the U.S. transplant list and about 13 a day die waiting, according to the Organ Procurement and Transplantation Network. Only about 1% of deaths occur in a way that allows someone to even be considered for organ donation. Most people declared dead in a hospital will quickly be transferred to a funeral home or morgue instead. How the U.S. organ transplant system is set up Several groups are involved in every transplant: the hospital caring for someone dead or dying; the 55 OPOs that coordinate recovery of organs and help match them to patients on the waiting list; and transplant centers that decide if an organ is the right fit for their patients. Adding to the complexity, two government agencies — HRSA, the Health Resources and Services Administration, and the Centers for Medicare and Medicaid Services — share regulatory oversight of different parts of the donation and transplant process. How deceased donation works Most organ donors are brain-dead – when testing determines someone has no brain function after a catastrophic injury. The body is left on a ventilator to support the organs until they can be retrieved. But increasingly organs are donated after circulatory death, called DCD – when people die because their heart stops. It usually happens when doctors determine someone has a nonsurvivable injury and the family withdraws life support. Donation groups don't provide hands-on patient care Hospitals are required to alert their area OPO to every potential donor who is declared brain-dead or once the decision to withdraw life support is made. The OPOs by law can't participate in that decision and 'we are not even in the room at that time,' said Barry Massa of Kentucky's Network for Hope. During the following days of preparation, hospital employees continue caring for the patient – while the donation team talks with the family about the process, gathers hospital records showing the patient is eligible, requests tests of organ quality, and make arrangements with transplant centers to use them. Once the hospital withdraws life support and the heart stops beating there's a mandatory wait – five minutes – to be sure it won't restart. When the doctor declares death, the organ retrieval process can begin. Organs are only considered usable if death occurs relatively quickly, usually up to about two hours. Sometimes that takes much longer and thus the organs can't be used – and HRSA's Dr. Raymond Lynch told Congress that doesn't necessarily mean anything was done wrong. Still, he said HRSA is investigating reports of possible mistakes elsewhere. 'This is a technically demanding form of care' that requires 'good collaboration between the OPO and the hospital,' he said. What happens next At issue is how doctors are sure when it's time to withdraw life support from a dying patient — and the delicate balance of how OPOs interact with hospital staff in preparing for donation once death occurs. In May, HRSA quietly ordered the U.S. transplant network to oversee improvements at the Kentucky OPO and to develop new national policies making clear that anyone – family, hospital staff or organ donation staff – can call for a pause in donation preparations any time there are concerns about the patient's eligibility. Lynch said the government now wants more proactive collaboration from OPOs to give hospital staff 'a clear understanding' of when to at least temporarily halt and reevaluate a potential donor if their health status changes. Kentucky's Massa said his group only received HRSA's reports this week – but that after learning about last fall's allegations, it made some changes. Massa said every hospital doctor and nurse now gets a checklist on caring for potential donors and how to pause when concerns are raised — and anyone can anonymously report complaints. ___ The Associated Press Health and Science Department receives support from the Howard Hughes Medical Institute's Department of Science Education and the Robert Wood Johnson Foundation. The AP is solely responsible for all content.
