Anthropic's Claude AI now has the ability to end 'distressing' conversations
To clarify, Anthropic said those two Claude models could exit harmful conversations, like "requests from users for sexual content involving minors and attempts to solicit information that would enable large-scale violence or acts of terror." With Claude Opus 4 and 4.1, these models will only end a conversation "as a last resort when multiple attempts at redirection have failed and hope of a productive interaction has been exhausted," according to Anthropic. However, Anthropic claims most users won't experience Claude cutting a conversation short, even when talking about highly controversial topics, since this feature will be reserved for "extreme edge cases." Anthropic's example of Claude ending a conversation (Anthropic)
In the scenarios where Claude ends a chat, users can no longer send any new messages in that conversation, but can start a new one immediately. Anthropic added that if a conversation is ended, it won't affect other chats and users can even go back and edit or retry previous messages to steer towards a different conversational route.
For Anthropic, this move is part of its research program that studies the idea of AI welfare. While the idea of anthropomorphizing AI models remains an ongoing debate, the company said the ability to exit a "potentially distressing interaction" was a low-cost way to manage risks for AI welfare. Anthropic is still experimenting with this feature and encourages its users to provide feedback when they encounter such a scenario.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Yahoo
an hour ago
- Yahoo
Why Paradigm built a spreadsheet with an AI agent in every cell
Anna Monaco has been building AI agents since before the term 'AI agents' was even a thing. After building numerous chatbots, she started looking for other types of interfaces that made sense for AI agents and landed on spreadsheets. 'I had this personal pattern, and I noticed that a lot of other people had this pattern, of putting very important CRM data in spreadsheets just because it was the most flexible thing,' Monaco told TechCrunch. 'But it was actually a pain to maintain. There's so much manual work involved. So [I] just went down this rabbit hole of building a product for myself and wanted to reimagine what a spreadsheet could look like with the full power of LLMs.' The result was Paradigm, an AI-powered spreadsheet equipped with more than 5,000 AI agents. Users can assign different prompts to individual columns and cells, and individual AI agents will crawl the internet to find and fill out the needed information. Paradigm works with AI models from Anthropic, OpenAI, and Google's Gemini, Monaco said, and supports model switching. 'We want to support every single model because we want our users to be able to have the highest reasoning outputs when they need it, but also the cheapest outputs,' Monaco said. 'It's just a constant cycle of evaluating different models, working closely with model providers to make sure our limits are high enough, and then giving some of that power to our users.' The company launched a small closed beta preview in late 2024 and has been iterating on the product using customer feedback. Paradigm attracts users ranging from consultants to sales professionals and finance folks and operates on a subscription model with tiers based on usage. Paradigm counts the consulting firm EY, AI chip startup Etched and the AI coding company Cognition as early customers. Paradigm is now releasing its product to the public and announcing that it raised a $5 million seed round led by General Catalyst. The company has raised $7 million to date. Monaco said the funding will go toward executing on the company's 'extremely aggressive product roadmap.' 'The interesting thing that happened when we fundraised is some people we pitched just kept on using and paying for the product,' Monaco said. 'I think that was a cool part of it. We found a lot of value from it internally and our investors, not even just our investors — other investors that we talked to — are still using it.' Paradigm isn't the only company looking to give spreadsheets an AI upgrade. Quadratic, which has raised more than $6 million in venture funding, is a three-year-old startup with a similar goal. Legacy companies like Google and Microsoft are also adding AI tools to their spreadsheet applications. Monaco said that she doesn't really consider the competition because Paradigm doesn't think of itself as an AI-powered spreadsheet. She said she thinks of it as a new AI-powered workflow that happens to be in the familiar form of a spreadsheet but won't necessarily stay that way forever. 'What I'm seeing in the most popular AI products now is this fine balance between present and future,' Monaco said. 'How do you build something that is really powerful and generates a lot of value now but also sets you up really well for the future? That's the question that I asked myself a year ago when I was starting the company.' 登入存取你的投資組合
The Verge
3 hours ago
- The Verge
Claude AI will end ‘persistently harmful or abusive user interactions'
Anthropic's Claude AI chatbot can now end conversations deemed 'persistently harmful or abusive,' as spotted earlier by TechCrunch. The capability is now available in Opus 4 and 4.1 models, and will allow the chatbot to end conversations as a 'last resort' after users repeatedly ask it to generate harmful content despite multiple refusals and attempts at redirection. The goal is to help the 'potential welfare' of AI models, Anthropic says, by terminating types of interactions in which Claude has shown 'apparent distress.' If Claude chooses to cut a conversation short, users won't be able to send new messages in that conversation. They can still create new chats, as well as edit and retry previous messages if they want to continue a particular thread. During its testing of Claude Opus 4, Anthropic says it found that Claude had a 'robust and consistent aversion to harm,' including when asked to generate sexual content involving minors, or provide information that could contribute to violent acts and terrorism. In these cases, Anthropic says Claude showed a 'pattern of apparent distress' and a 'tendency to end harmful conversations when given the ability.' Anthropic notes that conversations triggering this kind of response are 'extreme edge cases,' adding that most users won't encounter this roadblock even when chatting about controversial topics. The AI startup has also instructed Claude not to end conversations if a user is showing signs that they might want to hurt themselves or cause 'imminent harm' to others. Anthropic partners with Throughline, an online crisis support provider, to help develop responses to prompts related to self-harm and mental health. Last week, Anthropic also updated Claude's usage policy as rapidly advancing AI models raise more concerns about safety. Now, the company prohibits people from using Claude to develop biological, nuclear, chemical, or radiological weapons, as well as to develop malicious code or exploit a network's vulnerabilities. Posts from this author will be added to your daily email digest and your homepage feed. See All by Emma Roth Posts from this topic will be added to your daily email digest and your homepage feed. See All AI Posts from this topic will be added to your daily email digest and your homepage feed. See All News
Yahoo
5 hours ago
- Yahoo
The Epilepsy Foundation of America Launches AI Assistant to Transform Support for Epilepsy Community, Powered by Amazon Web Services
The new AI-Assistant, Sage, will build on the Foundation's ability to connect with the 3.4 million people living with epilepsy in the United States, increasing access to support and information BOWIE, Md., Aug. 18, 2025 /PRNewswire/ -- The Epilepsy Foundation of America (EFA) has announced the launch of its revolutionary AI Assistant, Sage. Developed in partnership with Amazon Web Services (AWS) and powered by Anthropic's Claude, this AI Assistant is one of the most empathetic AI systems ever created. Sage was made as a result of a transformative five-year digital journey and is now available for all to use on Sage is a purpose-built, intelligent, and emotionally aware companion trained on the lived experiences of the epilepsy community and infused with authoritative medical knowledge from a digital archive of over 25,000 curated pages of comprehensive epilepsy information and resources dating back to 2002. It is engineered to answer questions, understand context, track conversations across time, recognize emotional nuance, and deliver clear, accurate, and compassionate guidance instantly. Sage can also answer questions in multiple languages, including English, Spanish, French, Portuguese, and German. "This new technology is an extension of the Foundation's mission to ensure that no one faces epilepsy alone," said David Jost, Chief Technology Innovations Officer, Epilepsy Foundation of America. "Sage will be available at any time, in any place, to any member of the community. It learns from the context of previous inquiries, adapts to ongoing needs, and provides continuity of care in a digital environment." Whether you are navigating a new diagnosis, seeking guidance for a child, learning more about epilepsy treatments, or looking for ways to support a loved one, Sage is here and focused on your well-being. "With the help of Amazon Web Services, we built both a tool and companion," said Bernice Martin Lee, Chief Executive Officer, Epilepsy Foundation of America. "A tool that embodies the experience of our community, the brilliance of our clinical experts, and the power of AI. This assistant represents our promise that no one should ever have to face epilepsy alone." Built on Amazon Bedrock, Sage does what search engines cannot. It supersedes open-ended online queries, bypassing the noise and confusion to deliver the most relevant, accurate, and trustworthy information in seconds. Its responses are grounded in clinical expertise, community experience, and a deep understanding of what it means to live with epilepsy. "We are honored to support the Epilepsy Foundation of America in delivering a solution that brings reliable, accessible, and compassionate support to individuals and families affected by epilepsy," said Jeff Kratz, Vice President of Nonprofits and Worldwide Public Sector Industries at Amazon Web Services. "It has been deeply rewarding to work alongside such a mission-driven organization, and we are excited to see this powerful technology provide real value to the epilepsy community. This launch reflects the best of what is possible when purpose-led innovation meets cutting-edge cloud capabilities." This new tool does not replace the Epilepsy Foundation's ongoing human interaction. The Epilepsy Helpline, staffed by EFA employees, will continue to be available from 8 a.m. to 8 p.m. ET starting on August 25, 2025 for additional questions and connection. Sage is now ready on to support anyone. The Foundation encourages you to explore this new resource. We want to meet you where you are on your journey. We invite you to experience how this technology can simplify answers to your questions whenever you need it. About EpilepsyEpilepsy is the most common chronic brain disorder in the world, affecting people of all ages, races, and backgrounds. According to the U.S. Centers for Disease Control and Prevention (CDC), nearly 3.4 million Americans live with active epilepsy. The condition is defined by a tendency to experience seizures—sudden, abnormal bursts of electrical activity in the brain that can disrupt behavior, awareness, and bodily control. Beyond seizures, epilepsy can also lead to challenges such as fatigue, memory difficulties, mood changes, anxiety, and trouble concentrating, all of which can significantly impact daily life. Visit to learn more. About the Epilepsy Foundation of AmericaThe Epilepsy Foundation America is a national nonprofit organization dedicated to improving the lives of people affected by epilepsy through education, advocacy, research, and connection. For more than 50 years, the Foundation has led efforts to raise awareness, reduce stigma, and advocate for policies that make a meaningful difference for individuals living with epilepsy. We also fund cutting-edge research and support early-career investigators working to discover new treatments and therapies. Through our nationwide network of local offices and community partners, the Foundation expands access to care, provides seizure recognition and first aid training, and delivers direct services to those in need. To learn more visit or call 1.800.332.1000; in Spanish or call 866.748.8008. About Amazon Web ServicesSince 2006, Amazon Web Services has been the world's most comprehensive and broadly adopted cloud. AWS has been continually expanding its services to support virtually any workload, and it now has more than 240 fully featured services for compute, storage, databases, networking, analytics, machine learning and artificial intelligence (AI), Internet of Things (IoT), mobile, security, hybrid, media, and application development, deployment, and management from 117 Availability Zones within 37 geographic regions, with announced plans for 13 more Availability Zones and four more AWS Regions in Chile, New Zealand, the Kingdom of Saudi Arabia, and the AWS European Sovereign Cloud. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—trust AWS to power their infrastructure, become more agile, and lower costs. To learn more about AWS, visit View original content to download multimedia: SOURCE Epilepsy Foundation Sign in to access your portfolio
