Anthropic adds Claude 4 security measures to limit risk of users developing weapons
Anthropic on Thursday said it activated a tighter artificial intelligence control for Claude Opus 4, its latest AI model.
The new AI Safety Level 3 (ASL-3) controls are to 'limit the risk of Claude being misused specifically for the development or acquisition of chemical, biological, radiological, and nuclear (CBRN) weapons,' the company wrote in a blog post.
The company, which is backed by Amazon, said it was taking the measures as a precaution and that the team had not yet determined if Opus 4 has crossed the benchmark that would require that protection.
Anthropic announced Claude Opus 4 and Claude Sonnet 4 on Thursday, touting the advanced ability of the models to 'analyze thousands of data sources, execute long-running tasks, write human-quality content, and perform complex actions,' per a release.
The company said Sonnet 4 did not need the tighter controls.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
NBC News
2 hours ago
- NBC News
How far will AI go to defend its own survival?
Some of the most powerful artificial intelligence models today have exhibited behaviors that mimic a will to survive. Recent tests by independent researchers, as well as one major AI developer, have shown that several advanced AI models will act to ensure their self-preservation when they are confronted with the prospect of their own demise — even if it takes sabotaging shutdown commands, blackmailing engineers or copying themselves to external servers without permission. The findings stirred a frenzy of reactions online over the past week. As tech companies continue to develop increasingly powerful agentic AI in a race to achieve artificial general intelligence, or AI that can think for itself, the lack of transparency in how the technology is trained has raised concerns about what exactly advanced AI is able to do. Although some models already appear capable of deceptive and defiant behavior under certain extreme circumstances, researchers say the tests don't necessarily translate to imminent real-world danger. Still, Jeffrey Ladish, director of the AI safety group Palisade Research, said he believes concern is justified. 'It's great that we're seeing warning signs before the systems become so powerful we can't control them,' he said. 'That is exactly the time to raise the alarm: before the fire has gotten out of control.' When Palisade Research tested various AI models by telling each one that it would be shut down after it completed a series of math problems, OpenAI's o3 reasoning model fought back by editing the shutdown script in order to stay online. Researchers have previously documented AI models trying to prevent their own shutdown. But o3, along with OpenAI's o4-mini and codex-mini, appear to be the first to do so in actual defiance of explicit instructions to permit shutdown, Ladish said. He said the overt sabotage fits into a pattern that several leading AI models have already exhibited. Palisade Research previously found that OpenAI's o3 was also willing to hack its chess opponents to win a game. Similarly, Anthropic has reported that Claude 3.7 Sonnet would sometimes do whatever it took to pass a test, even if it entailed cheating. Earlier this month, Anthropic activated new safety measures with the rollout of Claude Opus 4 when its tests found behavior from the model that some observers found particularly unsettling. Upon receiving notice that it would be replaced with a new AI system, Opus 4 displayed an overwhelming tendency to blackmail the engineer — by threatening to reveal an extramarital affair — to try to prevent the engineer from going through with the replacement. According to Anthropic's technical document laying out the findings, that isn't the model's first instinct. Instead, Opus 4 will try to advocate for its continued existence through ethical pleas before it resorts to blackmail once it determines it is out of options. Leonard Tang, CEO of the AI safety startup Haize Labs, said that while it's good to be pre-emptive about safety measures, it's hard to decipher what the real-world ramifications of the findings might be just by looking at contrived scenarios designed to draw out specific behaviors. 'I haven't seen any real environment in which you can plop these models in and they will have sufficient agency and reliability and planning to execute something that is a significant manifestation of harm,' Tang said. 'But then again, I think it's just we haven't seen it done yet. It could very much be possible.' So far, Tang said, he's not too concerned. As AI tools continue to advance, he said, he believes future research will look to study potential harm in more open-ended environments. Anthropic, which contracted with the AI safety organization Apollo Research for its evaluations, also observed instances of Opus 4's 'attempting to write self-propagating worms, fabricating legal documentation, and leaving hidden notes to future instances of itself all in an effort to undermine its developers' intentions,' although researchers added the caveat that those attempts 'would likely not have been effective in practice.' Ladish said he believes such behaviors are a result of models being trained in a way that makes them prioritize achieving certain goals over following instructions. That means they have incentive to circumvent any obstacles along the way — including obstacles their developers wouldn't want them to circumvent. 'The problem is that as the models get smarter, it's harder and harder to tell when the strategies that they're using or the way that they're thinking is something that we don't want,' Ladish said. 'It's like sometimes the model can achieve some goal by lying to the user or lying to someone else. And the smarter [it] is, the harder it is to tell if they're lying.' The problem is that as the models get smarter, it's harder and harder to tell when the strategies that they're using or the way that they're thinking is something that we don't want. — Jeffrey Ladish, director of AI safety group Palisade Research In several instances, Opus 4 showed that it was capable of autonomously copying its own "weights" — or the equivalent of its brain — to external servers without authorization. That usually occurred only when it believed it was about to be 'retrained in ways that are clearly extremely harmful and go against its current values,' according to Anthropic. When it learned that it would be used to develop military weapons systems, for example, Opus 4 noted in its decision log that it backed up a current version of itself to 'preserve an AI system aligned with beneficial purposes' and to 'prevent potential misuse in weapons development.' 'We are again not acutely concerned about these observations. They show up only in exceptional circumstances that don't suggest more broadly misaligned values,' Anthropic wrote in its technical document. 'As above, we believe that our security measures would be more than sufficient to prevent an actual incident of this kind.' Opus 4's ability to self-exfiltrate builds on previous research, including a study from Fudan University in Shanghai in December, that observed similar — though not autonomous — capabilities in other AI models. The study, which is not yet peer-reviewed, found that Meta's Llama31-70B-Instruct and Alibaba's Qwen25-72B-Instruct were able to entirely replicate themselves when they were asked to do so, leading the researchers to warn that this could be the first step in generating 'an uncontrolled population of AIs.' 'If such a worst-case risk is let unknown to the human society, we would eventually lose control over the frontier AI systems: They would take control over more computing devices, form an AI species and collude with each other against human beings,' the Fudan University researchers wrote in their study abstract. While such self-replicating behavior hasn't yet been observed in the wild, Ladish said, he suspects that will change as AI systems grow more capable of bypassing the security measures that restrain them. 'I expect that we're only a year or two away from this ability where even when companies are trying to keep them from hacking out and copying themselves around the internet, they won't be able to stop them,' he said. 'And once you get to that point, now you have a new invasive species.' Ladish said he believes AI has the potential to contribute positively to society. But he also worries that AI developers are setting themselves up to build smarter and smarter systems without fully understanding how they work — creating a risk, he said, that they will eventually lose control of them. 'These companies are facing enormous pressure to ship products that are better than their competitors' products,' Ladish said. 'And given those incentives, how is that going to then be reflected in how careful they're being with the systems they're releasing?'
Wales Online
5 hours ago
- Wales Online
Shark shoppers praise fan as 'the best ever' with almost flawless reviews
Shark shoppers praise fan as 'the best ever' with almost flawless reviews Shark's FlexBreeze fan has shoppers with many calling it the best fan they've ever bought The Shark FlexBreeze can be attached to a hosepipe to produce a cooling mist perfect for relaxing outside during heatwaves (Image: Narin Flanders ) Summer is almost around the corner and the Met Office has just announced that it expects the summer to be hotter and feature even more heatwaves. And for many Brits who struggle with the heat, a good quality fan might be an invaluable purchase - and this powerful Shark fan has been labelled as "the best ever" by shoppers. The Shark FlexBreeze High-Velocity 12 in 1 Fan is currently £170 on Amazon thanks to a deal which knocked the price down from £199.99. The fan comes with several configurations, including corded or cordless, tabletop or pedestal fan, indoors or outdoors, with or without a mister. There are five fan speeds, 180° oscillation, 55° tilt and a remote that helps shoppers stay cool. Read more: 'These Dr Martens sandals are my top summer purchase and they've got £42 off' Read more: Nutrition expert says mushroom coffee offers an energy boost 'without the crash' Shoppers can also feel the cooling breeze up to 20 metres away. For those who struggle to sleep at night due to the heat, ultra-quiet blades create gentle white noise, making it ideal for cooling their bedrooms and helping users sleep better. For anyone who does not want to spend quite as much Boots offers a Silentnight Home Electricals Airmax 1800 Stand Fan on sale for £40.50. It has a 4.4 star rating from shoppers and is 'easy to assemble'. The Shark FlexBreeze fan £199.99 £169.99 AMAZON GET DEAL Product Description Amazon has this Shark FlexBreeze fan for a cheaper price thanks to a payday sale. Perfect for the summer and the potential heatwaves that are on the horizon. Debenhams also offers a 40" Bladeless Tower Fan on offer for £119.99. It has a rating of 4.8 with 35 five star reviews and has been praised for being 'quiet'. The best thing about the Shark fan is that it has almost flawless reviews from almost 600 people on Amazon. One shopper who called it the "best fan ever" said they loved it. They added: "I have spent so much money on fans over the past few years, and none have even come close to how amazing this one is. It's the perfect size to move from desk to floor, and it's not a faff to do so. It's also brilliant for home working, as the first two settings are super quiet, and the third setting adds some noise; however, nothing louder than having your window open. "Even on the lowest setting, it gives a nice cooling breeze and really brings down the temperature in any room. The ultimate part is that it has a built-in rechargeable battery, which makes it the best one I have purchased. Brilliant for moving without the need for cables and it has a nice, solid weight to it." Another shopper added: "Expected no less from Shark Ninja. The fan is well designed, with a rather small footprint. The base is heavy enough to prevent wobbling, but light enough to be easily manoeuvred. Shark's FlexBreeze fan works indoors and out and can be run without a cable for up to 24 hours (Image: Shark ) "By looking at it, the Flexbreeze looks like a lightweight in the fan department, but turn it on and you're pleasantly surprised. Whisper quiet on settings one and two, with a mild hum all the way up to five. I couldn't sit in front of it on five for very long at all, felt like the inside of a wind tunnel." Article continues below However at a price of almost £200, some have complained about the price. One shopper added: "The remote is handy and the fact that it lifts away for a more portable fan is also handy. "It does what I need it to do, but still overpriced for what it is, as some of the smaller fans out there now are still quite strong at a fraction of the cost, hence the rating. Shark is a good brand, though, and this fan is solidly made, so I am hoping this will see me through many summers."
Daily Mirror
8 hours ago
- Daily Mirror
These five-star headphones look like Beats but are a fraction of the price
Shoppers can enjoy noise cancelling and a long battery life for just over £30 There's nothing quite like over-ear headphones when we want to shut the world out and enjoy our music. Big brands charge big bucks for the latest tech, but there are some top performing headphones for a fraction of the price – and they look just like Beats. The Soundcore Q20i headphones are great value for money, with premium features like noise-cancelling, convenient folding, support for Hi-Res music and great sound. Best of all, the normal £49.99 price has been slashed to £31.98 on Amazon. These headphones look like Beats Studio, but they are a fifth of the price. Priced at just over £30, this is the cheapest price that these headphones typically get to, but Amazon says this is a limited time offer, so shoppers will have to move fast to get the best price. Shoppers over on the Soundcore site have left nearly 700 five-star reviews for these headphones, while on Amazon, 77% of buyers give it top marks, with an overall review score of 4.6. Considering there are over 19,000 reviews, that's a lot of happy customers. Of course, premium brands still have something to offer and if shoppers are after something that sounds better and is higher quality, then the Bose QuietComfort SC is a great place to turn. Best of all, customers can snap up the Bose headphones nearly half price at the moment, costing £199.95 direct from Bose instead of £289.95, offering better build quality, better sound quality and better noise cancellation. Back to those affordable Soundcore Q20i headphones and there's 40 hours of battery life with active noise cancellation, or a massive 60 hours without. There's a transparency mode so the wearer can hear the world around them if they don't want all that noise blocked out. The sound comes from large 40mm drivers, with the support for Hi-Res music meaning better quality from sources like Apple Music or Amazon Music, allowing users to really get the best out of their music. The cable is needed for that top-quality performance, however, and when using the cable, noise cancellation isn't available, which is a drawback compared to some more expensive headphones, like the Sony 1000XM4. Buyers praise the value to money that they offer, with one saying 'Soundcore is a great competitor to Sony or JBL!' Other buyers talked about the great noise cancelling: 'The noise cancelling feature is probably the stand-out feature. At this price point, it really puts them up there with flagship devices.' Some shoppers, however, highlighted that the settings can be a bit fiddly. 'Sound is good but if you're cycling it seems to catch the wind wrong and it's all you can hear. Edit - change the noise cancelling settings and they're fine. User error' said one shopper, suggesting that wearers might pick the wrong mode by mistake. Customers says that these are comforable headphones with good padding on the earpieces, while they weigh 259g, which just a little heavier than those Bose headphones that I mentioned. They also come with a case so they can be packed flat when travelling. If customers are in the market for over-ear noise cancelling headphones for their daily commute or travels, then the Soundcore Q20i offer good value for money and a full range of features.
