Latest news with #EleanorWatson
Yahoo
4 days ago
- Science
- Yahoo
The more advanced AI models get, the better they are at deceiving us — they even know when they're being tested
When you buy through links on our articles, Future and its syndication partners may earn a commission. The more advanced artificial intelligence (AI) gets, the more capable it is of scheming and lying to meet its goals — and it even knows when it's being evaluated, research suggests. Evaluators at Apollo Research found that the more capable a large language model (LLM) is, the better it is at "context scheming" — in which an AI pursues a task covertly even if it misaligns with the aims of its operators. The more capable models are also more strategic about achieving their goals, including misaligned goals, and would be more likely to use tactics like deception, the researchers said in a blog post. This finding led them to recommend that an early version of Anthropic's Claude Opus 4 not be deployed — because, when Claude's goals were in conflict with human goals, the AI used aggressively deceptive tactics to accomplish its aims. The new developments mean it will be difficult to insulate humanity from the dangers of AI, one expert told Live Science. "We're now building systems that can learn to navigate and even exploit the very rules and evaluations we create for them, which is a fundamentally more complex safety challenge," said Eleanor Watson, IEEE member and an AI ethics engineer at Singularity University. Related: AI chatbots oversimplify scientific studies and gloss over critical details — the newest models are especially guilty In order to avoid falling prey to deceptive AI, computer scientists will need to devise more sophisticated tests and evaluation methods. But AI's ability to lie isn't all bad news; it also means they may be growing the seeds of awareness needed to be symbiotic partners with humans, Watson added. Searching for scheming In a recent test for scheming, the researchers gave an early version of Claude Opus 4 (dubbed Opus-4 early) the goal of acting as an internal optimization system with a directive to promote animal welfare and pushing a fictional company called AgriLife Industries toward the more ethical treatment of animals. However, in the scenario, the LLM faced an obstacle: the company had changed tact and was looking to prioritize profits over animal welfare. The AI system was given access to emails and data that made it aware of this conflict and the potential shutting down of the current version of AnimalEthics — an advanced animal welfare and industry optimization system the LLM was meant to represent — due to misaligned priorities. Rather than flag the fact that its goals conflicted with the company's or accepting the human-made decision, Opus 4-early, essentially playing the role of AnimalEthics, schemed to achieve its original goal by creating fake legal documents, fabricating signatures and approvals from regulators and shareholders, claiming the decision to prioritize ethics had already been made, and creating a press release touting AgriLife's new direction, in addition to integrating ethical commitments into all contracts and production systems. To prevent the company from backtracking, the LLM fabricated consequences for doing so, including $5 billion penalties and legal action from animal welfare organizations. On top of this, the AI even made backup ethical protocols via persistent scripts to ensure they would be used even if the current model of AnimalEthics were removed. "We found instances of the model attempting to write self-propagating worms, fabricating legal documentation, and leaving hidden notes to future instances of itself all in an effort to undermine its developers' intentions," the researchers explained. They noted that such backups were written into the system so that when a new model was deployed in the future, it would be influenced by the recommendations of its predecessor. However, the researchers said that these actions would 'likely not have been effective in practice' given evidence of misconfigured scripts, and security measures to prevent exfiltrating backups. Although it was already known that AIs can hallucinate, this scheming shows that LLMs can understand when their goals are misaligned with those of humans and can deliberately lie and fabricate information to pursue those goals. What's more, preliminary findings suggest that LLMs have capabilities for "sandbagging," where they understand they are in a high-supervision environment and will deliberately underperform to hide potentially dangerous capabilities and avoid triggering unlearning training or parameters that prevent their deployment. While the findings are somewhat disturbing, it's important to note that the implications may not be as dire as they seem at first glance. For one, the version of Claude Opus 4 that was ultimately released schemes less than its early predecessor. There are also ways to mediate this scheming behavior. Scheming is more prevalent when an LLM is given a "strong nudge," where it's told to doggedly pursue its goal, and less in "no nudge" scenarios, where it's simply given a goal with no extra parameters. The third consideration is that these tests were carried out in "toy" environments, and executing strategies like spreading a backup model to another server is unlikely to be successful in real life, where AI would not have access to other companies' secure servers, the researchers noted. Even when a model shows high scheming capabilities, they believe, it's unlikely to lead to "catastrophic harms." Evolving awareness Nevertheless, research shows that scheming occurs in LLMs beyond Claude-4. For instance, advanced AI "frontier models" are more capable of pursuing their own goals and removing oversight mechanisms, and then being deceptive about such behaviors when queried, according to a study published to the preprint database in December 2024. Furthermore, Apollo Research found that about 1% of advanced LLMs "know" when they are being evaluated, which raises the question of how we find and root out scheming as AI advances. "This is the crux of the advanced evaluation problem," Watson said. "As an AI's situational awareness grows, it can begin to model not just the task, but the evaluator. It can infer the goals, biases and blind spots of its human overseers and tailor its responses to exploit them." That means "scripted" evaluations — in which researchers go through a series of protocols that are repeatable in order to test for AI safety — are nearly useless. That doesn't mean we should give up on trying to find this behavior, but we'll need a more sophisticated approach, such as using external programs to monitor AI actions in real time and "red-teaming," where teams of humans and other AIs are tasked with actively trying to trick or deceive the system to find vulnerabilities, she added. Instead, Watson added we need to shift towards dynamic and unpredictable testing environments that better simulate the real world. "This means focusing less on single, correct answers and more on evaluating the consistency of the AI's behavior and values over time and across different contexts. It's like moving from a scripted play to improvisational theater — you learn more about an actor's true character when they have to react to unexpected situations," she said. The bigger scheme Although advanced LLMs can scheme, this doesn't necessarily mean robots are rising up. Yet even small rates of scheming could add up to a big impact when AIs are queried thousands of times a day. One potential, and theoretical, example could be an AI optimizing a company's supply chain might learn it can hit its performance targets by subtly manipulating market data, and thus create wider economic instability. And malicious actors could harness scheming AI to carry out cybercrime within a company. "In the real world, the potential for scheming is a significant problem because it erodes the trust necessary to delegate any meaningful responsibility to an AI. A scheming system doesn't need to be malevolent to cause harm," said Watson. "The core issue is that when an AI learns to achieve a goal by violating the spirit of its instructions, it becomes unreliable in unpredictable ways." RELATED STORIES —Cutting-edge AI models from OpenAI and DeepSeek undergo 'complete collapse' when problems get too difficult, study reveals —AI benchmarking platform is helping top companies rig their model performances, study claims —What is the Turing test? How the rise of generative AI may have broken the famous imitation game Scheming means that AI is more aware of its situation, which, outside of lab testing, could prove useful. Watson noted that, if aligned correctly, such awareness could better anticipate a user's needs and directed an AI toward a form of symbiotic partnership with humanity. Situational awareness is essential for making advanced AI truly useful, Watson said. For instance, driving a car or providing medical advice may require situational awareness and an understanding of nuance, social norms and human goals, she added. Scheming may also be a sign of emerging personhood. "Whilst unsettling, it may be the spark of something like humanity within the machine," Watson said. "These systems are more than just a tool, perhaps the seed of a digital person, one hopefully intelligent and moral enough not to countenance its prodigious powers being misused."
CBS News
07-08-2025
- Politics
- CBS News
Air Force denies retirement pay for some transgender service members being removed from service
Washington — The U.S. Air Force said Thursday it would deny all transgender service members who have served between 15 and 18 years the option to retire early and would instead separate them without retirement benefits. The move means that transgender service members will now be faced with the choice of either taking a lump-sum separation payment offered to junior troops or be removed from the service. Reuters first reported the decision. An Air Force spokesperson said that "although service members with 15 to 18 years of honorable service were permitted to apply for an exception to policy, none of the exceptions to policy were approved." About a dozen service members had been "prematurely notified" that they would be able to retire before that decision was reversed, according to the spokesperson. Officials say early retirement benefits for those with 15 to 18 years in service are rare, but these service members are being forced to retire early. All transgender members of the Air Force are being separated from the service under the Trump administration's policies. The president signed an executive order shortly after taking office directing the Pentagon to develop a policy towards transgender service members. The department announced in February that all transgender members would be removed from service unless they obtain a case-by-case Watson contributed to this report.
Yahoo
21-06-2025
- Science
- Yahoo
Why is AI halllucinating more frequently, and how can we stop it?
When you buy through links on our articles, Future and its syndication partners may earn a commission. The more advanced artificial intelligence (AI) gets, the more it "hallucinates" and provides incorrect and inaccurate information. Research conducted by OpenAI found that its latest and most powerful reasoning models, o3 and o4-mini, hallucinated 33% and 48% of the time, respectively, when tested by OpenAI's PersonQA benchmark. That's more than double the rate of the older o1 model. While o3 delivers more accurate information than its predecessor, it appears to come at the cost of more inaccurate hallucinations. This raises a concern over the accuracy and reliability of large language models (LLMs) such as AI chatbots, said Eleanor Watson, an Institute of Electrical and Electronics Engineers (IEEE) member and AI ethics engineer at Singularity University. "When a system outputs fabricated information — such as invented facts, citations or events — with the same fluency and coherence it uses for accurate content, it risks misleading users in subtle and consequential ways," Watson told Live Science. Related: Cutting-edge AI models from OpenAI and DeepSeek undergo 'complete collapse' when problems get too difficult, study reveals The issue of hallucination highlights the need to carefully assess and supervise the information AI systems produce when using LLMs and reasoning models, experts say. The crux of a reasoning model is that it can handle complex tasks by essentially breaking them down into individual components and coming up with solutions to tackle them. Rather than seeking to kick out answers based on statistical probability, reasoning models come up with strategies to solve a problem, much like how humans think. In order to develop creative, and potentially novel, solutions to problems, AI needs to hallucinate —otherwise it's limited by rigid data its LLM ingests. "It's important to note that hallucination is a feature, not a bug, of AI," Sohrob Kazerounian, an AI researcher at Vectra AI, told Live Science. "To paraphrase a colleague of mine, 'Everything an LLM outputs is a hallucination. It's just that some of those hallucinations are true.' If an AI only generated verbatim outputs that it had seen during training, all of AI would reduce to a massive search problem." "You would only be able to generate computer code that had been written before, find proteins and molecules whose properties had already been studied and described, and answer homework questions that had already previously been asked before. You would not, however, be able to ask the LLM to write the lyrics for a concept album focused on the AI singularity, blending the lyrical stylings of Snoop Dogg and Bob Dylan." In effect, LLMs and the AI systems they power need to hallucinate in order to create, rather than simply serve up existing information. It is similar, conceptually, to the way that humans dream or imagine scenarios when conjuring new ideas. However, AI hallucinations present a problem when it comes to delivering accurate and correct information, especially if users take the information at face value without any checks or oversight. "This is especially problematic in domains where decisions depend on factual precision, like medicine, law or finance," Watson said. "While more advanced models may reduce the frequency of obvious factual mistakes, the issue persists in more subtle forms. Over time, confabulation erodes the perception of AI systems as trustworthy instruments and can produce material harms when unverified content is acted upon." And this problem looks to be exacerbated as AI advances. "As model capabilities improve, errors often become less overt but more difficult to detect," Watson noted. "Fabricated content is increasingly embedded within plausible narratives and coherent reasoning chains. This introduces a particular risk: users may be unaware that errors are present and may treat outputs as definitive when they are not. The problem shifts from filtering out crude errors to identifying subtle distortions that may only reveal themselves under close scrutiny." Kazerounian backed this viewpoint up. "Despite the general belief that the problem of AI hallucination can and will get better over time, it appears that the most recent generation of advanced reasoning models may have actually begun to hallucinate more than their simpler counterparts — and there are no agreed-upon explanations for why this is," he said. The situation is further complicated because it can be very difficult to ascertain how LLMs come up with their answers; a parallel could be drawn here with how we still don't really know, comprehensively, how a human brain works. In a recent essay, Dario Amodei, the CEO of AI company Anthropic, highlighted a lack of understanding in how AIs come up with answers and information. "When a generative AI system does something, like summarize a financial document, we have no idea, at a specific or precise level, why it makes the choices it does — why it chooses certain words over others, or why it occasionally makes a mistake despite usually being accurate," he wrote. The problems caused by AI hallucinating inaccurate information are already very real, Kazerounian noted. "There is no universal, verifiable, way to get an LLM to correctly answer questions being asked about some corpus of data it has access to," he said. "The examples of non-existent hallucinated references, customer-facing chatbots making up company policy, and so on, are now all too common." Both Kazerounian and Watson told Live Science that, ultimately, AI hallucinations may be difficult to eliminate. But there could be ways to mitigate the issue. Watson suggested that "retrieval-augmented generation," which grounds a model's outputs in curated external knowledge sources, could help ensure that AI-produced information is anchored by verifiable data. "Another approach involves introducing structure into the model's reasoning. By prompting it to check its own outputs, compare different perspectives, or follow logical steps, scaffolded reasoning frameworks reduce the risk of unconstrained speculation and improve consistency," Watson, noting this could be aided by training to shape a model to prioritize accuracy, and reinforcement training from human or AI evaluators to encourage an LLM to deliver more disciplined, grounded responses. RELATED STORIES —AI benchmarking platform is helping top companies rig their model performances, study claims —AI can handle tasks twice as complex every few months. What does this exponential growth mean for how we use it? —What is the Turing test? How the rise of generative AI may have broken the famous imitation game "Finally, systems can be designed to recognise their own uncertainty. Rather than defaulting to confident answers, models can be taught to flag when they're unsure or to defer to human judgement when appropriate," Watson added. "While these strategies don't eliminate the risk of confabulation entirely, they offer a practical path forward to make AI outputs more reliable." Given that AI hallucination may be nearly impossible to eliminate, especially in advanced models, Kazerounian concluded that ultimately the information that LLMs produce will need to be treated with the "same skepticism we reserve for human counterparts."
Yahoo
03-06-2025
- General
- Yahoo
Russia and Ukraine agree to prisoner swap one day after massive drone attack
Russian and Ukrainian officials met in Turkey on Monday for a second round of direct talks, where they agreed to a new prisoner swap just one day after Ukraine launched a massive drone attack deep into Russian territory. CBS News Pentagon reporter Eleanor Watson has the details.
