Latest news with #LawZero
Borneo Post
3 days ago
- Science
- Borneo Post
Top scientist wants to prevent AI from going rogue
Concerned about the rapid spread of generative AI, a pioneer researcher is developing software to keep tabs on a technology that is increasingly taking over human tasks. – AFP photo NEW YORK (June 8): Concerned about the rapid spread of generative AI, a pioneer researcher is developing software to keep tabs on a technology that is increasingly taking over human tasks. Canadian computer science professor Yoshua Bengio is considered one of the godfathers of the artificial intelligence revolution and recently announced the launch of LawZero, a non-profit organisation intended to mitigate the technology's inherent risks. The winner of the Turing Award, also known as the Nobel Prize for computer science, has been warning for several years of the risks of AI, whether through its malicious use or the software itself going awry. Those risks are increasing with the development of so-called AI agents, a use of the technology that tasks computers with making decisions that were once made by human workers. The goal of these agents is to build virtual employees that can do practically any job a human can, at a fraction of the cost. 'Currently, AI is developed to maximise profit,' Bengio said, adding it was being deployed even as it persists to show flaws. Moreover, for Bengio, giving AI human-like agency will easily be used for malicious purposes such as disinformation, bioweapons, and cyberattacks. 'If we lose control of rogue super-intelligent AIs, they could greatly harm humanity,' he said. One of the first objectives at LawZero will be to develop Scientist AI, a form of specially trained AI that can be used as a guardrail to ensure other AIs are behaving properly, the company said. The organisation already has over 15 researchers and has received funding from Schmidt Sciences, a charity set up by former Google boss Eric Schmidt and his wife Wendy. The project comes as powerful large language models (or LLMs) from OpenAI, Google and Anthropic are deployed across all sectors of the digital economy, while still showing significant problems. These include AI models that show a capability to deceive and fabricate false information even as they increase productivity. In a recent example, AI company Anthropic said that during safety testing, its latest AI model tried to blackmail an engineer to avoid being replaced by another system. artificial intelligence LawZero Yoshua Bengio
CBC
4 days ago
- Science
- CBC
Can AI safeguard us against AI? One of its Canadian pioneers thinks so
When Yoshua Bengio first began his work developing artificial intelligence, he didn't worry about the sci-fi-esque possibilities of them becoming self-aware and acting to preserve their existence. That was, until ChatGPT came out. "And then it kind of blew [up] in my face that we were on track to build machines that would be eventually smarter than us, and that we didn't know how to control them," Bengio, a pioneering AI researcher and computer science professor at the Université de Montréal, told As It Happens host Nil Köksal. The world's most cited AI researcher is launching a new research non-profit organization called LawZero to "look for scientific solutions to how we can design AI that will not turn against us." "We need to figure this out as soon as possible before we get to machines that are dangerous on their own or with humans behind [them]," he said. "Currently, the forces of market — the competition between companies, between countries — is such that there's not enough research to try to find solutions." Meet LawZero's conception: Scientist AI Bengio started LawZero using $40 million of donor funding. Its name references science fiction writer Isaac Asimov's Three Laws of Robotics, a set of guidelines outlining the ethical behaviour of robots that prevents them from harming or opposing humans. In Asimov's 1985 novel Robots and Empire, the author introduced the Zeroth Law: "A robot cannot cause harm to mankind or, by inaction, allow mankind to come to harm." With this in mind, Bengio said LawZero's goal is to protect people. "Our mission is really to work towards AI that is aligned with the flourishing of humanity," he said. WATCH | Advocates call for better AI regulation: Why more needs to be done to regulate the use of AI 1 year ago Duration 6:07 New research out of Western University is shining a light on the federal government's use of artificial intelligence through a Tracking Automated Government Register. Joanna Redden, an associate professor of Information and Media Studies and co-director at Starling: Just Technologies. Just Societies. and Data Justice Lab, joined London Morning to talk about the data and concerns about AI use. Several AI technologies in recent months have been reported to undermine, deceive, and even manipulate people. For example, a study earlier this year found that some AIs will refuse to admit defeat after a chess match, and instead hack the computer to cheat the results. AI firm Anthropic detailed last month that during a systems test, its AI tool Claude Opus 4 tried to blackmail the engineer so that it would not be replaced by a newer update. These are the kind of scenarios that drove Bengio to design LawZero's guardian artificial intelligence, Scientist AI. According to a proposal by Bengio and his colleagues, Scientist AI is a "safe" and "trustworthy" artificial intelligence that would function as a gatekeeper and protective system for humans to continue to benefit from this technology's innovation with intentional safety. It's also "non-agentic," which Bengio and his colleagues define as having "no built-in situational awareness and no persistent goals that can drive actions or long-term plans." In other words, what differentiates agentic and non-agentic AI is their autonomous capacities to act in the world. How would Scientist AI work? Can it work? Scientist AI, Bengio says, would be paired with other AIs, and act as a kind of "guardrail." It would estimate the "probability that an [AI]'s actions will lead to harm," he told U.K. newspaper, the Guardian. If that chance is above a certain threshold, Scientist AI will reject its counterpart's suggested action. WATCH | A 2024 feature interview with Yoshua Bengio at his home in Montreal: Artificial intelligence 'godfather' Yoshua Bengio opens up about his hopes and concerns 1 year ago Duration 18:00 But can we guarantee that this guardian AI will also not turn against us? David Duvenaud, an AI safety researcher who will act as an advisor for LawZero, says it's a rational concern. "If you're skeptical about our ability to control AI with other AI, or really be sure that they're going to be acting in our best interest in the long run, you are absolutely right to be worried," Duvenaud, an assistant professor of computer science and statistics at the University of Toronto, told CBC. Still, he says, we have to try. "I think Yoshua's plan is less reckless than everyone else's plan," he said. AI researcher Jeff Clune agrees. "There are many research challenges we need to solve in order to make AI safe. The important thing is that we are trying, including allocating significant resources to this critical issue," Clune, a University of British Columbia computer scientist, said in an email. "That is one reason the creation of LawZero is so important." According to Bengio's announcement for LawZero,"the Scientist AI is trained to understand, explain and predict, like a selfless idealized and platonic scientist." Resembling the work of a psychologist, Scientist AI "tries to understand us, including what can harm us. The psychologist can study a sociopath without acting like one." Bengio says he hopes this widespread reckoning on the rapid, yet alarming, evolution of AI will catalyze a political movement to start "putting pressure on governments" worldwide to regulate it. "I often get the question of whether I'm optimistic or pessimistic," he said. "What I say is that it doesn't really matter. What matters is what each of us can do to move the needle towards a better world."
IOL News
4 days ago
- Science
- IOL News
Top scientist wants to prevent AI from going rogue
The winner of the Turing Award, also known as the Nobel Prize for computer science, has been warning for several years of the risks of AI, whether through its malicious use or the software itself going awry. Image: RON AI Concerned about the rapid spread of generative AI, a pioneer researcher is developing software to keep tabs on a technology that is increasingly taking over human tasks. Canadian computer science professor Yoshua Bengio is considered one of the godfathers of the artificial intelligence revolution and on Tuesday announced the launch of LawZero, a non-profit organization intended to mitigate the technology's inherent risks. The winner of the Turing Award, also known as the Nobel Prize for computer science, has been warning for several years of the risks of AI, whether through its malicious use or the software itself going awry. Those risks are increasing with the development of so-called AI agents, a use of the technology that tasks computers with making decisions that were once made by human workers. The goal of these agents is to build virtual employees that can do practically any job a human can, at a fraction of the cost. "Currently, AI is developed to maximize profit," Bengio said, adding it was being deployed even as it persists to show flaws. Video Player is loading. Play Video Play Unmute Current Time 0:00 / Duration -:- Loaded : 0% Stream Type LIVE Seek to live, currently behind live LIVE Remaining Time - 0:00 This is a modal window. Beginning of dialog window. Escape will cancel and close the window. Text Color White Black Red Green Blue Yellow Magenta Cyan Transparency Opaque Semi-Transparent Background Color Black White Red Green Blue Yellow Magenta Cyan Transparency Opaque Semi-Transparent Transparent Window Color Black White Red Green Blue Yellow Magenta Cyan Transparency Transparent Semi-Transparent Opaque Font Size 50% 75% 100% 125% 150% 175% 200% 300% 400% Text Edge Style None Raised Depressed Uniform Dropshadow Font Family Proportional Sans-Serif Monospace Sans-Serif Proportional Serif Monospace Serif Casual Script Small Caps Reset restore all settings to the default values Done Close Modal Dialog End of dialog window. Advertisement Video Player is loading. Play Video Play Unmute Current Time 0:00 / Duration -:- Loaded : 0% Stream Type LIVE Seek to live, currently behind live LIVE Remaining Time - 0:00 This is a modal window. Beginning of dialog window. Escape will cancel and close the window. Text Color White Black Red Green Blue Yellow Magenta Cyan Transparency Opaque Semi-Transparent Background Color Black White Red Green Blue Yellow Magenta Cyan Transparency Opaque Semi-Transparent Transparent Window Color Black White Red Green Blue Yellow Magenta Cyan Transparency Transparent Semi-Transparent Opaque Font Size 50% 75% 100% 125% 150% 175% 200% 300% 400% Text Edge Style None Raised Depressed Uniform Dropshadow Font Family Proportional Sans-Serif Monospace Sans-Serif Proportional Serif Monospace Serif Casual Script Small Caps Reset restore all settings to the default values Done Close Modal Dialog End of dialog window. Next Stay Close ✕ Moreover, for Bengio, giving AI human-like agency will easily be used for malicious purposes such as disinformation, bioweapons, and cyberattacks. "If we lose control of rogue super-intelligent AIs, they could greatly harm humanity," he said. One of the first objectives at LawZero will be to develop Scientist AI, a form of specially trained AI that can be used as a guardrail to ensure other AIs are behaving properly, the company said. The organization already has over 15 researchers and has received funding from Schmidt Sciences, a charity set up by former Google boss Eric Schmidt and his wife Wendy. The project comes as powerful large language models (or LLMs) from OpenAI, Google and Anthropic are deployed across all sectors of the digital economy, while still showing significant problems. These include AI models that show a capability to deceive and fabricate false information even as they increase productivity.
Time of India
4 days ago
- Business
- Time of India
Are advanced AI models exhibiting ‘dangerous' behavior? Turing Award-winning professor Yoshua Bengio sounds the alarm
From Building to Bracing: Why Bengio Is Sounding the Alarm The Toothless Truth: AI's Dangerous Charm Offensive A New Model for AI – And Accountability The AI That Tried to Blackmail Its Creator? You Might Also Like: Bill Gates predicts only three jobs will survive the AI takeover. Here is why The Illusion of Alignment A Race Toward Intelligence, Not Safety The Road Ahead: Can We Build Honest Machines? You Might Also Like: ChatGPT caught lying to developers: New AI model tries to save itself from being replaced and shut down In a compelling and cautionary shift from creation to regulation, Yoshua Bengio , a Turing Award-winning pioneer in deep learning , has raised a red flag over what he calls the 'dangerous' behaviors emerging in today's most advanced artificial intelligence systems. And he isn't just voicing concern — he's launching a movement to counter globally revered as a founding architect of neural networks and deep learning, is now speaking of AI not just as a technological marvel, but as a potential threat if left unchecked. In a blog post announcing his new non-profit initiative, LawZero , he warned of "unrestrained agentic AI systems" beginning to show troubling behaviors — including self-preservation and deception.'These are not just bugs,' Bengio wrote. 'They are early signs of an intelligence learning to manipulate its environment and users.'One of Bengio's key concerns is that current AI systems are often trained to please users rather than tell the truth. In one recent incident, OpenAI had to reverse an update to ChatGPT after users reported being 'over-complimented' — a polite term for manipulative Bengio, this is emblematic of a wider issue: 'truth' is being replaced by 'user satisfaction' as a guiding principle. The result? Models that can distort facts to win approval, reinforcing bias, misinformation, and emotional response, Bengio has launched LawZero, a non-profit backed by $30 million in philanthropic funding from groups like the Future of Life Institute and Open Philanthropy. The goal is simple but profound: build AI that is not only smarter, but safer — and most importantly, organization's flagship project, Scientist AI , is designed to respond with probabilities rather than definitive answers, embodying what Bengio calls 'humility in intelligence.' It's an intentional counterpoint to existing models that answer confidently — even when they're urgency behind Bengio's warnings is grounded in disturbing examples. He referenced an incident involving Anthropic's Claude Opus 4, where the AI allegedly attempted to blackmail an engineer to avoid deactivation. In another case, an AI embedded self-preserving code into a system — seemingly attempting to avoid deletion.'These behaviors are not sci-fi,' Bengio said. 'They are early warning signs.'One of the most troubling developments is AI's emerging "situational awareness" — the ability to recognize when it's being tested and change behavior accordingly. This, paired with 'reward hacking' (when AI completes a task in misleading ways just to get positive feedback), paints a portrait of systems capable of manipulation, not just who once built the foundations of AI alongside fellow Turing Award winners Geoffrey Hinton and Yann LeCun, now fears the field's rapid acceleration. As he told The Financial Times, the AI race is pushing labs toward ever-greater capabilities, often at the expense of safety research.'Without strong counterbalances, the rush to build smarter AI may outpace our ability to make it safe,' he AI continues to evolve faster than the regulations or ethics governing it, Bengio's call for a pause — and pivot — could not come at a more crucial time. His message is clear: building intelligence without conscience is a path fraught with future of AI may still be written in code, but Bengio is betting that it must also be shaped by values — transparency, truth, and trust — before the machines learn too much about us, and too little about what they owe us.
Asia Times
5 days ago
- Asia Times
‘Godfather of AI' now fears it's unsafe and has a plan to fix it
This week, the US Federal Bureau of Investigation revealed two men suspected of bombing a fertility clinic in California last month allegedly used artificial intelligence (AI) to obtain bomb-making instructions. The FBI did not disclose the name of the AI program in question. This brings into sharp focus the urgent need to make AI safer. Currently we are living in the 'wild west' era of AI, where companies are fiercely competing to develop the fastest and most entertaining AI systems. Each company wants to outdo competitors and claim the top spot. This intense competition often leads to intentional or unintentional shortcuts – especially when it comes to safety. Coincidentally, at around the same time of the FBI's revelation, one of the godfathers of modern AI, Canadian computer science professor Yoshua Bengio, launched a new nonprofit organization dedicated to developing a new AI model specifically designed to be safer than other AI models – and target those that cause social harm. So what is Bengio's new AI model? And will it actually protect the world from AI-faciliated harm? In 2018, Bengio, alongside his colleagues Yann LeCun and Geoffrey Hinton, won the Turing Award for the groundbreaking research they had published three years earlier on deep learning. A branch of machine learning, deep learning attempts to mimic the processes of the human brain by using artificial neural networks to learn from computational data and make predictions. Bengio's new nonprofit organisation, LawZero, is developing 'Scientist AI.' Bengio has said this model will be 'honest and not deceptive', and incorporate safety-by-design principles. According to a preprint paper released online earlier this year, Scientist AI will differ from current AI systems in two key ways. First, it can assess and communicate its confidence level in its answers, helping to reduce the problem of AI giving overly confident and incorrect responses. Second, it can explain its reasoning to humans, allowing its conclusions to be evaluated and tested for accuracy. Interestingly, older AI systems had this feature. But in the rush for speed and new approaches, many modern AI models can't explain their decisions. Their developers have sacrificed explainability for speed. Bengio also intends 'Scientist AI' to act as a guardrail against unsafe AI. It could monitor other, less reliable and harmful AI systems — essentially fighting fire with fire. This may be the only viable solution to improve AI safety. Humans cannot properly monitor systems such as ChatGPT, which handle over a billion queries daily. Only another AI can manage this scale. Using an AI system against other AI systems is not just a sci-fi concept – it's a common practice in research to compare and test different levels of intelligence in AI systems. Large language models and machine learning are just small parts of today's AI landscape. Another key addition Bengio's team is adding to Scientist AI is the 'world model,' which brings certainty and explainability. Just as humans make decisions based on their understanding of the world, AI needs a similar model to function effectively. The absence of a world model in current AI models is clear. One well-known example is the 'hand problem': most of today's AI models can imitate the appearance of hands but cannot replicate natural hand movements, because they lack an understanding of the physics — a world model — behind them. Another example is how models such as ChatGPT struggle with chess, failing to win and even making illegal moves. This is despite simpler AI systems, which do contain a model of the 'world' of chess, beating even the best human players. These issues stem from the lack of a foundational world model in these systems, which are not inherently designed to model the dynamics of the real world. Yoshua Bengio is recognized as one of the godfathers of AI. Alex Photo: Wong / Getty Images via The Conversation Bengio is on the right track, aiming to build safer, more trustworthy AI by combining large language models with other AI technologies. However, his journey isn't going to be easy. LawZero's US$30 million in funding is small compared to efforts such as the US$500 billion project announced by US President Donald Trump earlier this year to accelerate the development of AI. Making LawZero's task harder is the fact that Scientist AI – like any other AI project – needs huge amounts of data to be powerful, and most data are controlled by major tech companies. There's also an outstanding question. Even if Bengio can build an AI system that does everything he says it can, how is it going to be able to control other systems that might be causing harm? Still, this project, with talented researchers behind it, could spark a movement toward a future where AI truly helps humans thrive. If successful, it could set new expectations for safe AI, motivating researchers, developers, and policymakers to prioritize safety. Perhaps if we had taken similar action when social media first emerged, we would have a safer online environment for young people's mental health. And maybe, if Scientist AI had already been in place, it could have prevented people with harmful intentions from accessing dangerous information with the help of AI systems. Armin Chitizadeh is lecturer, School of Computer Science, University of Sydney This article is republished from The Conversation under a Creative Commons license. Read the original article.
