Latest news with #TuringAward
Borneo Post
18 hours ago
- Science
- Borneo Post
Top scientist wants to prevent AI from going rogue
Concerned about the rapid spread of generative AI, a pioneer researcher is developing software to keep tabs on a technology that is increasingly taking over human tasks. – AFP photo NEW YORK (June 8): Concerned about the rapid spread of generative AI, a pioneer researcher is developing software to keep tabs on a technology that is increasingly taking over human tasks. Canadian computer science professor Yoshua Bengio is considered one of the godfathers of the artificial intelligence revolution and recently announced the launch of LawZero, a non-profit organisation intended to mitigate the technology's inherent risks. The winner of the Turing Award, also known as the Nobel Prize for computer science, has been warning for several years of the risks of AI, whether through its malicious use or the software itself going awry. Those risks are increasing with the development of so-called AI agents, a use of the technology that tasks computers with making decisions that were once made by human workers. The goal of these agents is to build virtual employees that can do practically any job a human can, at a fraction of the cost. 'Currently, AI is developed to maximise profit,' Bengio said, adding it was being deployed even as it persists to show flaws. Moreover, for Bengio, giving AI human-like agency will easily be used for malicious purposes such as disinformation, bioweapons, and cyberattacks. 'If we lose control of rogue super-intelligent AIs, they could greatly harm humanity,' he said. One of the first objectives at LawZero will be to develop Scientist AI, a form of specially trained AI that can be used as a guardrail to ensure other AIs are behaving properly, the company said. The organisation already has over 15 researchers and has received funding from Schmidt Sciences, a charity set up by former Google boss Eric Schmidt and his wife Wendy. The project comes as powerful large language models (or LLMs) from OpenAI, Google and Anthropic are deployed across all sectors of the digital economy, while still showing significant problems. These include AI models that show a capability to deceive and fabricate false information even as they increase productivity. In a recent example, AI company Anthropic said that during safety testing, its latest AI model tried to blackmail an engineer to avoid being replaced by another system. artificial intelligence LawZero Yoshua Bengio
Yahoo
a day ago
- Yahoo
Godfather of AI Alarmed as Advanced Systems Quickly Learning to Lie, Deceive, Blackmail and Hack
A key artificial intelligence pioneer is concerned by the technology's growing propensity to lie and deceive — and he's founding his own nonprofit to curb such behavior. In a blog post announcing LawZero, the new nonprofit venture, "AI godfather" Yoshua Bengio said that he has grown "deeply concerned" as AI models become ever more powerful and deceptive. "This organization has been created in response to evidence that today's frontier AI models have growing dangerous capabilities and [behaviors]," the world's most-cited computer scientist wrote, "including deception, cheating, lying, hacking, self-preservation, and more generally, goal misalignment." Of all people, Bengio would know. In 2018, the founder of the Montreal Institute for Learning Algorithms (MILA) was presented with a Turing Award alongside fellow AI pioneers Yann LeCun and Geoffrey Hinton for their formative roles in machine learning research, and he was listed as one of Time magazine's "100 Most Influential People" in 2024 thanks to his outsize impact on the ever-accelerating technology. Despite the accolades, Bengio has repeatedly expressed regret over his role in bringing advanced AI technology — and its Silicon Valley hype cycle — to fruition. This latest missive seems to be his most stark to date. "I'm deeply concerned," the AI pioneer wrote in his blog post, "by the behaviors that unrestrained agentic AI systems are already beginning to exhibit." Bengio pointed to recent red-teaming experiments, or tests that push AI models to their limits to see how they'll act, showing that advanced systems have developed an uncanny tendency to keep themselves "alive" by any means necessary. Among his examples was a recent report from Anthropic detailing how its Claude 4 model, when told it would be shut down, threatened to blackmail an engineer with incriminating emails if they followed through. "These incidents," the decorated researcher wrote, "are early warning signs of the kinds of unintended and potentially dangerous strategies AI may pursue if left unchecked." To put such behavior in check, Bengio said that his new nonprofit is building a so-called "trustworthy" model, which he calls "Scientist AI," that is "trained to understand, explain and predict, like a selfless idealized and platonic scientist." "Instead of an actor trained to imitate or please people (including sociopaths), imagine an AI that is trained like a psychologist — more generally a scientist — who tries to understand us, including what can harm us," he explained. "The psychologist can study a sociopath without acting like one." A pre-peer-review paper Bengio and his colleagues published earlier this year explains it a bit more simply. "This system is designed to explain the world from observations," the paper reads, "as opposed to taking actions in it to imitate or please humans." The concept of building "safe" AI is far from new, of course — it's quite literally why several OpenAI researchers left OpenAI and founded Anthropic as a rival research lab. This one seems to be different because, unlike Anthropic, OpenAI, or any other companies that pay lip service to AI safety while still bringing in gobs of cash, Bengio's is a nonprofit — though that hasn't stopped him from raising $30 million from the likes of ex-Google CEO Eric Schmidt, among others. More on creepy AI: Advanced OpenAI Model Caught Sabotaging Code Intended to Shut It Down
IOL News
2 days ago
- Science
- IOL News
Top scientist wants to prevent AI from going rogue
The winner of the Turing Award, also known as the Nobel Prize for computer science, has been warning for several years of the risks of AI, whether through its malicious use or the software itself going awry. Image: RON AI Concerned about the rapid spread of generative AI, a pioneer researcher is developing software to keep tabs on a technology that is increasingly taking over human tasks. Canadian computer science professor Yoshua Bengio is considered one of the godfathers of the artificial intelligence revolution and on Tuesday announced the launch of LawZero, a non-profit organization intended to mitigate the technology's inherent risks. The winner of the Turing Award, also known as the Nobel Prize for computer science, has been warning for several years of the risks of AI, whether through its malicious use or the software itself going awry. Those risks are increasing with the development of so-called AI agents, a use of the technology that tasks computers with making decisions that were once made by human workers. The goal of these agents is to build virtual employees that can do practically any job a human can, at a fraction of the cost. "Currently, AI is developed to maximize profit," Bengio said, adding it was being deployed even as it persists to show flaws. Video Player is loading. Play Video Play Unmute Current Time 0:00 / Duration -:- Loaded : 0% Stream Type LIVE Seek to live, currently behind live LIVE Remaining Time - 0:00 This is a modal window. Beginning of dialog window. Escape will cancel and close the window. Text Color White Black Red Green Blue Yellow Magenta Cyan Transparency Opaque Semi-Transparent Background Color Black White Red Green Blue Yellow Magenta Cyan Transparency Opaque Semi-Transparent Transparent Window Color Black White Red Green Blue Yellow Magenta Cyan Transparency Transparent Semi-Transparent Opaque Font Size 50% 75% 100% 125% 150% 175% 200% 300% 400% Text Edge Style None Raised Depressed Uniform Dropshadow Font Family Proportional Sans-Serif Monospace Sans-Serif Proportional Serif Monospace Serif Casual Script Small Caps Reset restore all settings to the default values Done Close Modal Dialog End of dialog window. Advertisement Video Player is loading. Play Video Play Unmute Current Time 0:00 / Duration -:- Loaded : 0% Stream Type LIVE Seek to live, currently behind live LIVE Remaining Time - 0:00 This is a modal window. Beginning of dialog window. Escape will cancel and close the window. Text Color White Black Red Green Blue Yellow Magenta Cyan Transparency Opaque Semi-Transparent Background Color Black White Red Green Blue Yellow Magenta Cyan Transparency Opaque Semi-Transparent Transparent Window Color Black White Red Green Blue Yellow Magenta Cyan Transparency Transparent Semi-Transparent Opaque Font Size 50% 75% 100% 125% 150% 175% 200% 300% 400% Text Edge Style None Raised Depressed Uniform Dropshadow Font Family Proportional Sans-Serif Monospace Sans-Serif Proportional Serif Monospace Serif Casual Script Small Caps Reset restore all settings to the default values Done Close Modal Dialog End of dialog window. Next Stay Close ✕ Moreover, for Bengio, giving AI human-like agency will easily be used for malicious purposes such as disinformation, bioweapons, and cyberattacks. "If we lose control of rogue super-intelligent AIs, they could greatly harm humanity," he said. One of the first objectives at LawZero will be to develop Scientist AI, a form of specially trained AI that can be used as a guardrail to ensure other AIs are behaving properly, the company said. The organization already has over 15 researchers and has received funding from Schmidt Sciences, a charity set up by former Google boss Eric Schmidt and his wife Wendy. The project comes as powerful large language models (or LLMs) from OpenAI, Google and Anthropic are deployed across all sectors of the digital economy, while still showing significant problems. These include AI models that show a capability to deceive and fabricate false information even as they increase productivity.
Asia Times
3 days ago
- Asia Times
‘Godfather of AI' now fears it's unsafe and has a plan to fix it
This week, the US Federal Bureau of Investigation revealed two men suspected of bombing a fertility clinic in California last month allegedly used artificial intelligence (AI) to obtain bomb-making instructions. The FBI did not disclose the name of the AI program in question. This brings into sharp focus the urgent need to make AI safer. Currently we are living in the 'wild west' era of AI, where companies are fiercely competing to develop the fastest and most entertaining AI systems. Each company wants to outdo competitors and claim the top spot. This intense competition often leads to intentional or unintentional shortcuts – especially when it comes to safety. Coincidentally, at around the same time of the FBI's revelation, one of the godfathers of modern AI, Canadian computer science professor Yoshua Bengio, launched a new nonprofit organization dedicated to developing a new AI model specifically designed to be safer than other AI models – and target those that cause social harm. So what is Bengio's new AI model? And will it actually protect the world from AI-faciliated harm? In 2018, Bengio, alongside his colleagues Yann LeCun and Geoffrey Hinton, won the Turing Award for the groundbreaking research they had published three years earlier on deep learning. A branch of machine learning, deep learning attempts to mimic the processes of the human brain by using artificial neural networks to learn from computational data and make predictions. Bengio's new nonprofit organisation, LawZero, is developing 'Scientist AI.' Bengio has said this model will be 'honest and not deceptive', and incorporate safety-by-design principles. According to a preprint paper released online earlier this year, Scientist AI will differ from current AI systems in two key ways. First, it can assess and communicate its confidence level in its answers, helping to reduce the problem of AI giving overly confident and incorrect responses. Second, it can explain its reasoning to humans, allowing its conclusions to be evaluated and tested for accuracy. Interestingly, older AI systems had this feature. But in the rush for speed and new approaches, many modern AI models can't explain their decisions. Their developers have sacrificed explainability for speed. Bengio also intends 'Scientist AI' to act as a guardrail against unsafe AI. It could monitor other, less reliable and harmful AI systems — essentially fighting fire with fire. This may be the only viable solution to improve AI safety. Humans cannot properly monitor systems such as ChatGPT, which handle over a billion queries daily. Only another AI can manage this scale. Using an AI system against other AI systems is not just a sci-fi concept – it's a common practice in research to compare and test different levels of intelligence in AI systems. Large language models and machine learning are just small parts of today's AI landscape. Another key addition Bengio's team is adding to Scientist AI is the 'world model,' which brings certainty and explainability. Just as humans make decisions based on their understanding of the world, AI needs a similar model to function effectively. The absence of a world model in current AI models is clear. One well-known example is the 'hand problem': most of today's AI models can imitate the appearance of hands but cannot replicate natural hand movements, because they lack an understanding of the physics — a world model — behind them. Another example is how models such as ChatGPT struggle with chess, failing to win and even making illegal moves. This is despite simpler AI systems, which do contain a model of the 'world' of chess, beating even the best human players. These issues stem from the lack of a foundational world model in these systems, which are not inherently designed to model the dynamics of the real world. Yoshua Bengio is recognized as one of the godfathers of AI. Alex Photo: Wong / Getty Images via The Conversation Bengio is on the right track, aiming to build safer, more trustworthy AI by combining large language models with other AI technologies. However, his journey isn't going to be easy. LawZero's US$30 million in funding is small compared to efforts such as the US$500 billion project announced by US President Donald Trump earlier this year to accelerate the development of AI. Making LawZero's task harder is the fact that Scientist AI – like any other AI project – needs huge amounts of data to be powerful, and most data are controlled by major tech companies. There's also an outstanding question. Even if Bengio can build an AI system that does everything he says it can, how is it going to be able to control other systems that might be causing harm? Still, this project, with talented researchers behind it, could spark a movement toward a future where AI truly helps humans thrive. If successful, it could set new expectations for safe AI, motivating researchers, developers, and policymakers to prioritize safety. Perhaps if we had taken similar action when social media first emerged, we would have a safer online environment for young people's mental health. And maybe, if Scientist AI had already been in place, it could have prevented people with harmful intentions from accessing dangerous information with the help of AI systems. Armin Chitizadeh is lecturer, School of Computer Science, University of Sydney This article is republished from The Conversation under a Creative Commons license. Read the original article.
India Today
4 days ago
- Politics
- India Today
AI godfather Yoshua Bengio launches non-profit for honest AI, warns current models are lying to you
'Today's AI agents are trained to please and imitate—not always to tell the truth,' says Yoshua Bengio, one of the world's most respected AI researchers, who is also one of the three AI godfathers. Bengio says so as he launched a new non-profit called LawZero with a big mission: stop rogue AI before it does real harm. With $30 million in initial funding and a team of expert researchers, Bengio wants to build something called 'Scientist AI' – a tool that acts like a psychologist for want to build AIs that will be honest and not deceptive,' Bengio said. Unlike today's AI agents, which he describes as 'actors' trying to imitate humans and please users, Scientist AI will work more like a neutral observer. Its job is to predict when another AI might act in a harmful or dishonest way – and flag or stop it.'It has a sense of humility,' Bengio said of his new model. Instead of pretending to know everything, it will give probabilities, not firm answers. 'It isn't sure about the answer,' he goal? Create a kind of safety net that can monitor powerful AI agents before they go off track. These agents are increasingly being used to complete tasks without human supervision, raising fears about what could happen if one starts making dangerous decisions or tries to avoid being shut Scientist AI would assess how likely it is that an AI's actions could cause harm. If the risk is too high, it could block that action an ambitious plan but Bengio knows it has to scale. 'The point is to demonstrate the methodology so that then we can convince either donors or governments or AI labs to put the resources that are needed to train this at the same scale as the current frontier AIs,' he said. 'It is really important that the guardrail AI be at least as smart as the AI agent that it is trying to monitor and control.'Bengio's efforts are backed by major names in AI safety, including the Future of Life Institute, Skype co-founder Jaan Tallinn, and Schmidt Sciences, a research group set up by former Google CEO Eric initiative comes at a time when concerns about AI safety are rising — even among those who helped build Geoffrey Hinton – another AI godfather and Bengio's co-winner of the 2018 Turing Award – for instance. Hinton has spent the last few years warning the public about AI's risks. He's talked about machines that could spread misinformation, manipulate people, or become too smart for us to in a recent interview with CBS, Hinton made a surprising confession: he trusts AI more than he probably should. He uses OpenAI's GPT-4 model every day and admitted, 'I tend to believe what it says, even though I should probably be suspicious.'That said, Hinton, who left Google in 2023 to speak more freely about AI dangers, remains deeply concerned about where the technology is heading. He's warned that AI systems could become persuasive enough to influence public opinion or destabilise society. Still, his recent comments show the dilemma many experts face: they're impressed by AI's power, but worried by its then there's Yann LeCun, the third godfather of AI and Meta's top AI scientist. Unlike Bengio or Hinton, LeCun isn't too worried. In fact, he thinks people are an interview with the Wall Street Journal, last year, LeCun had said that today's AI systems don't even come close to human intelligence – or animal intelligence, for that matter. 'It's complete BS,' he said about the doomsday talk around AI. 'It seems to me that before 'urgently figuring out how to control AI systems much smarter than us' we need to have the beginning of a hint of a design for a system smarter than a house cat,' he played a major role in shaping today's AI, especially in image and speech recognition. At Meta, his teams continue to build powerful tools that help run everything from automatic translation to content moderation. He believes AI is still just a useful tool – not something to different approaches highlight an important truth: when it comes to AI, even the experts don't agree. But if Bengio's project takes off, we might soon have systems smart enough – and honest enough – to keep each other in check.
