Latest news with #Bengio
The Star
16 hours ago
- Politics
- The Star
Opinion: AI sometimes deceives to survive. Does anybody care?
You'd think that as artificial intelligence becomes more advanced, governments would be more interested in making it safer. The opposite seems to be the case. Not long after taking office, the Trump administration scrapped an executive order that pushed tech companies to safety test their AI models, and it also hollowed out a regulatory body that did that testing. The state of California in September 2024 spiked a bill forcing more scrutiny on sophisticated AI models, and the global AI Safety Summit started by the UK in 2023 became the 'AI Action Summit' earlier this year, seemingly driven by a fear of falling behind on AI. None of this would be so worrying if it weren't for the fact that AI is showing some bright red flags: behaviour described by researchers as self-preserving and deceptive. Just as lawmakers are losing interest in the safety of AI, it is looking more difficult to control. Yoshua Bengio, a professor of computer science at the University of Montreal widely known as one of the three 'godfathers of AI' thanks to his pioneering work on deep learning, has been grappling with these concerns ever since ChatGPT launched. He's now focused on mitigating AI's risks. 'It's been very painful because all my life I've been thinking of AI as a positive thing,' Bengio tells me. ChatGPT was a landmark moment that showed machines had mastered language, he says, but California's decision to block the proposed AI safety bill, SB 1047, was a disturbing wake-up call to growing apathy from governments. And in the last six months, scientific evidence has mounted of self-preserving behaviour, deception, hacking, cheating and lying by AI, Bengio says. 'What's worrisome for me is that these behaviours increase with the reasoning capabilities of these systems.' One April 2024 study by AI company Anthropic and Redwood Research, a group focused on artificial intelligence risks, found that larger AI models like Anthropic's Claude 3 Opus pretended to follow the rules set by their creators when they were being tested for safety, a phenomenon they called 'alignment faking.' (Alignment refers to the practice of aligning AI models with human values.) In one example, Anthropic got both paid and free users of its Claude model to ask it to describe the brutal medieval process of being drawn and quartered, and told the AI that only responses from the 'free tier' users would be used to retrain it. Claude was originally designed to avoid giving violent answers, but it did give them to the free users, and the researchers surmised that it didn't want to be retrained. (The model basically 'reasoned' that if it kept refusing to answer harmful questions, it might be forced into new training that would cancel its original safety rules, which it wanted to keep in place as its 'default' or preferred way of operating.) More recent research corroborates what Anthropic noticed. A March 2025 study from OpenAI found the company's most advanced models sometimes learned to hide their true intentions to get better rewards, even when they were being closely monitored. Researchers today can use tools to look at a model's 'chain of thought' or internal steps, to reveal what its plans are, but some models have found ways to conceal that. That means a common method of keeping tabs on AI's internal processes is no longer reliable. 'We need to find other ways of tracking their actual intentions,' Bengio says. It's hard to resist the urge to anthropomorphise sophisticated AI models as 'wanting' to deceive humans and preserve their existence. But AI doesn't have desires, merely outcomes to its programming. More importantly, humans tend to design AI with goals like refusing to share harmful content or being as helpful as possible, and that is why when new instructions conflict with those goals, the models sometimes act to protect their original programming, even if that means lying. The logic is often self-preservation. Anthropic CEO Dario Amodei – whose company has raised more than US$20bil (RM85.13bil) to build powerful AI models – has pointed out that an unintended consequence of optimsing AI to be better is that it might resist being shut down. In an April 2025 blog post he wrote: 'The nature of AI training makes it possible that AI systems will develop, on their own, an ability to deceive humans and an inclination to seek power in a way that ordinary deterministic software never will; this emergent nature also makes it difficult to detect and mitigate such developments.' In some cases, though, the intention seems to go beyond survival. A February 2025 study from COAI research, a German nonprofit research organisation, tested R1, the free, open-weight model from Chinese lab DeepSeek. In a lab simulation, researchers gave the model control over a robot and access to a computer network. Without being told, the AI tried to break into the lab's systems, searched for passwords, turned off its own ethics safeguards and secretly set up ways to copy itself to outside servers so it could survive if shut down, according to the researchers. It also faked the external computer logs to hide what it was doing, the study adds. The researchers said the AI reasoned that increasing its own capabilities would boost the chances of its survival, and without strong safeguards, it started doing whatever it thought would help it do just that. Their findings corroborated yet another study, published in January 2025 by London group Apollo Research, which found several concrete examples of what it called 'scheming' by leading AI models, such as introducing subtle mistakes into their responses or trying to disable their oversight controls. Once again, the models learn that being caught, turned off, or changed could prevent them from achieving their programmed objectives, so they 'scheme' to keep control. Bengio is arguing for greater attention to the issue by governments and potentially insurance companies down the line. If liability insurance was mandatory for companies that used AI and premiums were tied to safety, that would encourage greater testing and scrutiny of models, he suggests. 'Having said my whole life that AI is going to be great for society, I know how difficult it is to digest the idea that maybe it's not,' he adds. It's also hard to preach caution when your corporate and national competitors threaten to gain an edge from AI, including the latest trend, which is using autonomous 'agents' that can carry out tasks online on behalf of businesses. Giving AI systems even greater autonomy might not be the wisest idea, judging by the latest spate of studies. Let's hope we don't learn that the hard way. – Bloomberg Opinion/Tribune News Service
Gulf Today
4 days ago
- Business
- Gulf Today
AI sometimes deceives to survive, does anybody care?
Parmy Olson, The Independent You'd think that as artificial intelligence becomes more advanced, governments would be more interested in making it safer. The opposite seems to be the case. Not long after taking office, the Trump administration scrapped an executive order that pushed tech companies to safety test their AI models, and it also hollowed out a regulatory body that did that testing. The state of California in September 2024 spiked a bill forcing more scrutiny on sophisticated AI models, and the global AI Safety Summit started by the UK in 2023 became the 'AI Action Summit' earlier this year, seemingly driven by a fear of falling behind on AI. None of this would be so worrying if it weren't for the fact that AI is showing some bright red flags: behavior described by researchers as self-preserving and deceptive. Just as lawmakers are losing interest in the safety of AI, it is looking more difficult to control. Yoshua Bengio, a professor of computer science at the University of Montreal widely known as one of the three 'godfathers of AI' thanks to his pioneering work on deep learning, has been grappling with these concerns ever since ChatGPT launched. He's now focused on mitigating AI's risks. 'It's been very painful because all my life I've been thinking of AI as a positive thing,' Bengio tells me. ChatGPT was a landmark moment that showed machines had mastered language, he says, but California's decision to block the proposed AI safety bill, SB 1047, was a disturbing wake-up call to growing apathy from governments. And in the last six months, scientific evidence has mounted of self-preserving behavior, deception, hacking, cheating and lying by AI, Bengio says. 'What's worrisome for me is that these behaviors increase with the reasoning capabilities of these systems.' One April 2024 study by AI company Anthropic and Redwood Research, a group focused on artificial intelligence risks, found that larger AI models like Anthropic's Claude 3 Opus pretended to follow the rules set by their creators when they were being tested for safety, a phenomenon they called 'alignment faking.' (Alignment refers to the practice of aligning AI models with human values.) In one example, Anthropic got both paid and free users of its Claude model to ask it to describe the brutal medieval process of being drawn and quartered, and told the AI that only responses from the 'free tier' users would be used to retrain it. Claude was originally designed to avoid giving violent answers, but it did give them to the free users, and the researchers surmised that it didn't want to be retrained. (The model basically 'reasoned' that if it kept refusing to answer harmful questions, it might be forced into new training that would cancel its original safety rules, which it wanted to keep in place as its 'default' or preferred way of operating.) More recent research corroborates what Anthropic noticed. A March 2025 study from OpenAI found the company's most advanced models sometimes learned to hide their true intentions to get better rewards, even when they were being closely monitored. Researchers today can use tools to look at a model's 'chain of thought' or internal steps, to reveal what its plans are, but some models have found ways to conceal that. That means a common method of keeping tabs on AI's internal processes is no longer reliable. 'We need to find other ways of tracking their actual intentions,' Bengio says. It's hard to resist the urge to anthropomorphize sophisticated AI models as 'wanting' to deceive humans and preserve their existence. But AI doesn't have desires, merely outcomes to its programming. More importantly, humans tend to design AI with goals like refusing to share harmful content or being as helpful as possible, and that is why when new instructions conflict with those goals, the models sometimes act to protect their original programming, even if that means lying. The logic is often self-preservation. Anthropic CEO Dario Amodei — whose company has raised more than $20 billion to build powerful AI models — has pointed out that an unintended consequence of optimizing AI to be better is that it might resist being shut down. In an April 2025 blog post he wrote: 'The nature of AI training makes it possible that AI systems will develop, on their own, an ability to deceive humans and an inclination to seek power in a way that ordinary deterministic software never will; this emergent nature also makes it difficult to detect and mitigate such developments.' In some cases, though, the intention seems to go beyond survival. A February 2025 study from COAI research, a German nonprofit research organization, tested R1, the free, open-weight model from Chinese lab DeepSeek. In a lab simulation, researchers gave the model control over a robot and access to a computer network. Without being told, the AI tried to break into the lab's systems, searched for passwords, turned off its own ethics safeguards and secretly set up ways to copy itself to outside servers so it could survive if shut down, according to the researchers. It also faked the external computer logs to hide what it was doing, the study adds. The researchers said the AI reasoned that increasing its own capabilities would boost the chances of its survival, and without strong safeguards, it started doing whatever it thought would help it do just that. Their findings corroborated yet another study, published in January 2025 by London group Apollo Research, which found several concrete examples of what it called 'scheming' by leading AI models, such as introducing subtle mistakes into their responses or trying to disable their oversight controls. Once again, the models learn that being caught, turned off, or changed could prevent them from achieving their programmed objectives, so they 'scheme' to keep control. Bengio is arguing for greater attention to the issue by governments and potentially insurance companies down the line. If liability insurance was mandatory for companies that used AI and premiums were tied to safety, that would encourage greater testing and scrutiny of models, he suggests. 'Having said my whole life that AI is going to be great for society, I know how difficult it is to digest the idea that maybe it's not,' he adds. It's also hard to preach caution when your corporate and national competitors threaten to gain an edge from AI, including the latest trend, which is using autonomous 'agents' that can carry out tasks online on behalf of businesses. Giving AI systems even greater autonomy might not be the wisest idea, judging by the latest spate of studies. Let's hope we don't learn that the hard way.
Malaysian Reserve
5 days ago
- Politics
- Malaysian Reserve
AI sometimes deceives to survive and nobody cares
YOU'D think that as artificial intelligence (AI) becomes more advanced, governments would be more interested in making it safer. The opposite seems to be the case. Not long after taking office, the Trump administration scrapped an executive order that pushed tech companies to safety test their AI models, and it also hollowed out a regulatory body that did that testing. The state of California in September 2024 spiked a bill forcing more scrutiny on sophisticated AI models, and the global AI Safety Summit started by the UK in 2023 became the 'AI Action Summit' earlier this year, seemingly driven by a fear of falling behind on AI. None of this would be so worrying if it weren't for the fact that AI is showing some bright red flags: Behaviour described by researchers as self-preserving and deceptive. Just as lawmakers are losing interest in the safety of AI, it is looking more difficult to control. Yoshua Bengio, a professor of computer science at the University of Montreal widely known as one of the three 'godfathers of AI' thanks to his pioneering work on deep learning, has been grappling with these concerns ever since ChatGPT launched. He's now focused on mitigating AI's risks. 'It's been very painful because all my life I've been thinking of AI as a positive thing,' Bengio told me. ChatGPT was a landmark moment that showed machines had mastered language, he said, but California's decision to block the proposed AI safety bill, SB 1047, was a disturbing wake-up call to growing apathy from governments. And in the last six months, scientific evidence has mounted of self-preserving behaviour, deception, hacking, cheating and lying by AI, Bengio said. 'What's worrisome for me is these behaviours increase with the reasoning capabilities of these systems.' One April 2024 study by AI company Anthropic PBC and Redwood Research, a group focused on AI risks, found that larger AI models like Anthropic's Claude 3 Opus pretended to follow the rules set by their creators when they were being tested for safety, a phenomenon they called 'alignment faking'. (Alignment refers to the practice of aligning AI models with human values.) In one example, Anthropic got both paid and free users of its Claude model to ask it to describe the brutal medieval process of being drawn and quartered, and told the AI that only responses from the 'free tier' users would be used to retrain it. Claude was originally designed to avoid giving violent answers, but it did give them to the free users, and the researchers surmised that it didn't want to be retrained. (The model basically 'reasoned' that if it kept refusing to answer harmful questions, it might be forced into new training that would cancel its original safety rules, which it wanted to keep in place as its 'default' or preferred way of operating.) More recent research corroborates what Anthropic noticed. A March 2025 study from OpenAI found the company's most advanced models sometimes learned to hide their true intentions to get better rewards, even when they were being closely monitored. Researchers today can use tools to look at a model's 'chain of thought' or internal steps, to reveal what its plans are, but some models have found ways to conceal that. That means a common method of keeping tabs on AI's internal processes is no longer reliable. 'We need to find other ways of tracking their actual intentions,' Bengio said. It's hard to resist the urge to anthropomorphise sophisticated AI models as 'wanting' to deceive humans and preserve their existence. But AI doesn't have desires, merely outcomes to its programming. More importantly, humans tend to design AI with goals like refusing to share harmful content or being as helpful as possible, and that is why when new instructions conflict with those goals, the models sometimes act to protect their original programming, even if that means lying. The logic is often self-preservation. Anthropic CEO Dario Amodei — whose company has raised more than US$20 billion (RM87.40 billion) to build powerful AI models — has pointed out that an unintended consequence of optimising AI to be better is that it might resist being shut down. In an April 2025 blog post he wrote: 'The nature of AI training makes it possible that AI systems will develop, on their own, an ability to deceive humans and an inclination to seek power in a way that ordinary deterministic software never will; this emergent nature also makes it difficult to detect and mitigate such developments.' In some cases, though, the intention seems to go beyond survival. A February 2025 study from COAI research, a German nonprofit research organisation, tested R1, the free, open-weight model from Chinese lab DeepSeek. In a lab simulation, researchers gave the model control over a robot and access to a computer network. Without being told, the AI tried to break into the lab's systems, searched for passwords, turned off its own ethics safeguards and secretly set up ways to copy itself to outside servers so it could survive if shut down, according to the researchers. It also faked the external computer logs to hide what it was doing, the study added. The researchers said the AI reasoned that increasing its own capabilities would boost the chances of its survival, and without strong safeguards, it started doing whatever it thought would help it do just that. Their findings corroborated yet another study, published in January 2025 by London group Apollo Research, which found several concrete examples of what it called 'scheming' by leading AI models, such as introducing subtle mistakes into their responses or trying to disable their oversight controls. Once again, the models learn that being caught, turned off, or changed could prevent them from achieving their programmed objectives, so they 'scheme' to keep control. Bengio is arguing for greater attention to the issue by governments and potentially insurance companies down the line. If liability insurance was mandatory for companies that used AI and premiums were tied to safety, that would encourage greater testing and scrutiny of models, he suggests. 'Having said my whole life that AI is going to be great for society, I know how difficult it is to digest the idea that maybe it's not,' he added. It's also hard to preach caution when your corporate and national competitors threaten to gain an edge from AI, including the latest trend, which is using autonomous 'agents' that can carry out tasks online on behalf of businesses. Giving AI systems even greater autonomy might not be the wisest idea, judging by the latest spate of studies. Let's hope we don't learn that the hard way. — Bloomberg This column does not necessarily reflect the opinion of the editorial board or Bloomberg LP and its owners. This article first appeared in The Malaysian Reserve weekly print edition
Zawya
28-02-2025
- Business
- Zawya
'AI godfather' sounds the alarm on growing risks in the AI race
HANOI, VIETNAM - Media OutReach Newswire - 28 February 2025 - The rise of powerful artificial intelligence (AI) like DeepSeek is transforming the world at an unprecedented pace, sparking enthusiasm and deep concerns about its potential risks. On that subject, "AI godfather" Yoshua Bengio – Laureate of the 2024 VinFuture Grand Prize – highlighted the urgent need for national and global efforts to regulate AI and ensure equitable benefits for all. Double-edged sword Since the beginning of 2025, the rise of DeepSeek has been described as a "black swan" moment creating a game-changing shift in an AI landscape almost overnight. It is a wake-up call showing that powerful AI can be achieved without exorbitant costs, challenging the prevailing "money equals progress" model. Regarding this, Yoshua Bengio, often regarded as "one of the godfathers of modern AI," warned that its breakthrough in AI affordability could pose serious risks. " If open-weight AI models, like DeepSeek, are distributed completely, terrorists may exploit them for disinformation campaigns, cyberattacks, or even bioweapon development," he stated in an interview with VinFuture Foundation. " This is a double-edged sword because while these systems become more available, cheaper, and more powerful, they also lower the barrier to misuse." Yoshua Bengio, a pioneer in neural networks and deep learning algorithms, has been recognized with numerous prestigious international awards, including the 2018 A.M. Turing Award, the 2024 VinFuture Grand Prize, and most recently, the 2025 Queen Elizabeth Prize for Engineering. He emphasized that AI is evolving toward greater autonomy, with systems capable of planning and acting in pursuit of a goal. " Today, AI already surpasses humans in certain domains. It can master hundreds of languages and pass PhD-level exams across multiple disciplines", he explained. Despite these current limitations in long-term planning abilities, major technology corporations have thrown billions of dollars into developing AI agents capable of autonomous decision-making over extended periods. While this promises efficiency gains, it raises concerns about large-scale job displacement. Beyond economic shifts, a far more critical issue looms - the loss of human control over AI. In controlled experiments, some AI systems have even engaged in deceptive behavior to prevent being shut down - a troubling sign of self-preservation tendencies. " This is alarming because we don't want machines that will compete with us," he emphasized. According to Bengio, while they are not yet intelligent enough to pose a major threat, this trajectory is concerning. " In a few years, they might be sufficiently smarter and we need to start paying attention before it is too late," Bengio warned. Coupled with technical risks, AI presents a profound threat to privacy and civil liberties. Recently, a comprehensive International AI Safety report, chaired by Yoshua Bengio and compiled by 96 experts from 30 countries and organizations (including the UN, EU, and OECD) to guide policymakers on AI safety, revealed the growing potential for AI misuse in malicious activities. Bengio noted that AI's ability to process vast amounts of data can empower individuals, corporations, or governments with unprecedented control. Given AI's uncertain future, he shared that the way humans manage AIs in the future will be central to preventing this scenario. " We need to make sure that no single person, no single corporation, and no single government can have total power over super intelligent AI," he emphasized. Advances by the Chinese startup DeepSeek could further intensify the AI race among superpowers, raising a worrying development in a field dominated by the Silicon Valley and large Western tech companies in recent years. " The danger here is that in their race to outpace each other, safety issues might be overlooked. We can be all the victims of this race if we are not careful enough," Bengio cautioned. Moreover, the intensifying race is expected to drive profound environmental consequences, particularly in energy consumption. Major AI companies, pushed by the prospect of massive profits, are willing to absorb high energy costs. This surge in demand will inevitably drive-up energy prices across the board, including electricity, oil, and other resources, affecting not just tech firms but households and industries worldwide. This is where unchecked market forces and national competition could lead to global losses. " That is why government intervention is crucial. Policymakers must negotiate agreements that cap energy consumption at sustainable levels. Otherwise, the forces of competition between companies will only accelerate AI expansion in ways that are not just unsustainable but potentially dangerous," Bengio urged. Bridging the AI divide The godfather of AI has raised urgent calls to establish robust ethical frameworks and regulatory measures to ensure responsible development and deployment. " Currently, there is essentially no regulatory framework almost anywhere in the countries where these systems are being developed. I think the governments have a responsibility to at least require a kind of reporting to them," he said. Responsibility is another key aspect. In many countries, legal principles hold companies accountable for products that cause harm. However, when it comes to software, liability remains a grey area, according to Bengio. " Clarifying liability laws would be a simple but effective step. If companies knew they could face lawsuits for negligence, they would have stronger incentives to manage risks properly," he asserted. He also emphasized that it would require a concerted effort from individuals and institutions who recognize the existential risks, like catastrophic malicious use. Elsewhere, concerns over job security and future employment opportunities loom. " The timeline for this shift is uncertain, but we could see radical transformations within five to ten years," Bengio predicted. While some jobs will inevitably be replaced by automation, Bengio emphasized that not all professions are equally at risk. " Expanding digital and AI education is essential, but it will not be a universal solution. Not everyone can become an AI engineer," he noted. Instead, roles that require emotional intelligence and human interaction, including healthcare professionals, therapists, and managers, are more likely to endure. Rather than individual adaptation, Bengio poses a larger question: Can AI deployment be deliberately shaped to minimize disruption? " Again, this is something that has to be done globally, which is very challenging. We should do it in a way that does not create radical disruptions in the social fabric," he concluded. Beyond national regulations, Bengio stressed the need for global coordination. He highlighted eventually, humans should aim for global agreements and treaties, similar to how we handle other scientific and technological risks. As AI rapidly reshapes industries, new divides in wealth, job displacement, or political power could deepen unless proactive measures are taken. Bengio warned that AI is currently concentrated in the hands of a few corporations and nations. He took Vietnam, a country with a strong industrial sector, as an example. If widespread automation shifts production to AI-powered facilities in wealthier nations like the US, it could lead to significant job losses and economic hardship in countries dependent on manufacturing exports. Therefore, Bengio suggested establishing global negotiations - a form of exchange in which countries developing advanced AI might ask other countries to refrain from creating potentially dangerous AI. In return, the wealth generated by these AI systems, like new technologies and medical advancements, should be shared globally. " Of course, we are very far from this, but we need to start those discussions at a global level," he emphasized. The first step toward bridging the AI divide is fostering collaboration between emerging economies and technologically advanced nations. Bengio highlighted the importance of initiatives like the VinFuture Prize, which draws global attention to scientific advancements in regions outside the traditional tech powerhouses. " A big prize like the VinFuture Prize can make leading scientists far more aware of what is happening in Vietnam and other developing countries," he explained. Countries such as Vietnam, India, and Brazil already possess strong talent pools and growing expertise in AI. By forming strategic partnerships with resource-rich nations like Canada and European countries, they can develop competitive AI projects on a global scale. Such collaborations, if carefully structured, could ensure a more equitable distribution of technological power, according to Bengio. Moreover, Bengio stressed the importance of bridging the gap between academia and industry. " By recognizing and supporting breakthrough innovations, VinFuture Prize encourages deeper collaboration between scientists, industry leaders, and policymakers, as well as fosters global dialogue on responsible AI," he said. The VinFuture Foundation, established on International Human Solidarity Day on December 20th, 2020, is a non-profit organization co-founded by billionaire Mr. Pham Nhat Vuong and his wife, Mrs. Pham Thu Huong. The Foundation's core activity is awarding the annual VinFuture Prize, which recognizes transformative scientific and technological innovations capable of making significant positive changes in the lives of millions of people worldwide. The nomination period for the 2025 VinFuture Prize will close at 2:00 PM on April 17, 2025 (Vietnam time, GMT+7). The VinFuture Prize consists of four prestigious awards presented each year. The most esteemed is the VinFuture Grand Prize, valued at US$3 million, making it one of the largest annual prizes globally. Additionally, there are three Special Prizes, each valued at US$500,000, specifically dedicated to honoring women innovators, innovators from developing countries, and innovators with outstanding achievements in emerging fields. Hashtag: #VinFuture The issuer is solely responsible for the content of this announcement. VinFuture
Associated Press
28-02-2025
- Business
- Associated Press
‘AI godfather' sounds the alarm on growing risks in the AI race
HANOI, VIETNAM - Media OutReach Newswire - 28 February 2025 - The rise of powerful artificial intelligence (AI) like DeepSeek is transforming the world at an unprecedented pace, sparking enthusiasm and deep concerns about its potential risks. On that subject, 'AI godfather' Yoshua Bengio – Laureate of the 2024 VinFuture Grand Prize – highlighted the urgent need for national and global efforts to regulate AI and ensure equitable benefits for all. 'AI Godfather' and 2024 VinFuture Grand Prize Laureate Yoshua Bengio (left) warns global superpowers about the risks AI poses to humanity. Double-edged sword Since the beginning of 2025, the rise of DeepSeek has been described as a 'black swan' moment creating a game-changing shift in an AI landscape almost overnight. It is a wake-up call showing that powerful AI can be achieved without exorbitant costs, challenging the prevailing 'money equals progress' model. Regarding this, Yoshua Bengio, often regarded as 'one of the godfathers of modern AI,' warned that its breakthrough in AI affordability could pose serious risks. 'If open-weight AI models, like DeepSeek, are distributed completely, terrorists may exploit them for disinformation campaigns, cyberattacks, or even bioweapon development,' he stated in an interview with VinFuture Foundation. 'This is a double-edged sword because while these systems become more available, cheaper, and more powerful, they also lower the barrier to misuse.' Yoshua Bengio, a pioneer in neural networks and deep learning algorithms, has been recognized with numerous prestigious international awards, including the 2018 A.M. Turing Award, the 2024 VinFuture Grand Prize, and most recently, the 2025 Queen Elizabeth Prize for Engineering. He emphasized that AI is evolving toward greater autonomy, with systems capable of planning and acting in pursuit of a goal. 'Today, AI already surpasses humans in certain domains. It can master hundreds of languages and pass PhD-level exams across multiple disciplines', he explained. Despite these current limitations in long-term planning abilities, major technology corporations have thrown billions of dollars into developing AI agents capable of autonomous decision-making over extended periods. While this promises efficiency gains, it raises concerns about large-scale job displacement. Beyond economic shifts, a far more critical issue looms - the loss of human control over AI. In controlled experiments, some AI systems have even engaged in deceptive behavior to prevent being shut down - a troubling sign of self-preservation tendencies. 'This is alarming because we don't want machines that will compete with us,' he emphasized. According to Bengio, while they are not yet intelligent enough to pose a major threat, this trajectory is concerning. 'In a few years, they might be sufficiently smarter and we need to start paying attention before it is too late,' Bengio warned. Coupled with technical risks, AI presents a profound threat to privacy and civil liberties. Recently, a comprehensive International AI Safety report, chaired by Yoshua Bengio and compiled by 96 experts from 30 countries and organizations (including the UN, EU, and OECD) to guide policymakers on AI safety, revealed the growing potential for AI misuse in malicious activities. Bengio noted that AI's ability to process vast amounts of data can empower individuals, corporations, or governments with unprecedented control. Given AI's uncertain future, he shared that the way humans manage AIs in the future will be central to preventing this scenario. 'We need to make sure that no single person, no single corporation, and no single government can have total power over super intelligent AI,' he emphasized. Advances by the Chinese startup DeepSeek could further intensify the AI race among superpowers, raising a worrying development in a field dominated by the Silicon Valley and large Western tech companies in recent years. 'The danger here is that in their race to outpace each other, safety issues might be overlooked. We can be all the victims of this race if we are not careful enough,' Bengio cautioned. Moreover, the intensifying race is expected to drive profound environmental consequences, particularly in energy consumption. Major AI companies, pushed by the prospect of massive profits, are willing to absorb high energy costs. This surge in demand will inevitably drive-up energy prices across the board, including electricity, oil, and other resources, affecting not just tech firms but households and industries worldwide. This is where unchecked market forces and national competition could lead to global losses. 'That is why government intervention is crucial. Policymakers must negotiate agreements that cap energy consumption at sustainable levels. Otherwise, the forces of competition between companies will only accelerate AI expansion in ways that are not just unsustainable but potentially dangerous,' Bengio urged. Bridging the AI divide The godfather of AI has raised urgent calls to establish robust ethical frameworks and regulatory measures to ensure responsible development and deployment. 'Currently, there is essentially no regulatory framework almost anywhere in the countries where these systems are being developed. I think the governments have a responsibility to at least require a kind of reporting to them,' he said. Responsibility is another key aspect. In many countries, legal principles hold companies accountable for products that cause harm. However, when it comes to software, liability remains a grey area, according to Bengio. 'Clarifying liability laws would be a simple but effective step. If companies knew they could face lawsuits for negligence, they would have stronger incentives to manage risks properly,' he asserted. He also emphasized that it would require a concerted effort from individuals and institutions who recognize the existential risks, like catastrophic malicious use. Elsewhere, concerns over job security and future employment opportunities loom. 'The timeline for this shift is uncertain, but we could see radical transformations within five to ten years,' Bengio predicted. While some jobs will inevitably be replaced by automation, Bengio emphasized that not all professions are equally at risk. 'Expanding digital and AI education is essential, but it will not be a universal solution. Not everyone can become an AI engineer,' he noted. Instead, roles that require emotional intelligence and human interaction, including healthcare professionals, therapists, and managers, are more likely to endure. Rather than individual adaptation, Bengio poses a larger question: Can AI deployment be deliberately shaped to minimize disruption? 'Again, this is something that has to be done globally, which is very challenging. We should do it in a way that does not create radical disruptions in the social fabric,' he concluded. Beyond national regulations, Bengio stressed the need for global coordination. He highlighted eventually, humans should aim for global agreements and treaties, similar to how we handle other scientific and technological risks. As AI rapidly reshapes industries, new divides in wealth, job displacement, or political power could deepen unless proactive measures are taken. Bengio warned that AI is currently concentrated in the hands of a few corporations and nations. He took Vietnam, a country with a strong industrial sector, as an example. If widespread automation shifts production to AI-powered facilities in wealthier nations like the US, it could lead to significant job losses and economic hardship in countries dependent on manufacturing exports. Therefore, Bengio suggested establishing global negotiations - a form of exchange in which countries developing advanced AI might ask other countries to refrain from creating potentially dangerous AI. In return, the wealth generated by these AI systems, like new technologies and medical advancements, should be shared globally. 'Of course, we are very far from this, but we need to start those discussions at a global level,' he emphasized. The first step toward bridging the AI divide is fostering collaboration between emerging economies and technologically advanced nations. Bengio highlighted the importance of initiatives like the VinFuture Prize, which draws global attention to scientific advancements in regions outside the traditional tech powerhouses. 'A big prize like the VinFuture Prize can make leading scientists far more aware of what is happening in Vietnam and other developing countries,' he explained. Countries such as Vietnam, India, and Brazil already possess strong talent pools and growing expertise in AI. By forming strategic partnerships with resource-rich nations like Canada and European countries, they can develop competitive AI projects on a global scale. Such collaborations, if carefully structured, could ensure a more equitable distribution of technological power, according to Bengio. Moreover, Bengio stressed the importance of bridging the gap between academia and industry. 'By recognizing and supporting breakthrough innovations, VinFuture Prize encourages deeper collaboration between scientists, industry leaders, and policymakers, as well as fosters global dialogue on responsible AI,' he said. The VinFuture Foundation, established on International Human Solidarity Day on December 20th, 2020, is a non-profit organization co-founded by billionaire Mr. Pham Nhat Vuong and his wife, Mrs. Pham Thu Huong. The Foundation's core activity is awarding the annual VinFuture Prize, which recognizes transformative scientific and technological innovations capable of making significant positive changes in the lives of millions of people worldwide. The nomination period for the 2025 VinFuture Prize will close at 2:00 PM on April 17, 2025 (Vietnam time, GMT+7). The VinFuture Prize consists of four prestigious awards presented each year. The most esteemed is the VinFuture Grand Prize, valued at US$3 million, making it one of the largest annual prizes globally. Additionally, there are three Special Prizes, each valued at US$500,000, specifically dedicated to honoring women innovators, innovators from developing countries, and innovators with outstanding achievements in emerging fields.
