Top scientist wants to prevent AI from going rogue
Concerned about the rapid spread of generative AI, a pioneer researcher is developing software to keep tabs on a technology that is increasingly taking over human tasks.
Canadian computer science professor Yoshua Bengio is considered one of the godfathers of the artificial intelligence revolution and on Tuesday announced the launch of LawZero, a non-profit organization intended to mitigate the technology's inherent risks.
The winner of the Turing Award, also known as the Nobel Prize for computer science, has been warning for several years of the risks of AI, whether through its malicious use or the software itself going awry.
Those risks are increasing with the development of so-called AI agents, a use of the technology that tasks computers with making decisions that were once made by human workers.
The goal of these agents is to build virtual employees that can do practically any job a human can, at a fraction of the cost.
"Currently, AI is developed to maximize profit," Bengio said, adding it was being deployed even as it persists to show flaws.
Moreover, for Bengio, giving AI human-like agency will easily be used for malicious purposes such as disinformation, bioweapons, and cyberattacks.
"If we lose control of rogue super-intelligent AIs, they could greatly harm humanity," he said.
One of the first objectives at LawZero will be to develop Scientist AI, a form of specially trained AI that can be used as a guardrail to ensure other AIs are behaving properly, the company said.
The organization already has over 15 researchers and has received funding from Schmidt Sciences, a charity set up by former Google boss Eric Schmidt and his wife Wendy.
The project comes as powerful large language models (or LLMs) from OpenAI, Google and Anthropic are deployed across all sectors of the digital economy, while still showing significant problems.
These include AI models that show a capability to deceive and fabricate false information even as they increase productivity.
In a recent example, AI company Anthropic said that during safety testing, its latest AI model tried to blackmail an engineer to avoid being replaced by another system.
© 2025 AFP
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Japan Times
10 hours ago
- Japan Times
Alphabet CEO expects to keep hiring engineers while AI advances
Alphabet's Sundar Pichai said his company will keep expanding its engineering ranks at least into 2026, stressing human talent remains key even as Google's parent ramps up artificial intelligence investments. Speaking at the Bloomberg Tech conference in San Francisco, Pichai said he will continue to invest in engineering in the near future. U.S. tech leaders like Microsoft have trimmed more staff this year, reflecting in part the enormous investments needed to ensure leadership in AI. The firings have stoked fears about the technology replacing certain job functions. Google, itself, has conducted rounds of layoffs in recent years to free up resources.
Japan Today
2 days ago
- Japan Today
Canada, U.S. warn of air quality hazards as Canadian fire smoke reaches Europe
Huge plumes from Canada's wildfires sometimes reach very high altitude, where they pose less of an immediate health risk By Michel COMTE and Marion Thibaut Canada's wildfires, which have already forced evacuations of more than 26,000 people, continued their stubborn spread Tuesday, with heavy smoke choking millions of Canadians and Americans and reaching as far away as Europe. Alerts were issued for parts of Canada and the neighboring United States warning of hazardous air quality. A water tanker air base was consumed by flames in Saskatchewan province, oil production has been disrupted in Alberta, and officials warned of worse to come with more communities threatened each day. "We have some challenging days ahead of us," Saskatchewan Premier Scott Moe told a news conference, adding that the number of evacuees could rise quickly. Every summer, Canada grapples with forest fires, but an early start to the wildfire season this year and the scale of the blazes -- over two million hectares (494,000 acres) burned -- is worrying. The provinces of Saskatchewan and Manitoba have been hardest hit. Both declared wildfire emergencies in recent days. "This has been a very difficult time for many Canadians," federal Emergency Management Minister Eleanor Olszewski told reporters in Ottawa. "This wildfire season has started off more quickly, and it's stronger, more intense," she said, adding that the Canadian military has deployed aircraft to evacuate remote towns in Manitoba and was ready to also assist Saskatchewan and Alberta with firefighting. Climate change has increased the impact of extreme weather events in Canada, which is still recovering from the apocalyptic summer of 2023 when 15 million hectares of forests were scorched. As of Tuesday, there were 208 active fires across Canada. Half of them were listed as out of control, according to the Canadian Interagency Forest Fire Centre. Many of the affected populations are Indigenous, and some small communities have burned to the ground. Heavy smoke from the fires, meanwhile, has engulfed part of the continent, forcing residents of four Canadian provinces and the U.S. states of Michigan, Minnesota, Nebraska and Wisconsin to limit outdoor activities. "Smoke is causing very poor air quality and reduced visibility," Environment Canada said in a statement. Wildfire smoke is comprised of gaseous pollutants such as carbon monoxide, along with water vapor and particle pollution, which can be particularly hazardous to health. Some of the worst smoke was in Alberta where three major oil sands producers -- Canadian Natural Resources, MEG Energy Corp, and Cenovus Energy -- this week evacuated workers and temporarily shut down hundreds of thousands of barrels of production per day. Huge plumes of smoke even reached Europe, the European Union's climate monitoring service said Tuesday. Due to their very high altitude, they do not pose an immediate health risk, according to the Copernicus Atmosphere Monitoring Service (CAMS), but are likely to result in hazy skies and reddish-orange sunsets. Additional plumes are expected to shade both continents in the coming days. "Central regions of Canada have experienced a very intense few weeks in terms of wildfire emissions," noted Mark Parrington, scientific director at CAMS. Canadian authorities have forecast a more intense fire season than usual in central and western Canada, due in particular to severe or extreme drought. "The significant reduction in snowpack in the spring led to early exposure of soil and vegetation, accelerating surface drying," explained University of Ottawa professor Hossein Bonakdari. "This early exposure acted as a silent amplifier, subtly setting the stage for extreme fires long before the first flame ignited," he said. Elsewhere, extensive forest fires have been raging in Russia's Far Eastern Federal District since early April, particularly east of Lake Baikal, generating carbon emissions of around 35 million tons, Copernicus reported. © 2025 AFP
Japan Today
2 days ago
- Japan Today
Top scientist wants to prevent AI from going rogue
Concerned about the rapid spread of generative AI, a pioneer researcher is developing software to keep tabs on a technology that is increasingly taking over human tasks. Canadian computer science professor Yoshua Bengio is considered one of the godfathers of the artificial intelligence revolution and on Tuesday announced the launch of LawZero, a non-profit organization intended to mitigate the technology's inherent risks. The winner of the Turing Award, also known as the Nobel Prize for computer science, has been warning for several years of the risks of AI, whether through its malicious use or the software itself going awry. Those risks are increasing with the development of so-called AI agents, a use of the technology that tasks computers with making decisions that were once made by human workers. The goal of these agents is to build virtual employees that can do practically any job a human can, at a fraction of the cost. "Currently, AI is developed to maximize profit," Bengio said, adding it was being deployed even as it persists to show flaws. Moreover, for Bengio, giving AI human-like agency will easily be used for malicious purposes such as disinformation, bioweapons, and cyberattacks. "If we lose control of rogue super-intelligent AIs, they could greatly harm humanity," he said. One of the first objectives at LawZero will be to develop Scientist AI, a form of specially trained AI that can be used as a guardrail to ensure other AIs are behaving properly, the company said. The organization already has over 15 researchers and has received funding from Schmidt Sciences, a charity set up by former Google boss Eric Schmidt and his wife Wendy. The project comes as powerful large language models (or LLMs) from OpenAI, Google and Anthropic are deployed across all sectors of the digital economy, while still showing significant problems. These include AI models that show a capability to deceive and fabricate false information even as they increase productivity. In a recent example, AI company Anthropic said that during safety testing, its latest AI model tried to blackmail an engineer to avoid being replaced by another system. © 2025 AFP
