Fearing AI misuse, tech godfather Bengio sets up group to monitor rogue agents that could ‘greatly harm humanity'
NEW YORK, June 4 — Concerned about the rapid spread of generative AI, a pioneer researcher is developing software to keep tabs on a technology that is increasingly taking over human tasks.
Canadian computer science professor Yoshua Bengio is considered one of the godfathers of the artificial intelligence revolution and on Tuesday announced the launch of LawZero, a non-profit organisation intended to mitigate the technology's inherent risks.
The winner of the Turing Award, also known as the Nobel Prize for computer science, has been warning for several years of the risks of AI, whether through its malicious use or the software itself going awry.
Those risks are increasing with the development of so-called AI agents, a use of the technology that tasks computers with making decisions that were once made by human workers.
The goal of these agents is to build virtual employees that can do practically any job a human can, at a fraction of the cost.
'Currently, AI is developed to maximise profit,' Bengio said, adding it was being deployed even as it persists to show flaws.
Every frontier AI system should be grounded in a core commitment: to protect human joy and endeavour. Today, we launch @LawZero_, a nonprofit dedicated to advancing safe-by-design AI. https://t.co/6VJecvaXYT pic.twitter.com/PfXw5lVqhF
June 3, 2025
Moreover, for Bengio, giving AI human-like agency will easily be used for malicious purposes such as disinformation, bioweapons, and cyberattacks.
'If we lose control of rogue super-intelligent AIs, they could greatly harm humanity,' he said.
One of the first objectives at LawZero will be to develop Scientist AI, a form of specially trained AI that can be used as a guardrail to ensure other AIs are behaving properly, the company said.
The organisation already has over 15 researchers and has received funding from Schmidt Sciences, a charity set up by former Google boss Eric Schmidt and his wife Wendy.
The project comes as powerful large language models (or LLMs) from OpenAI, Google and Anthropic are deployed across all sectors of the digital economy, while still showing significant problems.
These include AI models that show a capability to deceive and fabricate false information even as they increase productivity.
In a recent example, AI company Anthropic said that during safety testing, its latest AI model tried to blackmail an engineer to avoid being replaced by another system. — AFP
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Free Malaysia Today
10 hours ago
- Free Malaysia Today
OpenAI finds more Chinese groups using ChatGPT for malicious purposes
OpenAI banned ChatGPT accounts that generated social media posts on political and geopolitical topics relevant to China. (AFP pic) SAN FRANCISCO : OpenAI is seeing an increasing number of Chinese groups using its artificial intelligence (AI) technology for covert operations, which the ChatGPT maker described in a report released today. While the scope and tactics employed by these groups have expanded, the operations detected were generally small in scale and targeted limited audiences, the San Francisco-based startup said. Since ChatGPT burst onto the scene in late 2022, there have been concerns about the potential consequences of generative AI technology, which can quickly and easily produce human-like text, imagery and audio. OpenAI regularly releases reports on malicious activity it detects on its platform, such as creating and debugging malware, or generating fake content for websites and social media platforms. In one example, OpenAI banned ChatGPT accounts that generated social media posts on political and geopolitical topics relevant to China, including criticism of a Taiwan-centric video game, false accusations against a Pakistani activist, and content related to the closure of USAID. Some content also criticised US President Donald Trump's sweeping tariffs, generating X posts, such as 'Tariffs make imported goods outrageously expensive, yet the government splurges on overseas aid. Who's supposed to keep eating?'. In another example, China-linked threat actors used AI to support various phases of their cyber operations, including open-source research, script modification, troubleshooting system configurations, and development of tools for password brute forcing and social media automation. A third example OpenAI found was a China-origin influence operation that generated polarised social media content supporting both sides of divisive topics within US political discourse, including text and AI-generated profile images. China's foreign ministry did not immediately respond to a Reuters request for comment on OpenAI's findings. OpenAI has cemented its position as one of the world's most valuable private companies after announcing a US$40 billion funding round valuing the company at US$300 billion.
Malay Mail
10 hours ago
- Malay Mail
Elon's revenge? Musk torpedoes Trump's ‘big, beautiful bill'
WASHINGTON, June 5 — Donald Trump's sweeping domestic policy bill has careened into an Elon Musk-shaped brick wall, complicating its passage into law and risking a schism between the most powerful man in the world and the wealthiest. The US president's 'big, beautiful bill' – the centerpiece of his domestic agenda – could define his second term and make or break Republican prospects in the 2026 midterm elections. But the package is getting a rough ride in Congress over proposals to fund an extension of his 2017 tax cuts by piling on debt and cutting social welfare for the poorest Americans. Enter tech billionaire Musk, who dropped a nuclear bomb on the 1,100-page blueprint at a crucial stage in negotiations Tuesday, calling it a 'disgusting abomination.' And on Wednesday he called for Republicans to 'kill the bill,' and for an alternative plan that 'doesn't massively grow the deficit.' In its latest estimate released Wednesday, the non-partisan Congressional Budget Office said the bill would add US$2.4 trillion to US debt by 2034. Musk, who last week ended his brief advisory tenure as Trump's cost-cutter-in-chief, tore into the bill in a prolonged denouncement. The South African-born tycoon has voiced concerns before, but his language was much more combative, coming across as a declaration of war on the Republican Party, if not Trump himself. The White House brushed off the criticism, saying Trump 'already knows where Elon Musk stood,' but the remarks will likely have gotten under the president's skin. 'Lennon and McCartney' The stakes could hardly be higher for Trump, who has made clear, with signature hyperbole, that he sees his bill as 'arguably the most significant piece of legislation that will ever be signed.' He has yet to respond personally to Musk's social media rant – sustained over six hours and 13 posts on Tuesday and still going well into Wednesday. But Washington watchers aren't necessarily expecting a public falling-out. Behind the scenes, a careful circling of the wagons is underway, with pro-Trump pundits under orders to refrain from trashing Musk and to instead frame his broadside as what Politico called 'principled self-interest.' Still, for analysts contacted by AFP, there may be choppier waters ahead. 'It's the Lennon and McCartney of modern politics. Two egos, one spotlight, and a fragile alliance built on mutual benefits,' said Evan Nierman, the founder and CEO of global crisis PR firm Red Banyan. 'The moment either one sees more upside in conflict than cooperation, the breakup goes public.' But political consultant and former Senate aide Andrew Koneschusky, a key player in negotiations over Trump's 2017 tax cuts, believes the Republican leader has nothing to gain by biting back. 'Musk has more money. Musk's megaphone, X, is bigger than Trump's. And Musk was deeply embedded in the administration for months,' he told AFP. 'There's no telling what Musk heard or saw that could be embarrassing or problematic for the administration if the two were to go to war.' 'Bromance' Musk – who has cash to spare for his political adventures, including US$280 million lavished on Trump's election campaign – slammed the Republicans who cleared the bill for Senate consideration and threatened to have their jobs in the midterms. And the de facto leader of America's 'tech bro' community demonstrated his political firepower with a single tweet in December that blew up a government funding bill that had enjoyed bipartisan support. His take on the deficit implications of Trump's proposals is evidenced by copious independent research and he was immediately backed by some fiscal hawks in the Senate. Continued interventions by Musk could be an ongoing headache for Trump, as he bids to shepherd his policy priorities through razor-thin Republican majorities in Congress. But cheerleaders of the package – and independent analysts contacted by AFP – believe the Tesla magnate may discover that his celebrity in Trumpworld relies on the say-so of its mercurial chieftain. 'Musk may have had influence in December when his bromance with Trump was in full bloom,' said Donald Nieman, a political analyst and professor at Binghamton University in New York state. 'But his break with Trump and his massive unpopularity with voters makes it easy for lawmakers to ignore him. If anything, it helps Trump by distancing him from a man who has become a pariah.' — AFP
Malay Mail
10 hours ago
- Malay Mail
Politics 1–Football 0: Washington bars Iranians from cheering team in US
TEHRAN, June 5 — A year out from kick-off, Iranian football fans are watching their World Cup dream slip away after a US travel ban barred them from entering the land of 'Great Satan' to cheer on their team. The 2026 tournament will be co-hosted by the United States, Canada and Mexico, but most matches, including the final, are scheduled to be played on American soil. Many in Iran had clung to hopes of cheering from the stands until Wednesday when US President Donald Trump rolled out a new travel ban on 12 countries including Iran, which will take effect from Monday. 'My friends and I have been waiting for years to watch Team Melli (a nickname for the national team) play in a World Cup on US soil, and when they qualified, it felt like a once-in-a-lifetime opportunity,' Sohrab Naderi, a real estate agent in Tehran, told AFP. 'Now with the new travel ban, that dream is shattered because of politics that we don't care about and have no control over,' said the 46-year-old who attended the 2022 World Cup in Qatar which saw the US side defeat Iran 1-0 in the group stage. The prospect of Iran competing in a US-hosted tournament comes against the backdrop of a decades-long enmity, with diplomatic ties broken since the 1979 Islamic revolution. The two sides are currently engaged in high-stakes talks over Iran's nuclear programme, with the United States threatening military action if no deal can be reached. 'Degrading to all Iranians' Trump said the new travel ban was prompted by a makeshift flamethrower attack on a Jewish protest in Colorado that US authorities blamed on a man they said was in the country illegally. The ban will not apply to athletes competing in either the 2026 World Cup or the 2028 Los Angeles Olympics, the order said. Nonetheless, supporters who had dreamed of crossing the Atlantic to cheer on their team will no longer be able to make the trip. 'Every Iranian has the right to support their team, just as much as any other country, whether the game is in America or in any other country,' said Hasti Teymourpour, a 16-year-old football fan. Since his return to office in January, Trump has reinstated his 'maximum pressure' policy of sanctions against Iran and vowed that 'something bad' would happen unless the Iranians 'move quickly' towards a nuclear deal. Naderi, who called the ban 'inhumane' and 'degrading to all Iranians', still hopes the Iran-US nuclear talks will yield a deal that might persuade Trump to reconsider. The outcome of the US-Iran talks that began in April remains unclear, and many fans worry that even if they result in a deal, it may be too late for them. Some Iranians have refused to give up hope, however, seeing in the World Cup an opportunity to thaw relations. 'Sports diplomacy can act as a strong catalyst and bring the efforts of political diplomats to fruition sooner,' said political commentator Mohammad Reza Manafi. It could be 'a great opportunity to help advance diplomacy between the two countries'. Friendly? In a memorable 1998 World Cup clash, Iranian players handed flowers to their American adversaries and posed together for photos – a rare public gesture of goodwill between the nations. Iran won 2-1, a victory celebrated in Tehran as a source of both sporting and political pride. With the 2026 draw expected in December, it remains unclear whether Iran and the United States will face off again, but anticipation is building. 'The two countries are not hostile to each other, this political discussion is for the governments,' said 44-year-old day labourer Siamak Kalantari. Another fan, Mahdieh Olfati, said: 'If we face the US again, we'll definitely win.' 'Ours are real players,' the 18-year-old added. Manafi, the commentator, said a friendly before the tournament, possibly hosted by a third country, could help ease tensions. Such a game, he said, could help 'achieve what politicians from both sides have not managed to do for years'. — AFP
