Top scientist wants to prevent AI from going rogue
[NEW YORK] Concerned about the rapid spread of generative AI, a pioneer researcher is developing software to keep tabs on a technology that is increasingly taking over human tasks.
Canadian computer science professor Yoshua Bengio is considered one of the godfathers of the artificial intelligence revolution and on Tuesday announced the launch of LawZero, a non-profit organization intended to mitigate the technology's inherent risks.
The winner of the Turing Award, also known as the Nobel Prize for computer science, has been warning for several years of the risks of AI, whether through its malicious use or the software itself going awry.
Those risks are increasing with the development of so-called AI agents, a use of the technology that tasks computers with making decisions that were once made by human workers.
The goal of these agents is to build virtual employees that can do practically any job a human can, at a fraction of the cost.
'Currently, AI is developed to maximise profit,' Bengio said, adding it was being deployed even as it persists to show flaws.
BT in your inbox
Start and end each day with the latest news stories and analyses delivered straight to your inbox.
Sign Up
Sign Up
Moreover, for Bengio, giving AI human-like agency will easily be used for malicious purposes such as disinformation, bioweapons, and cyberattacks.
'If we lose control of rogue super-intelligent AIs, they could greatly harm humanity,' he said.
One of the first objectives at LawZero will be to develop Scientist AI, a form of specially trained AI that can be used as a guardrail to ensure other AIs are behaving properly, the company said.
The organisation already has over 15 researchers and has received funding from Schmidt Sciences, a charity set up by former Google boss Eric Schmidt and his wife Wendy.
The project comes as powerful large language models (or LLMs) from OpenAI, Google and Anthropic are deployed across all sectors of the digital economy, while still showing significant problems.
These include AI models that show a capability to deceive and fabricate false information even as they increase productivity.
In a recent example, AI company Anthropic said that during safety testing, its latest AI model tried to blackmail an engineer to avoid being replaced by another system. AFP
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Nylon
16 hours ago
- Nylon
Google and Gentle Monster set to launch AI smart eyewear collaboration with Android XR
At the recent Google I/O 2025, the tech giant and South Korean global eyewear brand Gentle Monster announced that they will be teaming up to launch their first-ever AI smart eyewear collaboration with Android XR. Image courtesy of Google. During the announcement, General Manager and Vice President of Google XR, Shahram Izadi, shared that creativity and sophistication are essential design features for technology to integrate into everyday life and with Gentle Monster onboard as the first partner highlights a pivotal step in the evolution of AI smart eyewear into essential, lifestyle fashion items. Although no launch date for the collaboration has been set yet, Google foresees that the new partnership is looking to seamlessly integrate extended reality into everyday life by showcasing cutting-edge smart technology alongside the refined aesthetics of Gentle Monster.
Business Times
19 hours ago
- Business Times
We need a new deal for the Web
GIVEN that Google attracts more than 136 billion visits a month – as many as the next 12 most popular websites combined – it is easy to see how a US federal judge concluded last year that the company ran a monopoly in online search and had abused its market dominance. Last week, Judge Amit Mehta heard the final arguments about what remedies to impose. His conclusions are expected in August. The US justice department, which launched the antitrust lawsuit, is demanding that Google be broken up. It wants to force the company to sell its Chrome browser, ban the massive payments it makes to Apple, Samsung and Mozilla to be their default search engine, and share data with competitors. That would surely be a triumph for free-market competition, motherhood and apple pie. Yet, as ever with regulatory interventions, the skill lies in ensuring that any corrective action anticipates tomorrow's challenges rather than just trying to fix today's problems. That's especially tough right now. As the judge acknowledged, the digital economy is evolving fast thanks to the light-speed diffusion of artificial intelligence. Some suggest that AI might even undermine the financial viability of the open, human, ad-supported Web on which Google has gorged. However much it may (or may not) abuse its market dominance, Google remains the strongest champion of the existing Web and many users would miss it if it were gone. Since being invented 36 years ago, the Web has emerged as one of humanity's most precious resources, giving any online user anywhere access to almost all of the world's knowledge. It has also become the foundation on which many thousands of digital businesses have been built. But it has its darker deformities and there is a risk that AI might further degrade it in two insidious ways. First, the big AI companies are in effect strip-mining all websites for content to train their models and paying back almost nothing in return to sustain the ecosystem. At present, Google accounts for about 90 per cent of the global search market and directs a torrent of traffic – and advertising – to content sites. Even though they may (understandably) complain about the terms, some content creators fear the trade itself may be in jeopardy because far fewer users would visit their sites. BT in your inbox Start and end each day with the latest news stories and analyses delivered straight to your inbox. Sign Up Sign Up AI models aggregate and summarise the Web's content rather than encouraging users to visit the original content source. That may destroy the financial incentives for creating new content. 'I think that is a risk,' Laura Chambers, chief executive of the technology company Mozilla, tells me. 'How can we ensure that the Internet remains healthy?' The obvious survival mechanism for creators is to build walled gardens by erecting more paywalls around their content, or to move to closed channels off the open Web. For many companies, it may make sense to deepen direct-to-consumer business models via social media or their own apps. But both trends would further devalue the richness, utility and universality of the Web. The second threat is that AI models are increasingly flooding the Web with machine-generated slop. For the first time in a decade, bots have overtaken humans on the Web, accounting for an estimated 51 per cent of total traffic, according to the data and US cybersecurity group Imperva. The ability of generative AI to create plausible content at minimal cost means this trend is only likely to accelerate. An AI-mediated Web could move it further towards a 'dark forest', an increasingly hostile space populated by predatory bots that seize on any living thing. We may not like our current surveillance capitalism, in which users are tracked and targeted with ads. Subordination capitalism would be still worse. Yet, there are more positive visions of the Web's future, as recently sketched out by Kevin Scott, chief technology officer of Microsoft, which runs Bing, a very distant second in search. The increasing interaction of bots means that new open protocols are being developed to enable interoperability on this agentic Web. That creates the possibility for a different Web architecture and a 'new deal', in which 'everybody's incentives are aligned, where the creators and consumers have their interests balanced and there aren't a bunch of weird intermediaries constraining how utility and value gets exchanged', Scott told The Verge. No one yet knows exactly how to build such a glorious future. But whatever Judge Mehta can do to help nudge us in that direction would be appreciated. FINANCIAL TIMES
Business Times
a day ago
- Business Times
Local investor eyes acquisition of Singapore private club 1880 amid Hong Kong branch collapse
[SINGAPORE] A local investor is looking into the potential acquisition of Singapore private club 1880, following the sudden closure of its Hong Kong branch last Friday (Jun 6) – just seven months after opening in the territory. A source close to the matter told The Business Times that the potential buyer is 'in the very early stages' of doing due diligence, and sees the Singapore club as 'a viable and very decent' business. However, its ambitious expansion to Hong Kong has impacted the entire group. 1880 was founded in 2017 by Canadian entrepreneur, former adman and child actor Marc Nicholson, who has lived here since 2002. The club's luxurious 22,000 square foot space on the third level of InterContinental Singapore Robertson Quay was conceived by star British designer Timothy Oulton. Its name, 1880, refers to the decade Robertson Quay was founded. 1880 founder Marc Nicholson at the escalator shaft – designed to look like a giant kaleidoscope – leading to the club. PHOTO: BT FILE The club has since established itself as a hub where individuals can come together for discussions on topics ranging from global conflicts to social justice – just like the weekly gatherings of opinionated individuals that Nicholson's bohemian-minded parents used to organise in Montreal. There are about 100 to 150 programmes a month, including talks and dinners meant as platforms for social interaction and debate. The Hong Kong branch started operating in November last year. But unlike the Singapore version, part of it is open to the public. The club was also to have expanded to a six-storey beachside hotel it was constructing in Bali, but that business never opened. Citing former employees, the South China Morning Post (SCMP) said 1880 Hong Kong had about HK$20 million (S$3.3 million) of debts. It is now facing a probe by authorities after owing wages to over 100 staff. The club is also in rental arrears, having taken four storeys in Swire Properties' Two Taikoo Place office complex in Quarry Bay. A NEWSLETTER FOR YOU Friday, 2 pm Lifestyle Our picks of the latest dining, travel and leisure options to treat yourself. Sign Up Sign Up SCMP reported that each member in Hong Kong had to pay a joining fee of around HK$24,000 and a monthly subscription fee of HK$1,300, or HK$14,000 for a full year. In an email to its Hong Kong members, Nicholson said the company had 'run into cashflow difficulties' and had not been able to recover from them. 'We have made every attempt to raise funds to continue the operations but were unsuccessful,' he wrote. He said 1880 Singapore will retain its brand and 'continue under a new group'. 1880 Hong Kong's members will be granted a one-year overseas membership to the Singapore club, as well as all reciprocal clubs around the world. Separately, in a letter to employees, Nicholson acknowledged the failure to secure new funding and apologised. In Singapore, the source told BT that 1880 is still able to pay its employees here and make partial payments for rent and to suppliers. Work is underway to fully understand how extensive its liabilities are – including the potential impact of Hong Kong members using the Singapore club. 'From a standalone perspective, without the burden of the Hong Kong branch, the business in Singapore can sustain itself,' says the source. 'It's not as toxic as it is in Hong Kong.' BT understands that there are over 2,000 members in Singapore and membership fees are reported to be S$5,000+, with monthly fees of $180+. In a 2019 interview with BT, Nicholson said he was looking to attract a maximum of 2,000 members. 'A sufficient base of members who come frequently, use the facilities and bring their guests will keep us profitable,' he said, adding that the club here has seven business units, including a restaurant, bar, spa and co-working space. 'And if each of these is profitable, turning a 10 per cent or so, that can cover the rent, cost of goods sold, staff wages and so on.' Should the club be acquired, the source said the potential buyer plans to retain its brand and focus on running the business properly with fresh capital and new ideas.
