Computer scientist to develop 'honest' AI that will spot rogue systems and flag 'harmful behaviour'
An artificial intelligence pioneer has launched a non-profit dedicated to developing an 'honest' AI that will spot rogue systems attempting to deceive humans.
Yoshua Bengio, a renowned computer scientist described as one of the 'godfathers' of AI, will be president of LawZero, an organisation committed to the safe design of the cutting-edge technology that has sparked a $1tn (€877bn) arms race.
Starting with funding of about $30m (€26.3m) and more than a dozen researchers, Bengio is developing a system called Scientist AI that will act as a guardrail against AI agents — which carry out tasks without human intervention — showing deceptive or self-preserving behaviour, such as trying to avoid being turned off.
Describing the current suite of AI agents as 'actors' seeking to imitate humans and please users, he said the Scientist AI system would be more like a 'psychologist' that can understand and predict bad behaviour.
'We want to build AIs that will be honest and not deceptive,' Bengio said.
'It is theoretically possible to imagine machines that have no self, no goal for themselves, that are just pure knowledge machines — like a scientist who knows a lot of stuff.'
However, unlike current generative AI tools, Bengio's system will not give definitive answers and will instead give probabilities for whether an answer is correct.
'It has a sense of humility that it isn't sure about the answer,' he said.
Deployed alongside an AI agent, Bengio's model would flag potentially harmful behaviour by an autonomous system — having gauged the probability of its actions causing harm.
Scientist AI will 'predict the probability that an agent's actions will lead to harm' and, if that probability is above a certain threshold, that agent's proposed action will then be blocked.
LawZero's initial backers include AI safety body the Future of Life Institute, Jaan Tallinn, a founding engineer of Skype, and Schmidt Sciences, a research body founded by former Google chief executive Eric Schmidt.
Bengio said the first step for LawZero would be demonstrating the methodology behind the concept works — and then persuading companies or governments to support larger, more powerful versions. Open-source AI models, which are freely available to deploy and adapt, would be the starting point for training LawZero's systems, Bengio added.
'The point is to demonstrate the methodology so that then we can convince either donors or governments or AI labs to put the resources that are needed to train this at the same scale as the current frontier AIs. It is really important that the guardrail AI be at least as smart as the AI agent that it is trying to monitor and control,' he said.
Bengio, a professor at the University of Montreal, earned the 'godfather' moniker after sharing the 2018 Turing award — seen as the equivalent of a Nobel prize for computing — with Geoffrey Hinton, himself a subsequent Nobel winner, and Yann LeCun, the chief AI scientist at Mark Zuckerberg's Meta.
A leading voice on AI safety, he chaired the recent International AI Safety report, which warned autonomous agents could cause 'severe' disruption if they become 'capable of completing longer sequences of tasks without human supervision'.
The Guardian
Read More
The real cost of slightly funnier AI is the health of a poor black community
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Irish Examiner
9 hours ago
- Irish Examiner
Manufacturers need to invest in AI or risk falling behind, PwC advises
Irish-based manufacturers need to accelerate their adoption of AI tools to avoid being left behind Europe, Middle East and Africa, experts at PwC have advised. Just 3% of Irish manufacturing companies have fully integrated AI into their operations versus 8% for EMEA companies. While the gap cited in PwC's report 'AI in Operations — Revolutionising the manufacturing industry' isn't huge, Irish firms are being urged to act now and not let it widen. Áine Brassill, operations transformation partner, PwC Ireland, takes heart from the survey findings that there is a lot of AI innovation and piloting going on, with many Irish firms clearly looking to catch up. 'Around 70% of Irish respondents are piloting and scaling up their AI projects compared to 55% for EMEA counterparts. We can, therefore expect a surge in widespread AI implementation in Irish manufacturing in the years to come,' said Áine. 'While the initial focus regarding AI implementation is on operational and productivity improvements, the real interest lies in the potential to disrupt and fundamentally reinvent existing business models. AI agents will make the ability for AI systems to autonomously perform tasks a reality, enabling decision making and delivering real competitive differentiation. 'However, building trust in AI will be critical for customers, regulators and employees. Companies need to be confident in the integrity of solutions that will drive safe and secure AI outcomes. Taking a responsible approach, including upskilling employees, will be critical to getting the most from AI alongside confirming future compliance with regulators and the EU AI Act.' Áine Brassill, operations transformation partner, PwC Ireland. The report found that both Irish and EMEA manufacturers believe in AI's potential to increase profitability. Irish operations, however, are less optimistic than their EMEA counterparts as regards their belief in what AI can deliver. Some 73% of Irish manufacturing operations expect AI to increase profitability by 2030 versus 84% of EMEA respondents. Just 26% of Irish respondents expect AI to increase profit margins by at least 6% by 2030, trailing the 40% of EMEA manufacturers. The report features 400 manufacturing operations' executives in over 30 countries in Europe, Middle East and Africa, including Ireland. In Ireland, 43% of respondents were in pharma/life science and med-tech multinational operations with the balance in retail and consumer and industrial products. AI in Ireland is also maturing relatively slowly. Some 29% of Irish respondents reported no business benefits as yet from AI versus 14% for the EMEA. Just 4% of Irish and EMEA manufacturers have already enjoyed financial benefits and return on investment from AI. A further 11% report that they have received measurable financial benefits (EMEA: 13%). Irish manufacturing operations are also investing less in AI initiatives than their EMEA counterparts. In the last five years, 32% of Irish manufacturing operations invested less than €1m in AI initiatives (EMEA: 29%). 15% invested in excess of €6 million compared to 41% for their EMEA counterparts. In both Ireland and EMEA, progress is slow on AI. Just 3% of Irish manufacturing companies have fully integrated AI into their operations compared to 8% for EMEA companies. At the same time, there is a lot of AI innovation and piloting going on with many Irish firms clearly looking to catch up: 70% of Irish respondents are piloting and scaling up their AI projects compared to 55% for EMEA counterparts. We can therefore expect a surge in widespread AI implementation in Irish manufacturing in the years to come. Gary Hanniffy, director of manufacturing, PwC Ireland, said: 'Like many businesses, the manufacturing industry is facing significant uncertainty stemming from geopolitical disruption, economic fragmentation, supply chain volatility, tightening regulation, climate change, technology transformation and increasing costs. 'AI offers a real opportunity for business model reinvention for manufacturing operations and our study shows that the potential benefits from AI will justify the effort. The survey suggests that manufacturing operations can become more competitive as a result of full-scale AI adoption. 'The survey highlights that a majority of Irish manufacturing operations, consisting largely of pharma, life sciences and med-tech multinational companies, are piloting AI initiatives rather than having moved to scaling and integrating the technology right across their business operations while EMEA companies are more advanced in their implementation journey. 'At the same time, they do have high expectations for realising the benefits from AI in terms of profitability and other financial benefits,' Gary added. 'Getting to the next level requires investment and results here are mixed, with some companies planning significant investment levels, others are not yet ready to commit. In Ireland, in particular, more investment in AI is also needed to keep up with EMEA peers.' Read More Business movers: People starting new jobs in Ireland The survey highlights a number of key challenges for successful AI implementation. These include: data quality; IT & data security; reliability of AI-generated content and data availability. Gary Hanniffy said: 'Strong organisational structures and processes are essential for steering and delivering a successful AI strategy and to enable safe and secure outcomes. Those manufacturing organisations who have integrated and scaled up their AI projects are using an organisational governance model that involves a central AI team (Ireland: 100%; EMEA: 71%). 'On the other hand, the majority of those organisations that have just started piloting AI lack coordinated governance and are using non-centralised organisational structures (Ireland: 76%; EMEA: 58%). This finding is in line with PwC's earlier 2025 GenAI Business Leaders survey, indicating that more work needs to be done on AI trust and governance such as building appropriate governance structures.'
Irish Times
11 hours ago
- Irish Times
PwC Ireland boss focused on getting tech to do some heavy lifting
'The future is technology and you've got to lean in and embrace that,' says Enda McDonagh , managing partner of PwC Ireland , one of the Big Four accounting firms in the State. We're speaking in his office overlooking the Liffey in Dublin's north Docklands. The impressive double aspect view stretches down to Liberty Hall on one side and up towards Capital Dock the other way. 'It's a great view all right, but I usually have my back to it working on the computer,' he says. Technology is at the heart of the change that has been taking place on McDonagh's watch since he took over the reins at PwC almost two years ago. READ MORE 'The big thing we're trying to get after is a much better focus on the technology part of what we can bring to our clients,' he says, adding that PwC has long had a good reputation in that area. Artificial intelligence is the latest technology taking centre stage, particularly generative AI. 'We've got stuck into AI in a big way in the past couple of years,' he says. The firm converted to Microsoft M365, which brings Copilot capability with it. As part of a global collaboration with OpenAI , it has created Chat PwC to undertake tasks that 'previously would have been done by people manually researching, and helps speed up the productivity on a day-to-day basis'. Can he trust its accuracy? 'You always have validation checks, but because it's pulling from our own materials, it's very reliable,' he says. PwC is also using Harvey, a tax and legal tool, generating first-level content to use in the practice. McDonagh says AI is not replacing jobs at the firm. 'There hasn't really been any net replacement of our people with AI. We're trying to take tasks from our people in the administrative space and repurpose what they do,' he says. 'We're not seeing AI replacing humans, we're just seeing it changing what the human does in the dynamic.' Last year, PwC added 1,028 staff, including 383 graduates, to bring its headcount to 3,566. McDonagh says a similar number will be added in 2025, with 'broadly similar numbers' employed by the firm overall, reflecting the significant annual churn at professional services firms. PwC Ireland reported a 2.8 per cent increase in net revenue last year to €469 million . In gross terms, revenue was €569 million, up 9.6 per cent on 2023, but its 'expenses and disbursements on client engagements' rose by 59 per cent to €100 million. All the figures exclude VAT and relate to income generated by PwC's firms in the Republic and Northern Ireland. But it does not include the firm's advisory and consulting activities in the North, which are part of the UK firm and employ about 3,000 people. Advisory services was the big drag on PwC Ireland's numbers, declining by €16 million to €107 million. This included €63 million (up from €59 million a year earlier) in revenue from permitted non-audit services to entities that are audited by the Republic of Ireland firm of PwC. Its assurance and tax units recorded revenue increases of 9 per cent and 7.4 per cent respectively. McDonagh says the slowing down in its advisory business was a combination of fewer deals and more prudence from clients in terms of discretionary spend. 'What we've entered into a period of now is uncertainty, which is the worst possible thing for businesses,' he says by way of reference to US president Donald Trump and his flip-flops on tariffs . 'This ebb and flow period of going to bed at night and waking up in the morning and the goalposts have shifted again is the really tricky part. We are seeing a real slowdown in new investment allocations, M&A is on pause and there's a period of stasis.' Setting Trump aside, McDonagh says now is the time for Ireland to tackle its 'competitiveness issues', particularly around infrastructure, to 'stay agile' and continue to attract new investment. 'We're going to have to work harder than we have done in a while on the Irish story in the context of what's going on. The question for all of us where does the whole tariff debate land,' he says. In spite of the market being a 'bit more challenging', McDonagh says PwC's ambition is to 'keep growing' the firm, hiring in the right skills rather than making acquisitions. 'M&A would take place at a [global] network level more than a local level,' he says. PwC is the smallest of the Irish Big Four firms. EY is the clear leader with annual revenues last reported at €772 million for the island. KPMG's income was €625 million and Deloitte's €517 million. 'We just focus on what we're doing in terms of our own strategy. There are lots of apples and oranges in those numbers. We're happy with our own numbers ... when one of your businesses [advisory] is down by double digits and you're still able to grow and trade through, that's a good result for the year,' he says. In terms of this year's revenues, McDonagh is 'hopeful' of beating the 3 per cent growth achieved last time, while recognising the global economic uncertainty that exists. He declines to comment on the trends in pay and profits for Irish partners, amid reports that UK partner pay last year fell by 5 per cent on average while the firm's profits declined by 14 per cent. In the US, the firm has been laying off thousands of staff while profits in Asia-Pacific fell as scandals in China and Australia affected the firm. 'If your revenue is growing at less than it did before, that obviously has an impact, but I'm not getting into it beyond the revenue figure.' The Irish firm was itself dragged into a scandal last year involving PwC in Australia when it was revealed that a partner in its Australian tax practice had used confidential information from government meetings to assist colleagues in winning new business from multinational technology companies. It emerged that former PwC Australia tax partner Peter-John Collins, who was banned from practising as a tax agent after sharing secret information, sought in 2015 to tap PwC Ireland's contacts with technology companies in the United States to share his insights. PwC Ireland repeatedly declined to disclose the identity of the individual in its firm who received the email. McDonagh declined to comment on the Australia issue, saying it had been dealt with 'at international level' by the firm and there was no issue for the Irish business. Back to the day-to-day business at home, PwC last September enforced a three-day-a-week policy for staff attendance in its offices (it has bases in Dublin, Belfast, Cork, Galway, Kilkenny, Limerick, Waterford and Wexford). 'The reason we looked for a three-day commitment is because our business is about people and one of our core philosophies is about high-performing teams and people being together in that team environment so they can learn and grow and foster. Getting that three days in the office is the right balance for us at the moment,' he says. McDonagh says attendance at its headquarters in Dublin peaks at about 1,600 staff on days midweek, a similar level to pre-Covid days. 'Fridays are a lost day to the city centre,' he says. 'Like most everybody else, we went into Covid not thinking a business like ours could operate remotely. The business was incredibly resilient. But a huge amount of how you develop as a professional is the learned experience of being in a team environment and learning from interactions. It's working well for us now.' Born in Southend in southeast England, his family returned to Ireland when he was two and McDonagh grew up in Tallaght, southwest Dublin. This was a time before the M50 and the Square shopping centre when there was a lot of farmland in the area. 'A lot of football and other games would have been played on those spaces back then,' he says. His late father hailed from Enniskillen, Co Fermanagh, his mother from Clones, Co Monaghan. 'I'm a big GAA fan and I probably went to more Ulster finals [in Clones] as a kid growing up by dint of spending time with my mother's family in the summer,' he says. In spite of that childhood experience, he is a 'diehard Dubs fan, there's no ambiguity when it comes to that', but he always has a 'soft spot' for Ulster teams in the championship. His father worked in Irish Rail while his mother's family was behind a hardware store in Clones called Morgans. He attended Templeogue College for secondary school before taking a BComm business degree in University College Dublin and joining Craig Gardner Price Waterhouse (one half of what later became PwC) in 1994 via the annual milk-round recruitment sweep of universities that the big accounting firms all do. He later followed that with a master's degree in accounting, sponsored by the firm. Why accounting? 'I had a fantastic accounting teacher in the way that, in schools, you can get. That's what fostered the interest in accounting,' he says. He worked as an auditor, making his way up the ranks to manager level, at which point he took advantage of PwC's international transfer system to take a role in Boston in 2000, spending three years in the US city. 'I wanted to do something different. So I applied to go to the US and Boston came out of the hopper,' McDonagh says. 'Being Irish there is not culturally a shock to the system. Boston is one of the more European American cities if that's an appropriate analogy. The downtown area is not the block model of a lot of American cities. In terms of quality of life and opportunity, Boston at that time was brilliant. 'I also worked a lot with the pharmaceutical industry over there, so when I came back, with the growth in pharma here, it worked out very well for me.' In 2006, McDonagh became a partner in the firm, led the assurance practice and served eight years on the leadership team of his predecessor as managing partner, Feargal O'Rourke . He was chosen to lead the firm two years ago, taking over from O'Rourke in July 2023 for a four-year term. Will he go again for second term? 'That will be for the [146] partners to decide,' he says with a smile. 'I'm really enjoying the role. It's a privilege to lead a firm like this. As long as people want me around, I'm happy to stay. 'If you'd asked me in 1994 did I ever think I'd be sitting in a chair like this, I'd never have thought that in a million years. But there's an innate part of me that thinks I want to be the best that I can be.' CV Name: Enda McDonagh Job: Managing partner, PwC Age: 52 Lives: Blackrock, Co Dublin Family: Married, with three children: a 17-year-old son and 15-year-old twins (one boy, one girl). Hobbies: Big fan of American sports, having lived and worked in Boston for three years. Also a Manchester United season ticket holder and a 'diehard' Dublin GAA fan. Something we might expect: 'I'm a great believer in the importance of team. The highlight of any week is when I get a call from a client to tell me the team we have working with them have gone above and beyond to deliver for them.' Something that might surprise: 'Notwithstanding my well-known support of Irish sporting teams, I was actually born in England.'
Irish Independent
19 hours ago
- Irish Independent
Irish duo's Solidroad start-up lands $6.5m in funding as its AI takes over contact centre staff training
Mark Hughes is sitting in Solidroad's Dublin office, patiently waiting for his US visa to arrive before he rejoins the rest of the AI-powered start-up in San Francisco.
