AI pioneer launches non-profit to develop safe-by-design AI models
One of the world's most cited artificial intelligence (AI) researchers is launching a new non-profit that will design safe AI systems.
Yoshua Bengio, a Canadian-French AI scientist who has won the prestigious Alan Turing Prize for his work on deep learning and has been dubbed one of the "godfathers" of AI, announced the launch of LawZero in Montreal.
The new non-profit is assembling a "world-class" team of AI researchers that is dedicated to "prioritising safety over commercial imperatives," a statement from the non-profit reads.
"Today's frontier AI models are developing dangerous capabilities and behaviours, including deception, self-preservation, and goal misalignment," Bengio said in the statement, noting that the organisation will help unlock the "immense potential" of AI while reducing these risks.
Bengio said the non-profit was born of a new "scientific direction" he took in 2023, which has culminated in "Scientist AI," a new non-agentic AI system he and his team are developing to act as a guardrail against "uncontrolled" agentic AI systems.
This principle is different than other companies in that it wants to prioritise non-agentic AIs, meaning it needs direct instructions for each task instead of independently coming up with the answers, like most AI systems.
The non-agentic AIs built by LawZero will "learn to understand the world rather than act in it," and will be trained to give "truthful answers to questions based on [external] reasoning".
Bengio elaborated on Scientist AI in a recent opinion piece for Time, where he wrote that he is "genuinely unsettled by the behaviour unrestrained AI is already demonstrating, in particular self-preservation and deception".
"Rather than trying to please humans, Scientist AI could be designed to prioritise honesty," he wrote.
The organisation has received donations from other AI institutes like the Future of Life Institute, Jaan Tallin, and the Silicon Valley Community Foundation in its incubator phase.
LawZero will be working out of the MILA - Quebec AI Institute in Montreal, which Bengio helped co-found.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
France 24
19 hours ago
- France 24
Environmental DNA, a ‘revolutionary' key to unlocking the secrets of our oceans
Pierre Jorcin slides on a pair of gloves, attaches a plastic tube to a filter, plunges it into the water, presses the start button on a small pump and then slowly begins walking through the river stream. Thirty minutes later, he has gathered three litres of water and filtered thousands of particles. The entire procedure seems simple, banal even. But Jorcin's gesture is part of a microscopic revolution. In the process, the scientist has collected fragments of environmental DNA, also known as eDNA. 'Every living organism leaves traces of DNA behind, whether in water, soil or in the air. And those traces hold out for some time before eventually degrading,' Jorcin explains. 'By collecting them, we can identify and catalogue the organisms we find like bacteria, mammals, amphibians, fish, etc.' The samples Jorcin collects are then transported a few kilometres away to the University of Savoie Mont Blanc in Chambéry, an Alpine town in southeast France. That is where the offices of Spygen are based, a French pioneer in environmental DNA and the only company in the country that markets eDNA kits. Spygen sells the kits to NGOs, universities and private stakeholders, and then collects the samples to decode them. Dozens and dozens of samples are processed in the small premises of the company, which analyses the trapped eDNA to try and identify what species it belongs to. 'We extract the DNA from the filters and then run it through the sequencing machines,' says Jorcin, who is a project manager at the company. After sequencing, a long succession of four letters emerges – A, T, C, G – representing the genetic code of all species. 'Then it's up to us to find which species the code belongs to by looking through reference databases,' he explains. A burgeoning practice Spygen didn't end up at the foot of the Alps by accident. While the practice of sampling and sequencing eDNA was first tested by a US microbiologist in the 90s, this small revolution truly began about 60 kilometres south of Chambéry. Researchers from the Alpine ecology lab in Grenoble in 2008 found that the method could be used to detect bullfrogs, an invasive species, in places it had not yet been seen. A year later, scientist and Spygen co-founder Alice Valentini came up with the idea of using eDNA in her study on Himalayan brown bears, an endangered species. 'To get a better understanding of the bear's low reproductive capacity, Valentini tested the eDNA of food in its feces, which allowed her to get a precise rundown of its diet. It was a completely new method," says Benjamin Allegrini, president of the company. "That's when we understood the full scope of eDNA. Not only can we detect what organisms or species exist in different environments, but we can also understand how they interact with one another,' he says, beaming. A passionate birdwatcher since he was a child, Allegrini kicked off his career, binoculars in hand, as an ornithologist and went on to study bats. 'Then I discovered how much we could learn from DNA," he recounts. Once he had earned a degree in molecular biology, he decided to devote himself entirely to Spygen from 2018 onwards. 'The more we know about our environment, the more we'll be able to protect it,' says Allegrini, who recently published a book on eDNA. 'That's why it's important to develop new technologies like eDNA, which can revolutionise how we see the world.' The number of scientific papers on environmental DNA has skyrocketed since Spygen was founded in 2011, with each publication confirming the effectiveness of the method a little more. It is now a widely used approach, and is even part of France's new national strategy to protect biodiversity, which was published late last year. The country plans to 'regularly and extensively' document its national biodiversity with a census creating during a 'large-scale campaign to collect and analyse environmental DNA'. An ambitious inventory for underwater species And if there is one habitat where environmental DNA is especially useful, it's in our seas and oceans. Much of the life that exists in the vastness of these underwater worlds is invisible. 'In just one litre of seawater, there are 28 million DNA sequences. But only 14 percent of those sequences are identifiable. That means there are millions and millions of genetic sequences we know nothing about,' Allegrini explains. 'Until now, traditional methods used to document marine biodiversity yielded fragmented results,' says Yvan Griboval, a sailor who is also the head of the French NGO OceanoScientific, which works to collect scientific data from our oceans. 'Environmental DNA opens up a new range of possibilities by offering a tool that is no longer based on estimates or observations, but on factual data.' Scientists traditionally rely on fishing returns, diving expeditions, underwater cameras or acoustic surveys to observe life underwater. Collecting eDNA is a less invasive procedure, with no bait or electric fishing needed to gather data. 'Above all, it allows us to detect species that are invisible to the naked eye, especially those that are rare and difficult to find, or those that only come out at night or live in habitats that are hard for humans to reach,' Allegrini adds. 'It's also much faster and cheaper than traditional methods. A diver can only make four trips 50 metres underwater per day, and their visibility is limited to two metres… Imagine if they had to cover the entire Mediterranean basin. How many species would they miss?' the head of Spygen insists. Two years ago, Spygen and OceanoScientific banded together with six other partners to take on a wild dream. Between May and July, the group decided they would make an inventory of every species of fish, crustacean and marine mammal found along the Mediterranean coast. Over the course of four months, they gathered more than 700 eDNA samples. The project was called BioDivMed. 'When we started out, we didn't really know what the outcome would be. No one had ever tried to get that broad of an inventory of Mediterranean species,' Griboval recalls. 'And the results were good news. The biodiversity of the area was much richer than we had expected it to be.' 'We often talk about the Mediterranean as a dying sea, but this project showed that the coastline is healthier than we thought,' Griboval says. A total of 267 fish species were identified. 'It was a phenomenal result. Unprecedented,' adds David Mouillot, a professor at the University of Montpellier who took part in the project. And there were other surprises. Off the coast of Corsica, eDNA identified the presence of several angel sharks, a critically endangered species that was already thought to have disappeared from the area. 'We were able to find the animal's last refuge,' Mouillot says. 'We also found DNA traces of sunfish, which we also thought had disappeared from our waters.' For Mouillot, the discoveries were not only symbolic but vital for conservation efforts of the Mediterranean's biodiversity. 'Detecting rare species has a tendency to get things moving, especially when it comes to protecting a specific area or ecosystem. No one wants to destroy the habitat of the last living specimen of an endangered species,' he says. Better protections, better catch At a time when France, like several other countries, has promised to transform 30 percent of its land and marine ecosystems into protected areas – compiling inventories of living organisms could help identify and monitor priority zones. One hundred sites in the Mediterranean were identified thanks to the BioDivMed project from 2023. Surveys are conducted annually to observe how many species are left in each delineated zone. And every three or four years, a more complete inventory is taken to allow for long-term monitoring. 'That's another advantage of eDNA,' says Mouillot. 'There is a standard protocol that is easy to replicate, so we can conduct serious species monitoring. It allows us to determine whether the protection of a given area is actually working, whether a species has moved somewhere else, or if there is a proliferation of an invasive species that could pose a threat.' 'Inventories based on eDNA also help us inform fishers about available resources,' says Griboval. 'We know when we need to reduce fishing when it puts too much pressure on a given area, and we can help by directing fishers to other locations or even diversify their catch.' Exploring the deep sea Allegrini and his team are now setting their sights on new horizons – the deep sea. While some have their sights set on exploiting the resources that can be found, eDNA could help bolster research on this ecosystem we still know very little about. For some time now, Spygen has been developing an underwater drone that could collect data from these unexplored depths. But there is a limit to their ambitions. Organisms can only be identified if their genetic sequencing has already been catalogued in the past. In other words, unknown species would not be able to be identified – there would be no match in existing databases. 'So we have a parallel challenge of enriching the existing databases,' says Mouillot. 'The good news is that the DNA we collect can be saved, even if it's unknown for now. We can always identify it later when our inventories grow.' Another major challenge is "developing our identification process', says Allegrini. 'We can find out what species an organism is thanks to a fragment of its DNA. But we need to be able to identify what family or even what individual it belongs to.' 'It would be a revolution for species monitoring, especially for those at risk of becoming extinct. We could track them with precision, understand their behaviour, their movements…' Allegrini adds. For now, however, Allegrini admits that these ambitions feel a little more like science fiction than reality. But the president of Spygen has a tendency to think big, especially when it comes to observing something small.
France 24
21 hours ago
- France 24
Scientists on the front line in climate change ocean research
France 12:05 Issued on: Modified: From the show Ahead of the United Nations Oceans Conference in Nice, our reporters went to meet some of the French scientists working to better understand and preserve the ocean. From high-tech robots 6,000m below sea level to critical mineral exploration in the Pacific, the team takes a look at some of the initiatives across the country. They also explore how cuts to climate change projects in the US are having an impact in France.
Euronews
a day ago
- Euronews
veterans gather in Normandy to mark 81st anniversary of D-Day landing
D-Day veterans gathered in Normandy on Friday to mark the 81st anniversary of the pivotal military landing in World War II. Along the coastline and near the D-Day landing beaches, tens of thousands of onlookers attended the commemorations, which included parachute jumps, flyovers, remembrance ceremonies, parades, and historical reenactments. Many were there to cheer the ever-dwindling number of surviving veterans in their late 90s and older. All remembered the thousands who died. US Defence Secretary Pete Hegseth commemorated the anniversary of the D-Day landings, in which US soldiers played a major role, with veterans at the American Cemetery overlooking the shore in the village of Colleville-sur-Mer. On 6 June 1944, the D-Day invasion of Nazi-occupied France represented the largest-ever armada of ships, troops, planes and vehicles to breach Nazi leader Adolf Hitler's defences in western Europe. A total of 4,414 Allied troops were killed on D-Day itself. In the ensuing Battle of Normandy, 73,000 Allied forces were killed and 153,000 wounded. The battle — and especially Allied bombings of French villages and cities — killed around 20,000 French civilians between June and August 1944. The exact German casualties are unknown, but historians estimate between 4,000 and 9,000 men were killed, wounded or missing during the D-Day invasion alone. 'The heroism, honour and sacrifice of the Allied forces on D-Day will always resonate with the US Armed Forces and our Allies and partners across Europe,' said Lt Gen Jason T Hinds, deputy commander of US Air Forces in Europe – Air Forces Africa. 'So let us remember those who flew and fell." "Let us honour those who survived and came home to build a better world. And let us ensure that their sacrifice was not in vain by meeting today's challenges with the same resolve, the same clarity of purpose, and the same commitment to freedom.' Nearly 160,000 Allied troops landed on D-Day. Of 160,000 troops landing in Normandy on D-Day, 73,000 were from the US and 83,000 from the UK and Canada. Forces from several other countries took part in the fighting, including French troops under General Charles de Gaulle. The Allies faced around 50,000 German forces. More than 2 million Allied soldiers, sailors, pilots, medics and other people from a dozen countries were involved in the overall Operation Overlord, the battle to wrest western France from Nazi control that started on D-Day. A jury in Malta found two men guilty on Friday of complicity in the murder of investigative journalist Daphne Caruana Galizia, after a six-week-long trial covering two homicides ended late on Thursday. Jamie Vella and Robert Agius were found guilty of supplying the bomb that killed her. Caruana Galizia was murdered on 16 October 2017 by a car bomb that was detonated while she was driving near her home. In her career, she had written extensively about suspected corruption in political and business circles in Malta, and her murder shocked Europe and triggered angry protests in the Mediterranean island country. Caruana Galizia's investigative reports had targeted people in then-Prime Minister Joseph Muscat's inner circle whom she accused of having offshore companies in tax havens disclosed in the Panama Papers leak. She also targeted the opposition and at the time of her death was facing more than 40 libel suits. The Caruana Galizia family said in a statement that Thursday's verdict brings them a step closer to justice. "Yet, eight years after Daphne's brutal assassination, the institutional failures that enabled her murder remain unaddressed and unreformed," the family added. Vella and Robert Agius, together with two other men – George Degiorgio and Adrian Agius – also faced charges related to the separate murder of a lawyer, Carmel Chircop, who was shot and killed in 2015. Vella, Degiorgio and Adrian Agius were found guilty of charges tied to the murder, while Robert Agius was found not guilty. The judge will decide on sentencing at a later date. George Degiorgio and his brother Alfred Degiorgio both pleaded guilty in 2022 to carrying out the murder of Caruana Galizia and were each sentenced to 40 years in prison. A third man, Vincent Muscat, pleaded guilty in 2021 for his role in the Caruana Galizia murder and was sentenced to 15 years. He testified in the recent jury trial after being granted a presidential pardon for his role in the Chircop murder on the condition that he tell the whole truth. Yorgen Fenech, a prominent Maltese businessman, is currently out of jail on bail awaiting trial on charges of alleged complicity in Caruana Galizia's murder.
