
Encountered a problematic response from an AI model? More standards and tests are needed, say researchers
The emergence of these undesirable behaviors is compounded by a lack of regulations and insufficient testing of AI models, researchers told CNBC.
Getting machine learning models to behave the way it was intended to do so is also a tall order, said Javier Rando, a researcher in AI.
"The answer, after almost 15 years of research, is, no, we don't know how to do this, and it doesn't look like we are getting better," Rando, who focuses on adversarial machine learning, told CNBC.
However, there are some ways to evaluate risks in AI, such as red teaming. The practice involves individuals testing and probing artificial intelligence systems to uncover and identify any potential harm — a modus operandi common in cybersecurity circles.
Shayne Longpre, a researcher in AI and policy and lead of the Data Provenance Initiative, noted that there are currently insufficient people working in red teams.
While AI startups are now using first-party evaluators or contracted second parties to test their models, opening the testing to third parties such as normal users, journalists, researchers, and ethical hackers would lead to a more robust evaluation, according to a paper published by Longpre and researchers.
"Some of the flaws in the systems that people were finding required lawyers, medical doctors to actually vet, actual scientists who are specialized subject matter experts to figure out if this was a flaw or not, because the common person probably couldn't or wouldn't have sufficient expertise," Longpre said.
Adopting standardized 'AI flaw' reports, incentives and ways to disseminate information on these 'flaws' in AI systems are some of the recommendations put forth in the paper.
With this practice having been successfully adopted in other sectors such as software security, "we need that in AI now," Longpre added.
Marrying this user-centred practice with governance, policy and other tools would ensure a better understanding of the risks posed by AI tools and users, said Rando.
Project Moonshot is one such approach, combining technical solutions with policy mechanisms. Launched by Singapore's Infocomm Media Development Authority, Project Moonshot is a large language model evaluation toolkit developed with industry players such as IBM and Boston-based DataRobot.
The toolkit integrates benchmarking, red teaming and testing baselines. There is also an evaluation mechanism which allows AI startups to ensure that their models can be trusted and do no harm to users, Anup Kumar, head of client engineering for data and AI at IBM Asia Pacific, told CNBC.
Evaluation is a continuous process that should be done both prior to and following the deployment of models, said Kumar, who noted that the response to the toolkit has been mixed.
"A lot of startups took this as a platform because it was open source, and they started leveraging that. But I think, you know, we can do a lot more."
Moving forward, Project Moonshot aims to include customization for specific industry use cases and enable multilingual and multicultural red teaming.
Pierre Alquier, Professor of Statistics at the ESSEC Business School, Asia-Pacific, said that tech companies are currently rushing to release their latest AI models without proper evaluation.
"When a pharmaceutical company designs a new drug, they need months of tests and very serious proof that it is useful and not harmful before they get approved by the government," he noted, adding that a similar process is in place in the aviation sector.
AI models need to meet a strict set of conditions before they are approved, Alquier added. A shift away from broad AI tools to developing ones that are designed for more specific tasks would make it easier to anticipate and control their misuse, said Alquier.
"LLMs can do too many things, but they are not targeted at tasks that are specific enough," he said. As a result, "the number of possible misuses is too big for the developers to anticipate all of them."
Such broad models make defining what counts as safe and secure difficult, according to a research that Rando was involved in.
Tech companies should therefore avoid overclaiming that "their defenses are better than they are," said Rando.
Hashtags

Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles


CNBC
4 minutes ago
- CNBC
CNBC TechCheck Evening Edition: July 25, 2025
CNBC's TechCheck brings you the latest in tech news from CNBC's 1 Market in the heart of San Francisco.
Yahoo
30 minutes ago
- Yahoo
Spear AI raises first round of funding to apply AI to submarine data
By Stephen Nellis SAN FRANCISCO (Reuters) -A startup founded by U.S. Navy veterans aiming to help the U.S. military use artificial intelligence to decipher data gathered by submarines has raised its first round of outside capital. Washington-based Spear AI specializes in working with what is known as passive acoustic data, which is gathered by listening devices underwater. Its long-term aim is to use AI to help submarine operators understand whether an object heard could be a rain squall, a whale, or a vessel that could be a threat, and to detect where it is and how fast it is moving. The challenge is that most existing AI tools are trained on data such as words or images that have been painstakingly labeled and organized over years or decades by companies such as Scale AI, which recently signed a $14.8-billion deal with Meta Platforms. Data from acoustic sensors is different. Spear AI co-founders Michael Hunter, a former U.S. Navy SEAL analyst, and John McGunnigle, a former nuclear submarine commander for the U.S. Navy, are building a hardware and software platform that aims to prepare that data for AI algorithms. The company sells sensors that can be attached to buoys or vessels and a software tool to help label and sort the data gathered by the sensors to make it ready to be put into AI systems. The U.S. Navy this month awarded Spear AI a $6-million contract for its data-labeling tool. Spear AI, founded in 2021, has been self-funded and has about 40 employees. Hunter, the CEO, said it raised $2.3 million from AI-focused venture firm Cortical Ventures and private equity firm Scare the Bear. The funding will be used to double the company's headcount to support its government contracts and commercial business prospects, such as monitoring underwater pipelines and cables. Hunter said Spear AI also aims to sell consulting services, a model similar to defense tech firm Palantir. "We wanted to build the product and actually get it out the door before the contract came in to get it," Hunter told Reuters. "The only way you can do that is with private capital." Error in retrieving data Sign in to access your portfolio Error in retrieving data Error in retrieving data Error in retrieving data Error in retrieving data


New York Post
33 minutes ago
- New York Post
‘Possibly hostile' alien threat detected in unknown interstellar object, a shocking new study claims
A mysterious intergalactic object could potentially be a 'hostile' alien spacecraft that's slated to attack our planet in November, according to a controversial new study by a small group of scientists. 'The consequences, should the hypothesis turn out to be correct, could potentially be dire for humanity,' the researchers wrote in the inflammatory paper, which was published July 16 to the preprint server arXiv, South West News Service reported. 3 Comet 3I/ATLAS streaks across a dense star field in this image captured by the Gemini North telescope's Gemini Multi-Object Spectrograph, July 2025. NSF NOIRLab/ Ob et al. / SWNS Advertisement Dubbed 3I/ATLAS, the interstellar entity was discovered on July 1, rocketing toward the sun at more than 130,000 mph, Live Science reported. Less than 24 hours later, it was confirmed to be an interstellar object with initial observations suggesting that it could be a comet that measures up to 15 miles in diameter — larger than Manhattan. However, in the new paper, the trio of researchers suggested that it might be a piece of extraterrestrial spy technology in disguise. One of the researchers, Avi Loeb — a prominent Harvard astrophysicist known for linking extraterrestrial objects to alien life — previously made waves after floating the theory that 2017 interstellar object ʻOumuamua could be an artificial recon probe sent by an alien civilization, based on its odd shape and acceleration. Advertisement In this study, which he collaborated on with Adam Hibberd and Adam Crowl of the Initiative for Interstellar Studies in London, Loeb postulated that 3I/ATLAS's trajectory suggests a similarly alien origin. The trio felt the object's speed — which was significantly faster than ʻOumuamua and other objects — and the fact that it entered our solar system from a different angle than its predecessors offer 'various benefits to an extraterrestrial intelligence,' Loeb wrote in a blog post. 3 'The consequences, should the hypothesis turn out to be correct, could potentially be dire for humanity,' the researchers wrote in the inflammatory paper. ESA/Hubble/NASA/ESO/ / SWNS One benefit is that 3I/ATLAS will make close approaches to Jupiter, Mars and Venus, which could allow aliens to stealthily plant spy 'gadgets' there, Loeb wrote. Advertisement When the so-called undercover UFO reaches its closest to the Sun (perihelion) in late November, it will be concealed from Earth's view. 'This could be intentional to avoid detailed observations from Earth-based telescopes when the object is brightest or when gadgets are sent to Earth from that hidden vantage point,' Loeb declared. If this anomaly is a 'technological artifact,' this could support the dark forest hypothesis, which argues we haven't found signs of extraterrestrial entities because they are remaining undercover to shield themselves from predators or prey. Loeb warns that this could suggest that an attack is likely and would 'possibly require defensive measures to be undertaken.' 3 The Deep Random Survey telescope managed to capture images of interstellar object 3I/Atlas (pictured) in July 2025. K Ly/Deep Random Survey / SWNS Advertisement The problem is that 3I/ATLAS is traveling too fast for an Earth-based spacecraft to intercept it before it exits the solar system. 'It is therefore impractical for earthlings to land on 3I/ATLAS at closest approach by boarding chemical rockets, since our best rockets reach at most a third of that speed,' Loeb wrote. However, other scientists have thrown cold water on the so-called alien origins of the object, which they believe is a comet. 'All evidence points to this being an ordinary comet that was ejected from another solar system, just as countless billions of comets have been ejected from our own solar system,' added Samantha Lawler, an astronomer at the University of Regina in Canada who studies solar system dynamics, Live Science reported. In fact, even Loeb admitted in his blog that his alien spy probe theory is a bit far-fetched: 'By far, the most likely outcome will be that 3I/ATLAS is a completely natural interstellar object, probably a comet.' The researchers also warned the public to take the paper, which has not yet been peer-reviewed, with a grain of salt. 'This paper is contingent on a remarkable but, as we shall show, testable hypothesis, to which the authors do not necessarily ascribe, yet is certainly worthy of an analysis and a report,' they wrote. 'The hypothesis is an interesting exercise in its own right, and is fun to pursue, irrespective of its likely validity.' Advertisement However, critics have called their project a mockery of the work of other scientists, who have provided plenty of evidence that 3I/ATLAS is not evidence of a pending close encounter. 'Astronomers all around the world have been thrilled at the arrival of 3I/ATLAS, collaborating to use advanced telescopes to learn about this visitor,' Chris Lintott, an astronomer at the University of Oxford who helped simulate 3I/ATLAS's galactic origins, told Live Science. 'Any suggestion that it's artificial is nonsense on stilts, and is an insult to the exciting work going on to understand this object.'