
Encountered a problematic response from an AI model? More standards and tests are needed, say researchers
The emergence of these undesirable behaviors is compounded by a lack of regulations and insufficient testing of AI models, researchers told CNBC.
Getting machine learning models to behave the way it was intended to do so is also a tall order, said Javier Rando, a researcher in AI.
"The answer, after almost 15 years of research, is, no, we don't know how to do this, and it doesn't look like we are getting better," Rando, who focuses on adversarial machine learning, told CNBC.
However, there are some ways to evaluate risks in AI, such as red teaming. The practice involves individuals testing and probing artificial intelligence systems to uncover and identify any potential harm — a modus operandi common in cybersecurity circles.
Shayne Longpre, a researcher in AI and policy and lead of the Data Provenance Initiative, noted that there are currently insufficient people working in red teams.
While AI startups are now using first-party evaluators or contracted second parties to test their models, opening the testing to third parties such as normal users, journalists, researchers, and ethical hackers would lead to a more robust evaluation, according to a paper published by Longpre and researchers.
"Some of the flaws in the systems that people were finding required lawyers, medical doctors to actually vet, actual scientists who are specialized subject matter experts to figure out if this was a flaw or not, because the common person probably couldn't or wouldn't have sufficient expertise," Longpre said.
Adopting standardized 'AI flaw' reports, incentives and ways to disseminate information on these 'flaws' in AI systems are some of the recommendations put forth in the paper.
With this practice having been successfully adopted in other sectors such as software security, "we need that in AI now," Longpre added.
Marrying this user-centred practice with governance, policy and other tools would ensure a better understanding of the risks posed by AI tools and users, said Rando.
Project Moonshot is one such approach, combining technical solutions with policy mechanisms. Launched by Singapore's Infocomm Media Development Authority, Project Moonshot is a large language model evaluation toolkit developed with industry players such as IBM and Boston-based DataRobot.
The toolkit integrates benchmarking, red teaming and testing baselines. There is also an evaluation mechanism which allows AI startups to ensure that their models can be trusted and do no harm to users, Anup Kumar, head of client engineering for data and AI at IBM Asia Pacific, told CNBC.
Evaluation is a continuous process that should be done both prior to and following the deployment of models, said Kumar, who noted that the response to the toolkit has been mixed.
"A lot of startups took this as a platform because it was open source, and they started leveraging that. But I think, you know, we can do a lot more."
Moving forward, Project Moonshot aims to include customization for specific industry use cases and enable multilingual and multicultural red teaming.
Pierre Alquier, Professor of Statistics at the ESSEC Business School, Asia-Pacific, said that tech companies are currently rushing to release their latest AI models without proper evaluation.
"When a pharmaceutical company designs a new drug, they need months of tests and very serious proof that it is useful and not harmful before they get approved by the government," he noted, adding that a similar process is in place in the aviation sector.
AI models need to meet a strict set of conditions before they are approved, Alquier added. A shift away from broad AI tools to developing ones that are designed for more specific tasks would make it easier to anticipate and control their misuse, said Alquier.
"LLMs can do too many things, but they are not targeted at tasks that are specific enough," he said. As a result, "the number of possible misuses is too big for the developers to anticipate all of them."
Such broad models make defining what counts as safe and secure difficult, according to a research that Rando was involved in.
Tech companies should therefore avoid overclaiming that "their defenses are better than they are," said Rando.
Hashtags

Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles


Axios
2 hours ago
- Axios
Exclusive: Embedded tax startup April raises $38M
April, an embedded tax platform, has raised $38 million in a Series B round led by QED Investors, founder Ben Borodach tells Axios exclusively. Why it matters: Embedding tax tools directly into financial apps can improve financial decision-making and boost customer retention. Zoom in: Nyca Partners and Team8 also participated in the Series B round, bringing the total funding April has raised to date to $78 million. How it works: Fintech apps and financial institutions use April's APIs to integrate tax filing and planning directly into their platforms, enabling year-round, real-time tax management. April operates on a SaaS-based model, offering flat-rate pricing to fintech partners, who can choose to mark up services for their end customers. "Our vision is to embed tax in every financial decision," Borodach says. "Taxes should be happening where you're managing your money. They should be happening in real time, and they should be personalized to you." Context: New York-based April operates in a market dominated by legacy tax-preparation giants like Intuit, H&R Block, Thomson Reuters, and Wolters Kluwer. But it recently became the first new company in 15 years to achieve national e-file coverage in all 50 states, Borodach says. The company has also launched a series of new products over the past year, including pro-assisted and pro-led tax filing, quarterly estimate tools for small business owners, and paycheck withholding optimizers. As a result, it is seeing increased demand from wealth management platforms, including integrations with digital advisers catering to mass-affluent clients and an upcoming partnership with a trillion-dollar asset manager. By the numbers: April claims it can reduce the time it takes to prepare and file taxes from the IRS' reported 13‑hour average down to just 22 minutes. The company processed hundreds of thousands of returns through partnerships with over 50 fintech apps and financial institutions this past tax season. It has seen its business grow three times year-to-date and more than seven times over the past 12 months, Borodach says. What's next: The company is preparing to launch advanced tax planning tools around capital gains, retirement planning, and stock transactions.
Yahoo
2 hours ago
- Yahoo
Loop Capital Initiates Coverage on Autodesk, Inc. (ADSK) with ‘Hold' Rating
By earning a spot on Ethisphere's 2025 list of the World's Most Ethical Companies and attracting significant hedge fund interest, Autodesk, Inc. (NASDAQ:ADSK) secures a place on our list of the . A bridge under construction, watched over by a team of experienced engineers. On July 23, Loop Capital started coverage on Autodesk, Inc. (NASDAQ:ADSK) with a 'Hold' rating, setting a $320 price target. The company's share price is currently at $301.10, and this price target implies an upside potential of 6.28%. The analyst attributed the 'Hold' rating to softness in the company's core construction market, marked by high interest rates and rising material costs. However, the analyst remains optimistic about ADSK's long-term potential. Meanwhile, it was reported weeks ago that Autodesk, Inc. (NASDAQ:ADSK) was planning to acquire PTC, a Boston-based rival engineering company. However, those plans have since changed. On July 14, 2025, Bloomberg reported that ADSK is no longer thinking of acquiring PTC. The share price of Autodesk went off on a downward slope following the news of potential acquisition; however, it's gaining upward momentum now. The share price is up 3.06% over the past week. Autodesk, Inc. (NASDAQ:ADSK), a global 3D design, engineering, construction, and entertainment software company, offers cutting-edge tools for everything from infrastructure to animation. It is one of the best ESG stocks. While we acknowledge the potential of ADSK as an investment, we believe certain AI stocks offer greater upside potential and carry less downside risk. If you're looking for an extremely undervalued AI stock that also stands to benefit significantly from Trump-era tariffs and the onshoring trend, see our free report on the best short-term AI stock. READ NEXT: 14 Cheap Transportation Stocks to Buy According to Analysts and Top 10 AI Infrastructure Stocks to Buy Now. Disclosure: None. Error in retrieving data Sign in to access your portfolio Error in retrieving data Error in retrieving data Error in retrieving data Error in retrieving data


Newsweek
6 hours ago
- Newsweek
MrBeast Makes Astronomer Joke After Gwyneth Paltrow Video
Based on facts, either observed and verified firsthand by the reporter, or reported and verified from knowledgeable sources. Newsweek AI is in beta. Translations may contain inaccuracies—please refer to the original content. MrBeast, who was recently named the top content creator in the world by Forbes, has reacted to one of the year's most viral stories, the Astronomer Coldplay "kiss-cam scandal." In a post on X, formerly Twitter, MrBeast responded to the company's recent marketing video, which featured Gwyneth Paltrow. Newsweek has reached out to a representative for MrBeast via email for comment. Why It Matters MrBeast's social media post comes after a split-second clip of Astronomer CEO Andy Byron and his colleague Kristin Cabot was shared on social media and promptly broke the internet. The video, first shared by instaagraace on TikTok, has been viewed over 128 million times as of time of writing. Left, MrBeast speaks onstage during YouTube Brandcast 2025 at David Geffen Hall in New York City on May 14, 2025. Right, Gwyneth Paltrow attends the 2025 Breakthrough Prize Ceremony at Barker Hangar in Santa Monica,... Left, MrBeast speaks onstage during YouTube Brandcast 2025 at David Geffen Hall in New York City on May 14, 2025. Right, Gwyneth Paltrow attends the 2025 Breakthrough Prize Ceremony at Barker Hangar in Santa Monica, California, on April 5, 2025. More Taylor Hill/FilmMagic/for YouTube In the clip, the jumbotron lands on the pair and they promptly sprang apart. Coldplay's lead singer Chris Martin, says: "Either they're having an affair or they're just really shy." The pair were later identified as Byron, CEO of the tech firm Astronomer, and Cabot, the company's head of Human Resources. Both have now resigned. What To Know On July 26, Astronomer shared a light-hearted marketing video which capitalizes on the attention that has been brought to the company since the now infamous Coldplay canoodling. The video features Paltrow, Martin's Oscar-winning ex-wife, as a "very temporary spokesperson." MrBeast, real name Jimmy Donaldson, responded to this on X. In a post that has been viewed over 230,000 times, he wrote, "Can I be CEO." Astronomer's board of directors have announced that they will start a formal search for their new chief executive. Company co-founder Pete DeJoy has taken over as interim CEO. He said on Monday that Astronomer has faced an "unusual and surreal" amount of attention in recent days. Astronomer is a New York-based company that helps companies develop, grow, and analyze products using artificial intelligence. MrBeast, whose YouTube channel boasts over 416 million subscribers, is the owner of multiple companies, including Feastables and MrBeast Burger. He has previously shared posts on social media expressing interest in buying the social media platform X from Elon Musk. In June, as part of their list of Top Content Creators, Forbes reported that MrBeast had earnings of $85 million. Back in 2022, Forbes reported the 27-year-old could become the world's "first YouTuber billionaire," reporting at the time that he had a net worth of $500 million. What People Are Saying An Astronomer spokesperson previously told Newsweek in a statement: "Astronomer is committed to the values and culture that have guided us since our founding. Our leaders are expected to set the standard in both conduct and accountability, and recently, that standard was not met." What's Next The search for Astronomer's new CEO is ongoing.