Google released safety risks report of Gemini 2.5 Pro weeks after its release — but an AI governance expert said it was a ‘meager' and ‘worrisome' report
AI governance experts had criticized the company for releasing the model without publishing documentation detailing safety evaluations it had carried out and any risks the model might present, in apparent violation of promises it had made to the U.S. government and at multiple international AI safety gatherings.
A Google spokesperson said in an emailed statement that any suggestion that the company had reneged on its commitments was 'inaccurate.'
The company also said that a more detailed 'technical report' will come later when it makes a final version of the Gemini 2.5 Pro 'model family' fully available to the public.
But the newly published six-page model card has also been faulted by at least one AI governance expert for providing 'meager' information about the safety evaluations of the model.
Kevin Bankston, a senior advisor on AI Governance at the Center for Democracy and Technology, a Washington, D.C.-based think tank, said in a lengthy thread on social media platform X that the late release of the model card and its lack of detail was worrisome.
'This meager documentation for Google's top AI model tells a troubling story of a race to the bottom on AI safety and transparency as companies rush their models to market,' he said.
He said the late release of the model card and its lack key safety evaluation results—for instance, details of "red-teaming" tests to trick the AI model into serving up dangerous outputs like bioweapon instructions—suggested that Google 'hadn't finished its safety testing before releasing its most powerful model' and that 'it still hasn't completed that testing even now.'
Bankston said another possibility is that Google had finished its safety testing but has a new policy that it will not release its evaluation results until the model is released to all Google users. Currently, Google is calling Gemini 2.5 Pro a 'preview,' which can be accessed through its Google AI Studio and Google Labs products, with some limitations on what users can do with it. Google has also said it is making the model widely available to U.S. college students.
The Google spokesperson said the company would release a more complete AI safety report 'once per model family.' Bankson said on X that this might mean Google would no longer release separate evaluation results for fine-tuned versions of its models that it releases, such as those that have been tailored for coding or cybersecurity. This could be dangerous, he noted, because fine-tuned versions of AI models can exhibit behaviors that are markedly different from the 'base model' from which they've been adapted.
Google is not the only AI company seemingly retreating on AI safety. Meta's model card for its newly released Llama 4 AI model is of similar length and detail to the one Google just published for Gemini 2.5 Pro and was also criticized by AI safety experts. OpenAI said it was not releasing a technical safety report for its newly-released GPT-4.1 model because it said that the model was 'not a frontier model,' since the company's 'chain of thought' reasoning models, such as o3 and o4-mini, beat it on many benchmarks. At the same time, OpenAI touted that GPT-4.1 was more capable than its GPT-4o model, whose safety evaluation had shown that model could pose certain risks, although it had said these were below the threshold at which the model would be considered unsafe to release. Whether GPT-4.1 might now exceed those thresholds is unknown, since OpenAI said it does not plan to publish a technical report.
OpenAI did publish a technical safety report for its new o3 and o4-mini models, which were released on Wednesday. But at the same time, earlier this week it updated its 'Preparedness Framework' which describes how the company will evaluate its AI models for critical dangers—everything from helping someone build a biological weapon to the possibility that a model will begin to self-improve and escape human control—and seek to mitigate those risks. The update eliminated 'Persuasion'—a model's ability to manipulate a person into taking a harmful action or convince them to believe in misinformation—as a risk category that the company would assess during it pre-release evaluations. It also changed how the company would make decisions around releasing higher risk models, including saying the company would consider shipping an AI model that posed a 'critical risk' if a competitor had already debuted a similar model.
Those changes divided opinion among AI governance experts, with some praising OpenAI for being transparent about its process and also providing better clarity around its release policies, while others were alarmed at the changes.
This story was originally featured on Fortune.com
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Android Authority
28 minutes ago
- Android Authority
Hot water control is the next Nest feature coming to Google Home
Jimmy Westenberg / Android Authority TL;DR Google is preparing to add hot water controls to the Home app, as confirmed by the Google Nest and Home Chief Product Officer. The feature is in final testing and will roll out in the near future. It will let you control hot water from the app if your thermostat supports it. It follows last week's rollout of temperature scheduling for older Nest thermostats, further reducing reliance on the legacy Nest app. Google has been steadily migrating functions from the old Nest app to the Home app, with full temperature scheduling for older Nest thermostats now rolling out. That update, spotted by Android Authority's Mishaal Rahman last week, was a big deal as it was the last major reason many people still needed the old Nest app. Don't want to miss the best from Android Authority? Set us as a preferred source in Google Search to support us and make sure you never miss our latest exclusive reports, expert analysis, and much more. But the migration isn't quite finished. In a reply to Mishaal on X, Anish Kattukaran, the Chief Product Officer for Google Home and Nest, confirmed that hot water control is in 'final testing' and will be included in 'one of our next few releases.' His remarks came after readers of Mishaal's article commented that hot water control was still missing, and that the new schedule interface felt like a downgrade compared to the old Nest app. Hot water control lets you do things like set a schedule for heating water, quickly trigger a boost, or switch it on and off manually. That's if your thermostat is connected to a compatible hot water system. Until now, that functionality has required the older Nest app. The move fits into Google's broader effort to consolidate everything in the Home app, reducing the need for the legacy Nest app and streamlining controls across its smart home ecosystem. Follow
Yahoo
an hour ago
- Yahoo
Google tweaks Google Play conditions following EU pressure
PARIS (Reuters) -Alphabet unit Google said on Tuesday it will make it easier for app developers to steer customers to channels other than Google after the European Union competition watchdog charged it with breaching EU regulations. In March, Google was hit with two charges of breaching the EU's landmark Digital Markets Act (DMA), which aims to rein in the power of Big Tech. Google has been in the European Commission's crosshairs since early 2024 over whether it restricts app developers from informing users about offers outside its app store Google Play and whether it favours its vertical search services such as Google Flights. Regulators have said Alphabet technically prevents app developers from freely steering consumers to other channels for better offers. They said a service fee charged by the company for facilitating the initial acquisition of a new customer by an app developer via Google Play goes beyond what is justified. In a blog post, Google said that following discussions with the European Commission, developers, and other experts, Google is updating certain terms. "While we still have concerns that these changes could expose Android users to harmful content and make the app experience worse, we're updating our External Offers Program for the EU with revised fees and more options for Android developers, following DMA discussions with the European Commission," EMEA Senior Competition Counsel Clare Kelly said. The company, which has been fined more than 8 billion euros ($9.3 billion) by the EU for various antitrust violations, risks fines of up to 10% of its global annual sales if found guilty of breaching the DMA. ($1 = 0.8580 euros)
Gizmodo
an hour ago
- Gizmodo
Meta's First ‘Real' Smart Glasses Have One Huge Problem
When Google Glass came stumbling onto the scene in 2013, people were taken aback. Not only did Google's smart glasses, with their bordering-on-novel camera, present new and somewhat icky questions about personal privacy, but they also crossed an even more controversial line: they looked really, really dorky. Fast forward more than 10 years into the future, and all of those hangups (at least on the surface) seem to be in the rearview. Take Meta, for example. Sales of Meta's Ray-Ban smart glasses have been surprisingly high, keeping even its waning Quest mixed reality headset afloat. Meta is rumored to be on the precipice of launching its first-ever pair of 'real' smart glasses, which is to say glasses with an actual display in them. And naturally, more than a decade removed from Google Glass, the rhetoric around Meta's glasses (codenamed Hypernova) couldn't be more disparate from smart glasses of yore. People are excited; they're watching; even Apple is rumored to be looking at the field as a potential new frontier. So, this is all money in the bank for Meta, right? Maybe, but I wouldn't count those shiny new metaverse dollars just yet. As promising as Meta's push into smart glasses might be, there's one quirk with Hypernova that I can't seem to get off my mind. It's not the glasses themselves, necessarily, but the rumored 'neural wristband' that reportedly comes with them. Let me backtrack for a moment for those that haven't been keeping up; as I mentioned previously, Meta's next smart glasses will most likely have a display. If they have a screen, then they're going to need a way for people to control things on said display, including 'mini apps,' which are stripped-down versions of apps that can run on glasses and are a capability rumored by Bloomberg. There are a few ways you could go about controlling a pair of smart glasses with apps, but Meta's approach is allegedly a 'neural wristband,' which is a wearable that reads the electrical signals in your arm for finger- and hand-based inputs. This is where the weirdness comes in. On one hand (or wrist, I guess), a wristband that reads the signals sent through your brain and into your hand is objectively kind of awesome. Like, damn, that's some future shit if I've ever seen it. On the other, it's a big liability at best. As useful as a wristband like that may be, asking people to strap on a wearable to use their smart glasses properly is a tall order. My guess is most people paying $800 (the rumored price of Hypernova) for a futuristic pair of smart glasses will want an all-in-one package. It's not a big deal on paper to remember to strap your wristband on, but what if it needs a charge? What if you don't like wearing stuff on your wrist or arm? Or—and this is what I think is arguably the biggest problem—what if you don't like how it looks? One of the biggest problems with Google Glass, as I mentioned before, is that you looked like you were wearing a gadget when you had it on. So much so that anyone who dared slide them onto their face was dubbed a 'glasshole.' It was Google Glass's noticeable camera module and display prism that turned it into a pariah and prevented what could have been a fruitful category of device from growing past an infancy stage. It's not all about looks, obviously—2013 was also a very different time when it came to personal privacy, a concept that has been severely eroded as the years have gone on. But I think there's a reason Meta's very much glasses-presenting Ray-Bans, which do almost everything Google Glass did and more, have caught on, and presentation is a major part of that. But hey, things are definitely different now, and maybe the same can be said for what people are willing to wear on their bodies or not. All I know is that it seems like a lot of what could make Hypernova useful or special comes down to a wearable, and while smart glasses might be a semi-proven category, wristbands are not. Google Glass might be long dead, but just because those mistakes are in the rearview doesn't mean they can't be made again.
