OpenAI updated its safety framework—but no longer sees mass manipulation and disinformation as a critical risk
The company said it would now address those risks through its terms of service, restricting the use of its AI models in political campaigns and lobbying, and monitoring how people are using the models once they are released for signs of violations.
OpenAI also said it would consider releasing AI models that it judged to be 'high risk' as long as it has taken appropriate steps to reduce those dangers—and would even consider releasing a model that presented what it called 'critical risk' if a rival AI lab had already released a similar model. Previously, OpenAI had said it would not release any AI model that presented more than a 'medium risk.'
The changes in policy were laid out in an update to OpenAI's 'Preparedness Framework' yesterday. That framework details how the company monitors the AI models it is building for potentially catastrophic dangers—everything from the possibility the models will help someone create a biological weapon to their ability to assist hackers to the possibility that the models will self-improve and escape human control.
The policy changes split AI safety and security experts. Several took to social media to commend OpenAI for voluntarily releasing the updated framework, noting improvements such as clearer risk categories and a stronger emphasis on emerging threats like autonomous replication and safeguard evasion.
However, others voiced concerns, including Steven Adler, a former OpenAI safety researcher who criticized the fact that the updated framework no longer requires safety tests of fine-tuned models. 'OpenAI is quietly reducing its safety commitments,' he wrote on X. Still, he emphasized that he appreciated OpenAI's efforts: 'I'm overall happy to see the Preparedness Framework updated,' he said. 'This was likely a lot of work, and wasn't strictly required.'
Some critics highlighted the removal of persuasion from the dangers the Preparedness Framework addresses.
'OpenAI appears to be shifting its approach,' said Shyam Krishna, a research leader in AI policy and governance at RAND Europe. 'Instead of treating persuasion as a core risk category, it may now be addressed either as a higher-level societal and regulatory issue or integrated into OpenAI's existing guidelines on model development and usage restrictions.' It remains to be seen how this will play out in areas like politics, he added, where AI's persuasive capabilities are 'still a contested issue.'
Courtney Radsch, a senior fellow at Brookings, the Center for International Governance Innovation, and the Center for Democracy and Technology working on AI ethics went further, calling the framework in a message to Fortune 'another example of the technology sector's hubris." She emphasized that the decision to downgrade 'persuasion' 'ignores context – for example, persuasion may be existentially dangerous to individuals such as children or those with low AI literacy or in authoritarian states and societies.'
Oren Etzioni, former CEO of the Allen Institute for AI and founder of TrueMedia, which offers tools to fight AI-manipulated content, also expressed concern. 'Downgrading deception strikes me as a mistake given the increasing persuasive power of LLMs,' he said in an email. 'One has to wonder whether OpenAI is simply focused on chasing revenues with minimal regard for societal impact.'
However, one AI safety researcher not affiliated with OpenAI told Fortune that it seems reasonable to simply address any risks from disinformation or other malicious persuasion uses through OpenAI's terms of service. The researcher, who asked to remain anonymous because he is not permitted to speak publicly without authorization from his current employer, added that persuasion/manipulation risk is difficult to evaluate in pre-deployment testing. In addition, he pointed out that this category of risk is more amorphous and ambivalent compared to other critical risks, such as the risk AI will help someone perpetrate a chemical or biological weapons attack or will help someone in a cyberattack.
It is notable that some Members of the European Parliament have also voiced concern that the latest draft of the proposed code of practice for complying with the EU AI Act also downgraded mandatory testing of AI models for the possibility that they could spread disinformation and undermine democracy to a voluntary consideration.
Studies have found AI chatbots to be highly persuasive, although this capability itself is not necessarily dangerous. Researchers at Cornell University and MIT, for instance, found that dialogues with chatbots were effective at getting people question conspiracy theories.
Another criticism of OpenAI's updated framework centered on a line where OpenAI states: 'If another frontier AI developer releases a high-risk system without comparable safeguards, we may adjust our requirements.'
'They're basically signaling that none of what they say about AI safety is carved in stone,' said longtime OpenAI critic Gary Marcus in a LinkedIn message, who said the line forewarns a race to the bottom. 'What really governs their decisions is competitive pressure—not safety. Little by little, they've been eroding everything they once promised. And with their proposed new social media platform, they're signaling a shift toward becoming a for-profit surveillance company selling private data—rather than a nonprofit focused on benefiting humanity.'
Overall, it is useful that companies like OpenAI are sharing their thinking around their risk management practices openly, Miranda Bogen, director of the AI governance lab at the Center for Democracy & Technology, told Fortune in an email.
That said, she added she is concerned about moving the goalposts. 'It would be a troubling trend if, just as AI systems seem to be inching up on particular risks, those risks themselves get deprioritized within the guidelines companies are setting for themselves,' she said.
She also criticized the framework's focus on 'frontier' models when OpenAI and other companies have used technical definitions of that term as an excuse to not publish safety evaluations of recent, powerful models.(For example, OpenAI released its 4.1 model yesterday without a safety report, saying that it was not a frontier model). In other cases, companies have either failed to publish safety reports or been slow to do so, publishing them months after the model has been released.
'Between these sorts of issues and an emerging pattern among AI developers where new models are being launched well before or entirely without the documentation that companies themselves promised to release, it's clear that voluntary commitments only go so far,' she said.
This story was originally featured on Fortune.com
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Wall Street Journal
22 minutes ago
- Wall Street Journal
Scrapping AI Export Controls Is Self-Defeating
Aaron Ginn's op-ed 'China's and America's Self-Defeating AI Strategy' (Aug. 6) mischaracterizes the purpose and effectiveness of export controls. The policy was never intended as a brick wall but as a strategic speed bump—one essential tool among many for maintaining America's lead in artificial intelligence while limiting China's military capabilities. The controls on Nvidia's H20 chips appear to have been working until CEO Jensen Huang's lobbying secured a reversal that handed Beijing exactly what it wanted. DeepSeek's founder admitted that the chip controls were his company's biggest constraint. As AI's compute demands soar, export controls allow America's hardware advantage to deliver compounding benefits. Reversing course cedes those gains to China.
Forbes
22 minutes ago
- Forbes
Mark Zuckerberg And Personal Superintelligence
When the leaders of large American tech companies take the time to pen a notable broadside on AI, people tend to pay attention. Case in point, this week, 'the Zuck' himself has a new essay talking about the future of artificial intelligence, in which he calls the improvement in tech 'slow, but undeniable,' saying superintelligence is 'now in sight.' 'It seems clear that in the coming years, AI will improve all our existing systems and enable the creation and discovery of new things that aren't imaginable today,' Zuckerberg writes. 'But it is an open question what we will direct superintelligence towards.' I gad to read that last sentence a couple of times, because of its structure. Indeed, this is one of the central open questions, as we are soundly in uncharted waters. New and Old Shifts Contrasting AI with what came before, Zuckerberg makes the point that 200 years ago practically everybody was engaged in one pursuit: agriculture. '90% of people were farmers growing food to survive,' he writes. 'Advances in technology have steadily freed much of humanity to focus less on subsistence, and more on the pursuits we choose.' I would take this a step further and add that, curiously, almost all human societies up until the second half of the twentieth century faced challenges regarding getting enough calories to citizens, while in today's American society, amid seemingly infinite mountains of cheap food, the focus is more on keeping calorie counts down. Anyway, Zuckerberg points to a future where people continue to spend more time on things like creativity and building cultures, fostering relationships, and enjoying a better quality of life. 'I am extremely optimistic that superintelligence will help humanity accelerate our pace of progress,' he writes. AI and Personal Empowerment Zuckerberg's next point is around freedom, and enabling people to have greater agency. It struck me that a major part of this will be data ownership, and we can use that as a litmus test to see if industry leaders are actually serious about personal empowerment. Will we have systems that reward content creators and others for their creative work, or centralized companies either pirating IP outright, or getting around data ownership issues? 'As profound as the abundance produced by AI may one day be, an even more meaningful impact on our lives will likely come from everyone having a personal superintelligence that helps you achieve your goals, create what you want to see in the world, experience any adventure, be a better friend to those you care about, and grow to become the person you aspire to be,' Zuckerberg adds. Many of these goals are rather vague, but I was interested in the one about being a better friend, and thinking about how AI would apply to that. Is he talking about remembering someone's birthday, and sending a card? Thoughts on Job Displacement You don't have to look too far on the Internet to see the majority of various survey respondents saying that they're worried about job displacement. After all, AI can do many things better than people. That's not constrained to simple data crunching - it encompasses a wide spectrum of both physical and knowledge-based tasks. Zuckerberg again emphasizes that he wants the company to be human-centered, but he contrasts this with another industry view that he implies carries weight in the general community. 'This (view toward personal enablement) is distinct from others in the industry who believe superintelligence should be directed centrally towards automating all valuable work, and then humanity will live on a dole of its output,' he writes. 'At Meta, we believe that people pursuing their individual aspirations is how we have always made progress expanding prosperity, science, health, and culture. This will be increasingly important in the future as well.' Correspondingly, Zuck also suggests that in the new AI era, people will spend more time 'creating and connecting,' and presumably, less time working for the Yankee dollar. A New Interface I found another part of Zuckerberg's essay compelling, and rather predictive. We've known for a while now that our smartphones are not interfaces designed for AI. They were designed decades before AI really came to the forefront. So we're going to need new kinds of interfaces. Zuckerberg enumerates one of these very specifically, and in my view, this is one of the biggest contributions of this essay. 'Personal devices like glasses that understand our context because they can see what we see, hear what we hear, and interact with us throughout the day will become our primary computing devices,' he writes. This is important – just think about the difference between a smartphone that sits in your pocket, and a pair of glasses that automatically imbibes all of the information from your field of vision, every moment of every day. That massive data is going to unlock all kinds of interactions between you and your glasses that will make smartphones look like antiquated bricks. We just haven't gotten there yet. Some Amount of Idealism 'We believe the benefits of superintelligence should be shared with the world as broadly as possible,' Zukerberg writes, also conceding that we will see 'novel safety concerns' pop up. 'We'll need to be rigorous about mitigating these risks and careful about what we choose to open source,' he adds. That adds to the ongoing debate around open source versus closed source models: on the one hand, people want openness to keep the playing field level – on the other hand, closed source projects help to prevent some kinds of liability from powerful tech falling into the wrong hands. 'The rest of this decade seems likely to be the decisive period for determining the path this technology will take, and whether superintelligence will be a tool for personal empowerment or a force focused on replacing large swaths of society,' Zuckerberg writes. And then there's this: 'Meta believes strongly in building personal superintelligence that empowers everyone. We have the resources and the expertise to build the massive infrastructure required, and the capability and will to deliver new technology to billions of people across our products. I'm excited to focus Meta's efforts towards building this future.' In this essay, you get definite echoes of past screeds by Sam Altman, which I've reported on quite a bit. You also get some correlation to the kinds of principles that our conference presenters talk about when they analyze the difference between autonomous people and centralized corporate systems. As pointed out in his repeat appearances at annual events, we need to promote the idea that individual humans will have the rights to their own data. There will also need to be effective regulations and checks on centralized power and control of the AI systems. So, with this in mind, many of us would think that it's a good sign that people like Zuckerberg are thinking about these issues, and expressing their willingness to work on progress. As I've said so many times we're going to see a lot in the second half of this year. It's not even fall yet and as Zuckerberg mentioned, we're marching quickly towards super intelligence, AGI, the singularity – call it whatever you want it. It's looming larger on the horizon.
Forbes
22 minutes ago
- Forbes
Nvidia Stands To Grow Taller In The Tech World
In some ways, this is a strange market. It's not like the tech industry of the past – where we had linear progress on hardware, and a collection of hardware companies in vibrant competition. Now, there's one big standout, evidenced by some of the numbers getting thrown around this week after two of the other biggest tech companies, Microsoft and Meta, did earnings. Under the aegis of Jensen Huang and company, Nvidia has spiraled up into a towering monolith, becoming the biggest publicly traded tech company in the American market. Just a few years ago, it was just one of the chip makers powering laptops and other devices. But that was before AI turned everything on its head. With a stock increase of over 75% over the past year, Nvidia has reached the $4 trillion dollar mark in terms of market cap. That's staggering, but recent projections are even bigger – the common consensus is that the company will add another 20% to that, reaching $5 trillion, in another year. Market Forces Conventional wisdom holds that there are two keys to Nvidia's future growth: robust domestic demand for hardware, and access to the Chinese market. Nvidia recently made inroads in that second category, as the Trump administration has lifted export controls on H20s. The other prong of this gets a shot in the arm from publicly available data about Microsoft and Meta's AI planning. First, Meta has raised its 2025 capital expenditure (capex) forecast to around $66–$72 billion with plans for multi-gigawatt data centers, custom in-house compute systems, and NVIDIA GPU clusters. Microsoft is projecting over $30 billion in AI-focused capex in a single quarter, putting its full-year 2025 investment at $80 billion or more, and is expected to invest in NVIDIA hardware like Blackwell and H100 accelerators, while expanding global cloud data centers and supporting AI-powered products like Microsoft Copilot. It's strange for one (or two) company's investment news to make another company's stock jump, but that's where we're at, with the singular forces in play in the chip market. Nvidia went all in on AI hardware – and it paid off handsomely. Prometheus and Hyperion I was intrigued by a little aside in a tech column on this news, about how Zuckerberg talks about Meta's 'Prometheus/Hyperion' plans. Are these more GPU names? Well, it turns out that Prometheus and Hyperion refer to Meta's massive-scale AI datacenter and compute infrastructure projects, respectively, where Meta's Prometheus multi-GPU clusters will feature high-performance AI training and tens of thousands of GPUs in large-scale mesh systems, and the Hyperion AI data center design will have architecture optimized for AI workloads, power-hungry training jobs, and future in-house silicon tech called Meta Training and Inference Accelerator — a custom, in-house silicon chip family developed to optimize AI model inference and training workloads, especially for large-scale models like LLaMA and Meta AI assistants. The Promise of Fire and Other Mythology of Tech My interest was piqued by the Prometheus reference (and the Hyperion reference too, really) – I remember early screeds on AI comparing it to fire brought by the gods. So I did a little research. ChatGPT presented some interesting correlations to mythology, which I'll share in two pieces, first, for Prometheus: And then for Hyperion: One interesting aspect is Prometheus' punishment by Zeus, related to current needs for AI regulation. The entries for Hyperion are a little weirder (overseer of skies and cycles – echoes Meta's reality labs etc.) but I guess it fits. Here's one more in-depth comparison I was given that further mythologizes other contenders like Google and Amazon: People have been saying we need more of the humanities in AI, and this seems like a neat use case. Anyway, look for that Nvidia stock to continue its climb, and for GPUs to proliferate in an age where we are finding out more and more about the power of AI.
