OpenAI updates its system for evaluating AI risks
OpenAI is making several changes to the system it uses to evaluate the risks new models pose, adding new categories for models that could self-replicate or conceal their capabilities.
Why it matters: OpenAI uses its "preparedness framework" to decide whether AI models are safe and what, if any, safeguards are needed during development and for public release.
Driving the news: In another change, OpenAI will no longer specifically evaluate models on their persuasive capabilities — an area where its recent models had already risen to "medium" risk level.
The company is also doing away with distinguishing between "low" and "medium" risk and will focus on deciding whether or not risks reach the "high" or "critical" levels.
In addition to continuing to monitor the risk that AI might be used to create bioweapons or gain a capacity for self-improvement, OpenAI is adding several new "research" categories — such as whether a model can conceal capabilities, evade safeguards or seek to replicate itself or prevent shutdowns.
"We are on the cusp of systems that can do new science, and that are increasingly agentic — systems that will soon have the capability to create meaningful risk of severe harm," OpenAI said in the updated framework. "This means we will need to design and deploy safeguards we can rely on for safety and security."
The changes are the first OpenAI has made to the framework since it was unveiled in December 2023.
What they're saying: In an interview, OpenAI safety researcher Sandhini Agarwal told Axios the changes are designed to shift the company's efforts toward safeguards that protect against the most severe risks.
"The purpose of the framework is to focus on catastrophic risks," she said. "This is not the be-all, end-all of safety at OpenAI."
Between the lines: The new research categories align with broader industry discussion around the prospect that models might act differently in testing than in the real world and that they might try to conceal their capabilities.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Yahoo
23 minutes ago
- Yahoo
LinkedIn cofounder Reid Hoffman admits what you learn during college may not matter—it's this skill that can help Gen Z land entry-level jobs
LinkedIn cofounder Reid Hoffman says what young people learn in college isn't the most important thing in landing a job. Being able to leverage AI tools, tackle new labor market challenges, and leverage connections is more essential for Gen Z seeking employment—and his advice echoes that of Nvidia CEO Jensen Huang. The stable career path of going to college and landing a cushy six-figure office role is being disrupted by AI. Now, LinkedIn cofounder Reid Hoffman admits to rising college graduates that it may not even matter whether you majored in computer science or art history—connections and flexibility are the new hot commodities. 'What you should take forward from your college degree isn't necessarily the thing you learned in X-101,' Hoffman said in a recent video on his YouTube channel. 'It isn't specific degrees, specific courses, [or] even necessarily specific skills that are relevant to you.' Rather, the tech entrepreneur believes that being nimble in today's job market is a massive asset: 'It's your capacity to say, 'Hey, here is the new tool set, here's the new challenge.' That is actually what the future work's going to look like. One thing is to not focus on the degree, but to focus on how you learn and to be continually learning,' Hoffman said. 'The other part of college that's super important, that you should not forget, is that life is a team sport, not just an individual sport,' he continued. 'You can help each other.' Young job-seekers who effectively navigate the new world of work—by leveraging connections, constantly learning, and mastering AI—will have the upper hand, Hoffman concluded. And unfortunately for those saddled with debt, getting a college degree isn't the only way to develop these traits. There's no question that many Gen Zers have already had a rough start in their careers—graduating into a post-COVID way of work, with AI agents being positioned as their new coworkers. Some employers have even branded the generation as lazy and unorganized, but Hoffman thinks Gen Z has one advantage that hiring managers go crazy for. The LinkedIn cofounder said young people are part of 'generation AI': As digital natives who grew up with advanced technology at their fingertips, they are in the best position to leverage that skill. It may be Gen Z's ticket to landing a job. 'Bringing the fact that you have AI in your tool set is one of the things that makes you enormously attractive,' the 57-year-old billionaire said. It's why, despite all the noise around AI threatening to steal entry-level roles, the technology may be Gen Z's best weapon to find work. In the past month, both OpenAI CEO Sam Altman and LinkedIn chief economic opportunity officer Aneesh Raman have waved the warning flag that AI could rival junior employees. Hoffman agreed that AI may make the job search worse for young people—but recommended that Gen Z job searchers use the technology to create their own opportunities. 'AI is changing the [job] landscape, [and] may make entry-level jobs harder to get, may make employers uncertain about who they're looking for and employing,' Hoffman continued. 'Then you say, 'Well, okay, how do I use the current circumstances, the disruption, to make this better? How do I use AI to identify what possible new opportunities might be?'' Gen Z grew up thinking that doing well in college will score you a high-paying role after graduation—but that career trajectory is no longer a promise. Even Dario Amodei, CEO of AI company Anthropic, predicted that AI could eliminate roughly 50% of all entry-level white-collar jobs in the coming years. Instead of burying their heads in the sand, young people can redirect their strategy to be a hot hiring commodity, leaders say. Nvidia CEO Jensen Huang has been particularly outspoken on the issue; he's a huge proponent of the idea that being an AI user is a protective quality in job market disruption. 'Every job will be affected, and immediately. It is unquestionable,' Huang said at the Milken Institute's Global Conference in May. 'You're not going to lose your job to an AI, but you're going to lose your job to someone who uses AI.' This story was originally featured on
The Verge
28 minutes ago
- The Verge
Microsoft has started testing its AI agent in the Windows 11 Settings app.
First announced last month, the feature lets users describe what they need in Settings, like 'my mouse pointer is too small.' The AI agent will then provide suggestions about how to address the issue, and can even fix it for them if users give permission. Microsoft is bringing the AI agent to Windows 11 Insiders in the Dev Channel, but only for Snapdragon-based Copilot Plus PCs to start.
Business Insider
38 minutes ago
- Business Insider
What WWDC tells us about the future of Apple and the iPhone
The more serious question hanging over this year's WWDC was not answered. When will Siri get the AI upgrade it desperately needs? Software chief Craig Federighi delivered the bad news: It's still not ready. That knocked roughly $75 billion off Apple's market value. The stock recovered a bit, but it's still badly lagging behind rivals this month. Google, OpenAI, and other tech companies are launching powerful new AI models and products at a breakneck pace. Apple is running out of time to prove it's a real player in this important field. Analyst Dan Ives is usually bullish, but even he's concerned. "They have a tight window to figure this out," Ives wrote, after calling this year's WWDC a "yawner." AI is complex, expensive, and takes a long time to get right. Apple was late to start building the needed foundational technology, such as data centers, training data pipelines, and homegrown AI chips. By contrast, Google began laying its AI groundwork decades ago. It bought DeepMind in 2014, and this AI lab shapes Google's models in profound ways today. When I was at Google I/O last month, one or two insiders whispered a phrase. They cautiously described an "intelligence gap" that could open up between the iPhone and other smartphones. Many Android phones already feature Google's Gemini chatbot, which is far more capable than Siri. If Apple's AI upgrade takes too long, this intelligence gap could widen so much that some iPhone users might consider switching. At I/O, these insiders only whispered this idea. That's because it will take something pretty dramatic to get people to give up their iPhones. This device has become a utility that we can't live without — even for the few days (weeks?) it might take to get used to an Android replacement.
