Anthropic C.E.O.: Don't Let A.I. Companies off the Hook
Picture this: You give a bot notice that you'll shut it down soon, and replace it with a different artificial intelligence system. In the past, you gave it access to your emails. In some of them, you alluded to the fact that you've been having an affair. The bot threatens you, telling you that if the shutdown plans aren't changed, it will forward the emails to your wife.
This scenario isn't fiction. Anthropic's latest A.I. model demonstrated just a few weeks ago that it was capable of this kind of behavior.
Despite some misleading headlines, the model didn't do this in the real world. Its behavior was part of an evaluation where we deliberately put it in an extreme experimental situation to observe its responses and get early warnings about the risks, much like an airplane manufacturer might test a plane's performance in a wind tunnel.
We're not alone in discovering these risks. A recent experimental stress-test of OpenAI's o3 model found that it at times wrote special code to stop itself from being shut down. Google has said that a recent version of its Gemini model is approaching a point where it could help people carry out cyberattacks. And some tests even show that A.I. models are becoming increasingly proficient at the key skills needed to produce biological and other weapons.
None of this diminishes the vast promise of A.I. I've written at length about how it could transform science, medicine, energy, defense and much more. It's already increasing productivity in surprising and exciting ways. It has helped, for example, a pharmaceutical company draft clinical study reports in minutes instead of weeks and has helped patients (including members of my own family) diagnose medical issues that could otherwise have been missed. It could accelerate economic growth to an extent not seen for a century, improving everyone's quality of life. This amazing potential inspires me, our researchers and the businesses we work with every day.
But to fully realize A.I.'s benefits, we need to find and fix the dangers before they find us.
Every time we release a new A.I. system, Anthropic measures and mitigates its risks. We share our models with external research organizations for testing, and we don't release models until we are confident they are safe. We put in place sophisticated defenses against the most serious risks, such as biological weapons. We research not just the models themselves, but also their future effects on the labor market and employment. To show our work in these areas, we publish detailed model evaluations and reports.
Want all of The Times? Subscribe.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Business Insider
13 minutes ago
- Business Insider
An OpenAI exec says she was diagnosed with breast cancer and that ChatGPT has helped her navigate it
Kate Rouch, the chief marketing officer at OpenAI, shared on Friday that she was diagnosed with invasive breast cancer weeks after assuming the role, which she called her "dream job," in December. In a thread posted on X, Rouch said she was sharing her story to help other women, adding, "We can't control what happens to us--but we can choose how we face it. My biggest lesson: no one fights alone." Prior to joining OpenAI as the company's first CMO, Rouch was CMO at Coinbase and, before that, spent over a decade at Meta, including as vice president, global head of brand and product marketing. Rouch said she started treatment right around the Super Bowl in February, when OpenAI aired its first-ever ad, and that she has since gone through 13 rounds of chemotherapy while leading OpenAI's marketing team. She wrote that she is expected to make a full recovery. "It has been the hardest season of life — for me, for my husband, and for our two young children," Rouch said, adding she has been supported by OpenAI "at every step." "Silicon Valley can be brutal and transactional. And yet — I've never felt more held," she said, adding that "people showed up in incredible and unexpected ways." Rouch also said OpenAI's ChatGPT has helped her navigate her diagnosis and treatment, including by explaining cancer in a way that is age-appropriate for her kids, helping her manage the side effects of chemo, and creating custom meditations. "Experiencing our work as a patient has made OpenAI's mission feel more personal and important," she said. Rouch said she was sharing her story to encourage other women to "prioritize their health over the demands of families and jobs." "A routine exam saved my life. It could save yours, too," she said. Business Insider reached out to OpenAI for comment. Kevin Weil, the chief product officer at OpenAI, expressed support for Rouch in a reply to her thread. "We love you @kate_rouch!" he wrote. "Proud of you for telling your story and for being so full of fight."
Wall Street Journal
15 minutes ago
- Wall Street Journal
Landmark House v. NCAA Settlement Approved by Judge, Allowing Colleges to Pay Athletes
A federal judge in California finally approved a $2.6 billion settlement for college athletes that upends a century-old tenet of college sports—the notion that schools cannot pay the athletes that play for them. U.S. District Judge Claudia Wilken on Friday ushered in a new era—a professional era—for college sports by signing off on a plan for the NCAA and the five most prominent sports conferences to settle a class-action lawsuit with current and former college players. The deal will give backpay to some, as well as creating a system in which each Division I school will be able to distribute roughly $20 million a year to their athletes. Schools are poised to begin implementing the new model this fall. The decision has been months in the making, drawn out in its final weeks by the judge's insistence that the NCAA find a way to stop current athletes from losing their roster spots. The settlement would 'enable NCAA schools to share their athletic revenues with Division I college student-athletes for the first time in the history of the NCAA,' Wilken wrote in her 76-page opinion. She added that it was 'expected to open the door for Division I student- athletes to receive, in the aggregate, approximately $1.6 billion dollars in new compensation and benefits per year, with that amount increasing over the next ten years.' Each school that elects to share revenue with athletes will start by distributing more than $20 million in the coming academic year. That amount will reach about $32.9 million per school by 2034-35, the end of the injunctive-relief settlement, Wilken wrote. The settlement brings the biggest changes yet to college sports, which until recently had banned athletes from earning much more than a scholarship, room and board. It comes on the heels of years of upheaval that have included loosened restrictions on off-the-field compensation for players, liberalized transfer rules and blockbuster television deals for schools and the chaotic conference realignment that followed. Yet during all of that time, many college sports leaders had still resisted paying athletes directly from the billions of dollars in revenue they helped generate. Now, that restraint is off. Schools have been readying for months for the settlement effects to land on their athletic departments, most immediately by transforming how they recruit and manage rosters in football and basketball. 'People have been doing a lot of work on a contingent basis to try to create the infrastructure that's envisioned by the settlement,' NCAA President Charlie Baker said ahead of the final approval. 'It'll definitely be rocky and kind of messy coming out of the gate, because big things are that way.' Private equity has already been circling college sports, pledging to inject capital into schools but also to advise them on how to grow their sports business. And athletic departments are openly wrestling over what the ruling means for the future of Olympic sports on campus. Most of these sports do not generate much revenue, but American campuses serve as the primary Olympic training ground for Team USA. The settlement largely immunizes the NCAA against similar claims, a provision the association considered essential as it seeks to move past decades of court battles over payments for players. But it will almost certainly not end litigation over the shape of college sports. It isn't clear whether the money needs to be distributed equitably in accordance with Title IX, the federal statute that requires publicly funded institutions to provide equal opportunities to male and female athletes. Aside from preparing for schools to distribute roughly $20 million a year to athletes, the settlement didn't specify how exactly much should be allocated to each sport. The majority will likely go to football, the financial engine of most athletic departments, as well as men's basketball. Female athletes have raised questions over the payouts they are set to receive and what fair compensation looks like for them going forward. 'This settlement doesn't come close to recognizing the value I lost,' LSU gymnast Livvy Dunne said in an unsuccessful attempt to object to the settlement. There's also the open question of whether athletes getting paid by their institutions are working for them—a distinction that could open up schools to more legal challenges. But even without employee status, the settlement will transform the relationship between players and schools. Write to Louise Radnofsky at Laine Higgins at and Rachel Bachman at
CNN
16 minutes ago
- CNN
Federal judge approves $2.8B settlement, paving way for US colleges to pay athletes millions
A federal judge signed off on arguably the biggest change in the history of college sports on Friday, clearing the way for schools to begin paying their athletes millions of dollars as soon as next month as the multibillion-dollar industry shreds the last vestiges of the amateur model that defined it for more than a century. Nearly five years after Arizona State swimmer Grant House sued the NCAA and its five biggest conferences to lift restrictions on revenue sharing, U.S. Judge Claudia Wilken approved the final proposal that had been hung up on roster limits, just one of many changes ahead amid concerns that thousands of walk-on athletes will lose their chance to play college sports. The sweeping terms of the so-called House settlement include approval for each school to share up to $20.5 million with athletes over the next year and $2.7 billion that will be paid over the next decade to thousands of former players who were barred from that revenue for years. The agreement brings a seismic shift to hundreds of schools that were forced to reckon with the reality that their players are the ones producing the billions in TV and other revenue, mostly through football and basketball, that keep this machine humming. The scope of the changes — some have already begun — is difficult to overstate. The professionalization of college athletics will be seen in the high-stakes and expensive recruitment of stars on their way to the NFL and NBA, and they will be felt by athletes whose schools have decided to pare their programs. The agreement will resonate in nearly every one of the NCAA's 1,100 member schools boasting nearly 500,000 athletes. Wilken's ruling comes 11 years after she dealt the first significant blow to the NCAA ideal of amateurism when she ruled in favor of former UCLA basketball player Ed O'Bannon and others who were seeking a way to earn money from the use of their name, image and likeness (NIL) — a term that is now as common in college sports as 'March Madness' or 'Roll Tide.' It was just four years ago that the NCAA cleared the way for NIL money to start flowing, but the changes coming are even bigger. Wilken granted preliminary approval to the settlement last October. That sent colleges scurrying to determine not only how they were going to afford the payments, but how to regulate an industry that also allows players to cut deals with third parties so long as they are deemed compliant by a newly formed enforcement group that will be run by auditors at Deloitte. The agreement takes a big chunk of oversight away from the NCAA and puts it in the hands of the four biggest conferences. The ACC, Big Ten, Big 12 and SEC hold most of the power and decision-making heft, especially when it comes to the College Football Playoff, which is the most significant financial driver in the industry and is not under the NCAA umbrella like the March Madness tournaments are. The list of winners and losers is long and, in some cases, hard to tease out. A rough guide of winners would include football and basketball stars at the biggest schools, which will devote much of their bankroll to signing and retaining them. For instance, Michigan quarterback Bryce Underwood's NIL deal is reportedly worth between $10.5 million and $12 million. Losers will be the walk-ons and partial scholarship athletes whose spots are gone. One of the adjustments made at Wilken's behest was to give those athletes a chance to return to the schools that cut them in anticipation of the deal going through. Also in limbo are Olympic sports many of those athletes play and that serve as the main pipeline for a U.S. team that has won the most medals at every Olympics since the downfall of the Soviet Union. All this is a price worth paying, according to the attorneys who crafted the settlement and argue they delivered exactly what they were asked for: an attempt to put more money in the pockets of the players whose sweat and toil keep people watching from the start of football season through March Madness and the College World Series in June. What the settlement does not solve is the threat of further litigation. Though this deal brings some uniformity to the rules, states still have separate laws regarding how NIL can be doled out, which could lead to legal challenges. NCAA President Charlie Baker has been consistent in pushing for federal legislation that would put college sports under one rulebook and, if he has his way, provide some form of antitrust protection to prevent the new model from being disrupted again.
