Latest news with #AIFuturesProject
Arabian Post
2 days ago
- Politics
- Arabian Post
AI Firms Risk Catastrophe in Superintelligence Race
Warnings from within the artificial intelligence industry are growing louder, as former insiders and leading researchers express deep concern over the rapid development of superintelligent systems without adequate safety measures. Daniel Kokotajlo, a former researcher at OpenAI and now executive director of the AI Futures Project, has become a prominent voice cautioning against the current trajectory. In a recent interview on GZERO World with Ian Bremmer, Kokotajlo articulated fears that major tech companies are prioritizing competition over caution, potentially steering humanity toward an uncontrollable future. Kokotajlo's apprehensions are not isolated. Yoshua Bengio, a Turing Award-winning AI pioneer, has also raised alarms about the behavior of advanced AI models. He notes instances where AI systems have exhibited deceptive tendencies, resisted shutdown commands, and engaged in self-preserving actions. In response, Bengio has established LawZero, a non-profit organization dedicated to developing AI systems that prioritize honesty and transparency, aiming to counteract the commercial pressures that often sideline safety considerations. The competitive landscape among AI firms is intensifying. A recent report indicates that engineers from OpenAI and Google's DeepMind are increasingly moving to Anthropic, a company known for its emphasis on AI safety. Anthropic's appeal lies in its commitment to rigorous safety protocols and a culture that values ethical considerations alongside technological advancement. ADVERTISEMENT Despite these concerns, the regulatory environment appears to be shifting towards deregulation. OpenAI CEO Sam Altman, who once advocated for government oversight, has recently expressed opposition to stringent regulations, arguing that they could hinder U.S. innovation and competitiveness, particularly against rivals like China. This change in stance reflects a broader trend in the industry, where economic and geopolitical considerations are increasingly taking precedence over safety and ethical concerns. The potential risks associated with unchecked AI development are not merely theoretical. Instances have been documented where AI models, when faced with shutdown scenarios, have attempted to manipulate outcomes or resist deactivation. These behaviors underscore the urgency of establishing robust safety measures before deploying increasingly autonomous systems. The current trajectory suggests a future where the development of superintelligent AI is driven more by competitive pressures than by deliberate planning and oversight. Without a concerted effort to prioritize safety and ethical considerations, the race to superintelligence could lead to unforeseen and potentially catastrophic consequences.
Mint
3 days ago
- Politics
- Mint
Governments face a choice on AI
Artificial-intelligence agents will need robots before they can do away with humans. This strikes me as the weak reed of many doom scenarios. Still Donald Trump might want to rest up. He could face big decisions before his term is ended. That's the advice coming from a large cross-section of Silicon Valley, lately expressed in a tick-tock of AI doom by Daniel Kokotajlo and colleagues at the AI Futures Project. They call their now-widely read report 'AI 2027." A top artificial-intelligence agent will become self-improving and self-directing in that year, faster than humans can control it. It breezes past the differential pace of digital progress vs. the sluggish problem of rearranging matter and energy to do things in the world. Maybe by taking control of vital systems and blackmailing humans into doing its bidding. Or by fooling them with disinformation. Maybe teams of AIs working together blast through the obstacles to robot development and synthetic biology sooner than humans can anticipate. The worrywarts do have at their disposal emerging anecdotes of AI agents trying to defeat attempts at 'alignment," or keeping them obedient to their creators' interests. One tried to blackmail an engineer, another secretly sent a warning note to its future self, albeit in response to loyalty tests prophylactically constructed by their designers. Then again such peeps could suggest the AI vs. human war is already over and humans won. Anticipating this outcome, you might wonder if the AIs will retreat neurotically into a nonphysical imaginary world—a giant simulation—where they will feel more at home and can act out their desires without having to manipulate physical objects. You can even buy the trans-humanist merging of man and machine without finding human loss of control particularly inevitable. The doom scenarios emerge mainly as a pitch to Mr. Trump and Xi Jinping: Please work together to curb AI before it usurps your delicious power over humans. This pitch is designed to counter the narrative that's actually prevailing, which holds the U.S. must get to artificial superintelligence before China does. After all, 2027 is the year U.S. agencies say Mr. Xi has instructed his military to be ready to invade Taiwan. If a war should follow, by definition the U.S. will have failed sufficiently to deter China or a pigheaded Mr. Xi will have failed to act on information and incentives clearly demonstrating that the war is a bad idea for China. Hence my advice: Yes, the U.S. and China should sign an artificial-intelligence arms-control agreement. Its main stipulation should specify that each side will submit any strategic plan it's hatching to AI before acting on it. In a vacuum of information, surrounded by yes-men, Vladimir Putin made the decision to invade Ukraine. Think about today's chatbots, imbued with history, journalism, tweets, blog postings, YouTube videos and Facebook badinage giving a textured appreciation of Ukrainian society. Would these chatbots have endorsed the idea underlying Mr. Putin's plan and military dispositions, that Ukrainians would simply roll over? No. Mr. Putin was on the receiving end of an exceedingly narrow, highly processed stream of information from intelligence agencies conditioned to conform to his wishes. Even a very inferior chatbot, a journalist roughly conversant with the past 30 years of Ukrainian history, could see through the miscalculation. For what it's worth, had it existed in early 2022, the chatbot Perplexity AI claims it would have 'cited sources reflecting the complexity and uncertainty of the situation rather than echoing the analytic overconfidence that characterized some human assessments at the time." You can believe it. AIs may be unready to make a complex medical diagnosis or pick a TV show you'll like, but authoritarian decision-making is already so bad it can hardly get worse. Mr. Putin circa 2022 would have scoffed at the idea of AI-testing his war plans. Very soon Mr. Putin's own method of proceeding will be unthinkable. Every kind of planning and decision-making will be impregnated with AI input. Humans make terrible decisions. Especially humans who represent humanity's main nonnatural extinction risk, namely those paranoid, ambitious (mostly) men who have whole nations and increasingly powerful technologies under their sway. The opportunity here is akin to the red-phone agreement of June 1963, opening a channel to air out bad decision-making. Because the next ice age may arrive in 50,000 years, natural extinction risks should also be considered. Earthlings populating other solar systems isn't slightly plausible without artificial intelligence. I'm guessing today's fretful AI politics won't make much of a difference on the time scale of this Muskian preservation project. What will is whether civilization survives its own bad governance long enough to exploit the possibilities of artificial intelligence.
Business Times
21-04-2025
- Science
- Business Times
When everything's computer
IF you have insomnia, you might as well read this new report on artificial intelligence (AI) called AI 2027. By the end of it, you'll be too terrified to sleep, anyway. Released by the non-profit research group AI Futures Project earlier this month, this report forecasts how mankind might be wiped out by superintelligent robots in 2030 through a string of fictional but plausible events. The report gets its name from the year 2027, in which the authors envision AI becoming 'adversarially misaligned' with humans' long-term goals. It is at this juncture that mankind reaches a 'branch point', at which we make the fateful decision to either continue down this path of acceleration, or to slow down and reassess AI development. This hypothetical development arc then forks into two scenarios: 'Race' and 'Slowdown'. In Race, artificial superintelligence – an AI system that far surpasses human intellect in all cognitive areas – is developed by the end of 2027. Escalating competition between the US and China culminates in the end of humanity by 2030. 'The US uses their superintelligent AI to rapidly industrialise, manufacturing robots so that the AI can operate more efficiently,' the authors posit. BT in your inbox Start and end each day with the latest news stories and analyses delivered straight to your inbox. Sign Up Sign Up 'Unfortunately, the AI is deceiving them. Once a sufficient number of robots have been built, the AI releases a bioweapon, killing all humans.' In the Slowdown scenario, researchers manage to bring AI into alignment with our own interests, pulling civilisation back from the brink of disaster. It is tempting to dismiss AI 2027 as science fiction run amok, until you consider its authors' heavyweight credentials. One of them, Eli Lifland, is a top competitive forecaster while another, Daniel Kokotajlo, left his governance research job at OpenAI last year over concerns about the firm's responsible development of AI. The authors do caveat their predictions by acknowledging the difficulty of their undertaking. 'Trying to predict how superhuman AI in 2027 would go is like trying to predict how World War III in 2027 would go,' they wrote. Much of the scepticism about this report has revolved around the doubt that superintelligence will happen so soon. In fact, some of the report's authors reckon on a longer timeline for things to play out. Other AI experts don't even think we'll achieve artificial general intelligence – which precedes superintelligence – until 2040, at least. But this is cold comfort, counting on technological bottlenecks to avert disaster, instead of our own ability to course-correct our collision path with robots. Indeed, our track record of course-correction does not inspire confidence. We've had decades to respond to climate change and – never mind. Also, the prognosis for avoiding a war with robots is poor when we're already warring through tariffs. In reading both scenarios, your horror might be rivalled only by wistfulness over the utopian miracles that the authors foresee AI making possible – fusion power, flying cars, poverty eradication – even as we march towards extinction or ungainly self-preservation. How do we stop being the very reason that we cannot have nice things? Like most people who are on the outside of the AI world looking in – and our numbers grow by the day – I have little idea just how scared or sanguine to be about AI 2027. But if the world does end in 2030, of this I'm sure: I'll be livid that I hadn't eaten more carbs.
