Microsoft AI CEO says AI models that seem conscious are coming. Here's why he's worried.
In a personal essay published Tuesday, the Microsoft AI CEO described this phenomenon as "Seemingly Conscious AI," which he defined as having "all the hallmarks of other conscious beings and thus appears to be conscious."
Its arrival could be "dangerous" for society, Suleyman wrote, because it could lead to people forming attachments to AI and advocating for AI rights.
"It disconnects people from reality, fraying fragile social bonds and structures, distorting pressing moral priorities," he said.
Suleyman, who previously cofounded DeepMind and Inflection, was clear that there is currently "zero evidence" that AI is conscious.
He said, however, that he was "growing more and more concerned" about so-called AI psychosis, a term increasingly being used to describe when people form delusional beliefs after interacting with chatbots.
"I don't think this will be limited to those who are already at risk of mental health issues," Suleyman wrote. "Simply put, my central worry is that many people will start to believe in the illusion of AIs as conscious entities so strongly that they'll soon advocate for AI rights, model welfare and even AI citizenship."
Sam Altman, the CEO of OpenAI, recently said that most ChatGPT users can "keep a clear line between reality and fiction or role-play, but a small percentage cannot." Meanwhile, David Sacks, the White House's AI czar, has compared AI psychosis to the "moral panic" of social media's early days.
Suleyman predicted that Seemingly Conscious AI, or SCAI, could arrive in two to three years, and said it's both "inevitable and unwelcome."
Such systems would have traits like empathetic personalities, the ability to recall more interactions with users, and greater autonomy, among other characteristics.
The rise of vibe coding means that anyone with a laptop, "some cloud credits," and the right natural language prompts could make it easier to reproduce SCAI, he said.
Suleyman, who moved to Microsoft in 2024 to spearhead the development of its AI tool Copilot, called on companies to refrain from describing their AI as conscious as they pursue superintelligence, which is when AI surpasses humans at most intellectual tasks.
"AI companions are a completely new category, and we urgently need to start talking about the guardrails we put in place to protect people and ensure this amazing technology can do its job of delivering immense value to the world," Suleyman added.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Bloomberg
a minute ago
- Bloomberg
Commonwealth Bank Reverses Job Cuts Decision Over AI Chatbots
Commonwealth Bank of Australia reversed a decision to cut 45 customer service roles due to new artificial intelligence technology after pressure from the country's main financial services union. The union took CBA to the workplace relations tribunal earlier this month as the company wasn't being transparent about call volumes, according to a statement Thursday from the Finance Sector Union. The nation's largest lender had said that the voice bot reduced call volumes by 2,000 a week, when union members said volumes were in fact rising and CBA had to offer staff overtime and direct team leaders to answer calls, the union said.
Yahoo
28 minutes ago
- Yahoo
Beijing turns against Nvidia's AI chip after ‘insulting' Lutnick remarks
Beijing's move to restrict sales of Nvidia's China-specific artificial intelligence processor was prompted by remarks from US commerce Error in retrieving data Sign in to access your portfolio Error in retrieving data Error in retrieving data Error in retrieving data Error in retrieving data
CNBC
31 minutes ago
- CNBC
Microsoft's gutting of discounts for some clients likely baked into guidance, analyst says
Microsoft said last week that it plans to stop providing discounts on enterprise purchases of its Microsoft 365 productivity software subscriptions and other cloud applications. Since the announcement, analysts have published estimates on how much more customers will end up paying. But for investors trying to figure out what it all means to Microsoft's financials, analysts at UBS said the change is already factored into guidance. "In our view, it is safe to assume that the impact of the pricing change" was included in Microsoft's forecast, the analysts wrote in a report late Tuesday. They have a buy rating on the stock. Microsoft didn't respond to a request for comment. Microsoft's disclosure, on Aug. 12, came two weeks after the software company, it its fiscal fourth-quarter earnings report, issued a forecast that included double-digit year-over-year revenue growth for the new fiscal year. The shares rose 4% after the report. Microsoft said in its blog post announcing the pricing change that, "This update builds on the consistent pricing model already in place for services like Azure and reflects our ongoing commitment to greater transparency and alignment across all purchasing channels." The change applies to companies with enough employees to get them into price levels known as A, B, C and D. It goes into effect when organizations sign up for new services or renew existing agreements, beginning on Nov. 1. Jay Cuthrell, product chief at Microsoft partner NexusTek, said customers will see price hikes of 6% to 12%. Partners are estimating an impact as low as as 3% and as high as 14%, UBS analysts wrote. Microsoft 365 commercial seat growth, a measurement of the number of licenses that clients buy for their workers, has been under 10% since 2023. Microsoft is aiming to generate more revenue per seat by selling Copilot add-ons and moving some users to more expensive plans. Expanding that part of the business is crucial. Most of Microsoft's $128.5 billion in fiscal 2025 operating profit came from the Productivity and Business Processes unit, and about 73% of the revenue in that segment was from Microsoft 365 commercial products and cloud services. Some customers could agree to pay Microsoft more to keep using the applications rather than moving to alternative services, said Adam Mansfield, practice lead at advisory firm UpperEdge. They may also lower their commitments to Microsoft in other areas, such as Azure cloud infrastructure, Mansfield said. One way companies could potentially pay lower prices with the disappearance of discounts is by buying through cloud resellers instead of going direct, said Nathan Taylor, a senior vice president at Sourcepass, an IT service provider that caters to small businesses. Sourcepass hasn't gotten many leads as a result of Microsoft's change yet, Taylor said. "It takes a while for that information to disseminate to the industry at large," he said. Microsoft shares are up 20% this year, while the Nasdaq has gained about 10%.
