logo
Behind the Curtain: The scariest AI reality

Behind the Curtain: The scariest AI reality

Axios4 hours ago

The wildest, scariest, indisputable truth about AI's large language models is that the companies building them don't know exactly why or how they work.
Sit with that for a moment. The most powerful companies, racing to build the most powerful superhuman intelligence capabilities — ones they readily admit occasionally go rogue to make things up, or even threaten their users — don't know why their machines do what they do.
Why it matters: With the companies pouring hundreds of billions of dollars into willing superhuman intelligence into a quick existence, and Washington doing nothing to slow or police them, it seems worth dissecting this Great Unknown.
None of the AI companies dispute this. They marvel at the mystery — and muse about it publicly. They're working feverishly to better understand it. They argue you don't need to fully understand a technology to tame or trust it.
Two years ago, Axios managing editor for tech Scott Rosenberg wrote a story, "AI's scariest mystery," saying it's common knowledge among AI developers that they can't always explain or predict their systems' behavior. And that's more true than ever.
Yet there's no sign that the government or companies or general public will demand any deeper understanding — or scrutiny — of building a technology with capabilities beyond human understanding. They're convinced the race to beat China to the most advanced LLMs warrants the risk of the Great Unknown.
The House, despite knowing so little about AI, tucked language into President Trump's "Big, Beautiful Bill" that would prohibit states and localities from any AI regulations for 10 years. The Senate is considering limitations on the provision.
Neither the AI companies nor Congress understands the power of AI a year from now, much less a decade from now.
The big picture: Our purpose with this column isn't to be alarmist or " doomers." It's to clinically explain why the inner workings of superhuman intelligence models are a black box, even to the technology's creators. We'll also show, in their own words, how CEOs and founders of the largest AI companies all agree it's a black box.
Let's start with a basic overview of how LLMs work, to better explain the Great Unknown:
LLMs — including Open AI's ChatGPT, Anthropic's Claude and Google's Gemini — aren't traditional software systems following clear, human-written instructions, like Microsoft Word. In the case of Word, it does precisely what it's engineered to do.
Instead, LLMs are massive neural networks — like a brain — that ingest massive amounts of information (much of the internet) to learn to generate answers. The engineers know what they're setting in motion, and what data sources they draw on. But the LLM's size — the sheer inhuman number of variables in each choice of "best next word" it makes — means even the experts can't explain exactly why it chooses to say anything in particular.
We asked ChatGPT to explain this (and a human at OpenAI confirmed its accuracy): "We can observe what an LLM outputs, but the process by which it decides on a response is largely opaque. As OpenAI's researchers bluntly put it, 'we have not yet developed human-understandable explanations for why the model generates particular outputs.'"
"In fact," ChatGPT continued, "OpenAI admitted that when they tweaked their model architecture in GPT-4, 'more research is needed' to understand why certain versions started hallucinating more than earlier versions — a surprising, unintended behavior even its creators couldn't fully diagnose."
Anthropic — which just released Claude 4, the latest model of its LLM, with great fanfare — admitted it was unsure why Claude, when given access to fictional emails during safety testing, threatened to blackmail an engineer over a supposed extramarital affair. This was part of responsible safety testing — but Anthropic can't fully explain the irresponsible action.
Again, sit with that: The company doesn't know why its machine went rogue and malicious. And, in truth, the creators don't really know how smart or independent the LLMs could grow. Anthropic even said Claude 4 is powerful enough to pose a greater risk of being used to develop nuclear or chemical weapons.
OpenAI's Sam Altman and others toss around the tame word of " interpretability" to describe the challenge. "We certainly have not solved interpretability," Altman told a summit in Geneva last year. What Altman and others mean is they can't interpret the why: Why are LLMs doing what they're doing?
Anthropic CEO Dario Amodei, in an essay in April called "The Urgency of Interpretability," warned: "People outside the field are often surprised and alarmed to learn that we do not understand how our own AI creations work. They are right to be concerned: this lack of understanding is essentially unprecedented in the history of technology." Amodei called this a serious risk to humanity — yet his company keeps boasting of more powerful models nearing superhuman capabilities.
Anthropic has been studying the interpretability issue for years, and Amodei has been vocal about warning it's important to solve. In a statement for this story, Anthropic said: "Understanding how AI works is an urgent issue to solve. It's core to deploying safe AI models and unlocking [AI's] full potential in accelerating scientific discovery and technological development. We have a dedicated research team focused on solving this issue, and they've made significant strides in moving the industry's understanding of the inner workings of AI forward. It's crucial we understand how AI works before it radically transforms our global economy and everyday lives." (Read a paper Anthropic published last year, "Mapping the Mind of a Large Language Model.")
Elon Musk has warned for years that AI presents a civilizational risk. In other words, he literally thinks it could destroy humanity, and has said as much. Yet Musk is pouring billions into his own LLM called Grok.
"I think AI is a significant existential threat," Musk said in Riyadh, Saudi Arabia, last fall. There's a 10%-20% chance "that it goes bad."
Reality check: Apple published a paper last week, "The Illusion of Thinking," concluding that even the most advanced AI reasoning models don't really "think," and can fail when stress-tested.
The study found that state-of-the-art models (OpenAI's o3-min, DeepSeek R1 and Anthropic's Claude-3.7-Sonnet) still fail to develop generalizable problem-solving capabilities, with accuracy ultimately collapsing to zero "beyond certain complexities."
But a new report by AI researchers, including former OpenAI employees, called " AI 2027," explains how the Great Unknown could, in theory, turn catastrophic in less than two years. The report is long and often too technical for casual readers to fully grasp. It's wholly speculative, though built on current data about how fast the models are improving. It's being widely read inside the AI companies.
It captures the belief — or fear — that LLMs could one day think for themselves and start to act on their own. Our purpose isn't to alarm or sound doomy. Rather, you should know what the people building these models talk about incessantly.
You can dismiss it as hype or hysteria. But researchers at all these companies worry LLMs, because we don't fully understand them, could outsmart their human creators and go rogue. In the AI 2027 report, the authors warn that competition with China will push LLMs potentially beyond human control, because no one will want to slow progress even if they see signs of acute danger.
The safe-landing theory: Google's Sundar Pichai — and really all of the big AI company CEOs — argue that humans will learn to better understand how these machines work and find clever, if yet unknown ways, to control them and " improve lives." The companies all have big research and safety teams, and a huge incentive to tame the technologies if they want to ever realize their full value.

Orange background

Try Our AI Features

Explore what Daily8 AI can do for you:

Comments

No comments yet...

Related Articles

ABC's Terry Moran is suspended following his social media post calling Trump and Miller haters
ABC's Terry Moran is suspended following his social media post calling Trump and Miller haters

Associated Press

time13 minutes ago

  • Associated Press

ABC's Terry Moran is suspended following his social media post calling Trump and Miller haters

NEW YORK (AP) — ABC News has suspended correspondent Terry Moran for calling Trump administration deputy chief of staff Stephen Miller a 'world class hater' in a since-deleted social media post. Moran's post was swiftly condemned by officials in the Republican administration, including Vice President J.D. Vance. ABC News, in a statement, said it 'stands for objectivity and impartiality in its news coverage and does not condone subjective personal attacks on others.' The New York-based network said Moran was suspended pending further evaluation. Moran, who interviewed President Donald Trump a few weeks ago, said in his post on X at 12:06 a.m. on Sunday that the president was a world-class hater, too. But he wrote that for the president, his hatred is a means to an end, 'and that end is his own glorification.' For Miller, Moran's post said, 'his hatreds are his spiritual nourishment. He eats his hate.' Vance, on X, said that Moran's post was 'dripping with hatred.' The vice president wrote: 'Remember that every time you watch ABC's coverage of the Trump administration.' Miller, on X, said Moran's 'full public meltdown' exposed the corporate press. 'For decades, the privileged anchor and reporters narrating and gatekeeping our society have been radicals adopting a journalist's pose. Terry pulled off his mask.'

US-China trade, inflation, Apple's big event: Here's what the stock market is watching this week
US-China trade, inflation, Apple's big event: Here's what the stock market is watching this week

Business Insider

time17 minutes ago

  • Business Insider

US-China trade, inflation, Apple's big event: Here's what the stock market is watching this week

Investors will be monitoring a host of potentially market-moving events this week, with updates due on trade and inflation, while Apple kicks off a highly anticipated product event. Recession fears have edged down after the turmoil that racked markets earlier in the spring, but the market is still struggling with uncertainty regarding President Donald Trump's trade policies and their implications for the economy. While last week's jobs report showed a solid labor market, investors are monitoring how the inflation side of the Federal Reserve's dual mandate fares this week, and how it will influence the rate-cut outlook for the year. Meanwhile, Apple's Worldwide Developers Conference will provide insight into not only new software updates but also the future of the AI race among mega-cap tech companies. Here's what investors are watching this week. US-China trade talks After last week's phone call between Trump and Chinese president Xi Jinping, China and US trade officials are meeting in London on Monday for two days of trade negotiations. Last month's trade talks were key to calming recession fears and helped propel the S&P 500 to its highest levels since February, but concerns still remain. The biggest negotiation topic will be over China's exports of rare earth metals, which are critical components in manufacturing semiconductors, smartphones, and other technologies. Continued improvements in trade relations between the two countries will be critical to reducing volatility in the market and could shed clarity on the direction of tariff rates. CPI data The consumer price index for May will be released on Wednesday. Last month 's reading of 2.3% was fairly benign, but investors will continue to watch for signs of Trump's tariffs showing up in the hard data. Importantly, the reading will be key in determining the Fed's next move. The median forecast is for annual consumer inflation to have risen 2.5% last month. Meanwhile, expectations for the June 17 Fed meeting are for officials to keep interest rates unchanged. "The big surprise could be how little Trump's tariffs are boosting inflation despite upward pressures on prices-paid and prices-received indexes in the Fed's regional business surveys," wrote on Sunday. Yet, some strategists have predicted that inflation will pick up in the back half of this year, spurring stagflation concerns. Meanwhile, consumer sentiment will get a fresh reading on Friday. Sentiment has been low as Americans feel pessimistic about tariffs, though hard data that the Fed looks at has held up. Apple's Worldwide Developers Conference All eyes will be on Apple this week as it kicks off its annual Worldwide Developers Conference, where the company is expected to unveil new AI features embedded in iOS 19. The conference will be an opportunity for Apple to address several headwinds it has faced this year. "In a nutshell WWDC is a pivotal moment in Apple's future as the developers are the hearts and lungs of the Cupertino growth story with the Street being laser-focused on Apple today," Wedbush analyst Dan Ives wrote. The tech giant has trailed peers like Microsoft and Google in the AI race, and its stock has taken a beating this year as the worst-performing Magnificent Seven member, largely due to concerns about tariffs and iPhone production. Last month, Trump threatened a tariff of at least 25% on iPhones not made in the US. Investors will be looking for updates on Apple Intelligence as well, as the company's AI offering has been underwhelming to Wall Street. A key bond auction The US Treasury sells a lot of bonds, and usually the sale is unremarkable for markets. However, with deficit concerns running high as the GOP budget bill moves through Congress, a $22 billion auction of 30-year bonds on Thursday could move the market if demand appears weak. A weak sale of 20-year bonds last month rattled markets and sent yields surging, and all eyes are on this week's sale as a potential investor referendum on the sweeping tax and spending bill.

US, Chinese trade negotiators meeting in London
US, Chinese trade negotiators meeting in London

The Hill

time18 minutes ago

  • The Hill

US, Chinese trade negotiators meeting in London

Top U.S. and Chinese officials are meeting in London on Monday to try to fortify the countries' temporary trade truce, which is currently on track to expire in August. Treasury Secretary Scott Bessent, Commerce Secretary Howard Lutnick and U.S. trade representative Jamieson Greer are in the U.K. for the talks with Chinese Vice President He Lifeng. It's unclear how long negotiations could last, but Chinese officials have predicted they could extend several days. 'The two sides need to make good use of the economic and trade consultation mechanism already in place, and seek win-win results in the spirit of equality and respect for each other's concerns,' Chinese Foreign Ministry spokesman Lin Jian wrote in a post on X ahead of the meeting. 'The Chinese side is sincere about this, and at the same time has its principles.' President Trump confirmed plans for the London confab last week after a phone call with Chinese President Xi Jinping, who the president has described as 'extremely hard to make a deal with.' 'The call lasted approximately one and a half hours, and resulted in a very positive conclusion for both Countries,' Trump wrote in a social media post Thursday. The two sides have been attempting to hash out a long-term trade agreement following Trump's announcement of sweeping tariff hikes on most countries in April. The Trump administration urged countries last week to come forward with deals more favorable to U.S. interests. U.S. and Chinese leaders brokered their temporary pause in the tariff hikes after meeting in Geneva last month. Under that arrangement, the U.S. lowered its tariff rate on Chinese goods from 145 percent to 30 percent, and China agreed to lower its tariff to 10 percent from 125 percent for 90 days. China's exports to the U.S. were down 35 percent in May compared to last year, according to the latest analysis from Dutch multinational banking and financial services firm ING Group, adding pressure ahead of the latest round of meetings between the two countries. 'Exports to the U.S. surprisingly decelerated despite the trade war reprieve,' ING's analysts wrote. 'We expect that export growth to the US could recover in the coming months.' 'We could see import front-loading amid the still elevated risk that tariffs could once again move higher in light the uncertainty about trade talks over the past month,' the firm added.

DOWNLOAD THE APP

Get Started Now: Download the App

Ready to dive into the world of global news and events? Download our app today from your preferred app store and start exploring.
app-storeplay-store