logo
Behind the Curtain: The scariest AI reality

Behind the Curtain: The scariest AI reality

Axios2 days ago

The wildest, scariest, indisputable truth about AI's large language models is that the companies building them don't know exactly why or how they work.
Sit with that for a moment. The most powerful companies, racing to build the most powerful superhuman intelligence capabilities — ones they readily admit occasionally go rogue to make things up, or even threaten their users — don't know why their machines do what they do.
Why it matters: With the companies pouring hundreds of billions of dollars into willing superhuman intelligence into a quick existence, and Washington doing nothing to slow or police them, it seems worth dissecting this Great Unknown.
None of the AI companies dispute this. They marvel at the mystery — and muse about it publicly. They're working feverishly to better understand it. They argue you don't need to fully understand a technology to tame or trust it.
Two years ago, Axios managing editor for tech Scott Rosenberg wrote a story, "AI's scariest mystery," saying it's common knowledge among AI developers that they can't always explain or predict their systems' behavior. And that's more true than ever.
Yet there's no sign that the government or companies or general public will demand any deeper understanding — or scrutiny — of building a technology with capabilities beyond human understanding. They're convinced the race to beat China to the most advanced LLMs warrants the risk of the Great Unknown.
The House, despite knowing so little about AI, tucked language into President Trump's "Big, Beautiful Bill" that would prohibit states and localities from any AI regulations for 10 years. The Senate is considering limitations on the provision.
Neither the AI companies nor Congress understands the power of AI a year from now, much less a decade from now.
The big picture: Our purpose with this column isn't to be alarmist or " doomers." It's to clinically explain why the inner workings of superhuman intelligence models are a black box, even to the technology's creators. We'll also show, in their own words, how CEOs and founders of the largest AI companies all agree it's a black box.
Let's start with a basic overview of how LLMs work, to better explain the Great Unknown:
LLMs — including Open AI's ChatGPT, Anthropic's Claude and Google's Gemini — aren't traditional software systems following clear, human-written instructions, like Microsoft Word. In the case of Word, it does precisely what it's engineered to do.
Instead, LLMs are massive neural networks — like a brain — that ingest massive amounts of information (much of the internet) to learn to generate answers. The engineers know what they're setting in motion, and what data sources they draw on. But the LLM's size — the sheer inhuman number of variables in each choice of "best next word" it makes — means even the experts can't explain exactly why it chooses to say anything in particular.
We asked ChatGPT to explain this (and a human at OpenAI confirmed its accuracy): "We can observe what an LLM outputs, but the process by which it decides on a response is largely opaque. As OpenAI's researchers bluntly put it, 'we have not yet developed human-understandable explanations for why the model generates particular outputs.'"
"In fact," ChatGPT continued, "OpenAI admitted that when they tweaked their model architecture in GPT-4, 'more research is needed' to understand why certain versions started hallucinating more than earlier versions — a surprising, unintended behavior even its creators couldn't fully diagnose."
Anthropic — which just released Claude 4, the latest model of its LLM, with great fanfare — admitted it was unsure why Claude, when given access to fictional emails during safety testing, threatened to blackmail an engineer over a supposed extramarital affair. This was part of responsible safety testing — but Anthropic can't fully explain the irresponsible action.
Again, sit with that: The company doesn't know why its machine went rogue and malicious. And, in truth, the creators don't really know how smart or independent the LLMs could grow. Anthropic even said Claude 4 is powerful enough to pose a greater risk of being used to develop nuclear or chemical weapons.
OpenAI's Sam Altman and others toss around the tame word of " interpretability" to describe the challenge. "We certainly have not solved interpretability," Altman told a summit in Geneva last year. What Altman and others mean is they can't interpret the why: Why are LLMs doing what they're doing?
Anthropic CEO Dario Amodei, in an essay in April called "The Urgency of Interpretability," warned: "People outside the field are often surprised and alarmed to learn that we do not understand how our own AI creations work. They are right to be concerned: this lack of understanding is essentially unprecedented in the history of technology." Amodei called this a serious risk to humanity — yet his company keeps boasting of more powerful models nearing superhuman capabilities.
Anthropic has been studying the interpretability issue for years, and Amodei has been vocal about warning it's important to solve. In a statement for this story, Anthropic said: "Understanding how AI works is an urgent issue to solve. It's core to deploying safe AI models and unlocking [AI's] full potential in accelerating scientific discovery and technological development. We have a dedicated research team focused on solving this issue, and they've made significant strides in moving the industry's understanding of the inner workings of AI forward. It's crucial we understand how AI works before it radically transforms our global economy and everyday lives." (Read a paper Anthropic published last year, "Mapping the Mind of a Large Language Model.")
Elon Musk has warned for years that AI presents a civilizational risk. In other words, he literally thinks it could destroy humanity, and has said as much. Yet Musk is pouring billions into his own LLM called Grok.
"I think AI is a significant existential threat," Musk said in Riyadh, Saudi Arabia, last fall. There's a 10%-20% chance "that it goes bad."
Reality check: Apple published a paper last week, "The Illusion of Thinking," concluding that even the most advanced AI reasoning models don't really "think," and can fail when stress-tested.
The study found that state-of-the-art models (OpenAI's o3-min, DeepSeek R1 and Anthropic's Claude-3.7-Sonnet) still fail to develop generalizable problem-solving capabilities, with accuracy ultimately collapsing to zero "beyond certain complexities."
But a new report by AI researchers, including former OpenAI employees, called " AI 2027," explains how the Great Unknown could, in theory, turn catastrophic in less than two years. The report is long and often too technical for casual readers to fully grasp. It's wholly speculative, though built on current data about how fast the models are improving. It's being widely read inside the AI companies.
It captures the belief — or fear — that LLMs could one day think for themselves and start to act on their own. Our purpose isn't to alarm or sound doomy. Rather, you should know what the people building these models talk about incessantly.
You can dismiss it as hype or hysteria. But researchers at all these companies worry LLMs, because we don't fully understand them, could outsmart their human creators and go rogue. In the AI 2027 report, the authors warn that competition with China will push LLMs potentially beyond human control, because no one will want to slow progress even if they see signs of acute danger.
The safe-landing theory: Google's Sundar Pichai — and really all of the big AI company CEOs — argue that humans will learn to better understand how these machines work and find clever, if yet unknown ways, to control them and " improve lives." The companies all have big research and safety teams, and a huge incentive to tame the technologies if they want to ever realize their full value.

Orange background

Try Our AI Features

Explore what Daily8 AI can do for you:

Comments

No comments yet...

Related Articles

Honeywell Finalizes Sundyne Acquisition to Boost Its Process Industry Strength
Honeywell Finalizes Sundyne Acquisition to Boost Its Process Industry Strength

Yahoo

time18 minutes ago

  • Yahoo

Honeywell Finalizes Sundyne Acquisition to Boost Its Process Industry Strength

Honeywell International Inc. (NASDAQ:HON) is one of the best Dow stocks to invest in. On June 9, the company announced that it had finalized its $2.16 billion all-cash acquisition of Sundyne from private equity firm Warburg Pincus. Sundyne, known for its advanced pumps and gas compressors used in process industries, is expected to contribute immediately to Honeywell International Inc. (NASDAQ:HON)'s revenue growth, segment margins, and adjusted earnings per share during the first full year of ownership. Sundyne brings strong customer relationships and top-tier technology, which are expected to enhance Honeywell UOP's presence in areas like refining, petrochemicals, LNG, and clean energy. Integrated with the Honeywell Forge platform, the combined offering strengthens Honeywell's position in the energy transition by providing a comprehensive, scalable solution. This acquisition comes on the heels of Honeywell International Inc. (NASDAQ:HON)'s recent announcement to spin off its Aerospace Technologies and Solstice Advanced Materials units, moves aimed at creating three separate publicly traded companies, each with focused strategies and growth opportunities. Since December 2023, Honeywell has pursued several strategic initiatives to boost organic growth and streamline its operations, including $13.5 billion in value-enhancing acquisitions. The company operates globally across a wide range of industries. HON has surged by over 7% in the past 12 months. While we acknowledge the potential of HON as an investment, we believe certain AI stocks offer greater upside potential and carry less downside risk. If you're looking for an extremely undervalued AI stock that also stands to benefit significantly from Trump-era tariffs and the onshoring trend, see our free report on the best short-term AI stock. READ NEXT: and Disclosure. None. Error in retrieving data Sign in to access your portfolio Error in retrieving data Error in retrieving data Error in retrieving data Error in retrieving data

LA-based Jimmy Kimmel claims ‘there's no riot outside' as he blasts ‘mentally ill' Trump
LA-based Jimmy Kimmel claims ‘there's no riot outside' as he blasts ‘mentally ill' Trump

New York Post

time19 minutes ago

  • New York Post

LA-based Jimmy Kimmel claims ‘there's no riot outside' as he blasts ‘mentally ill' Trump

Los Angeles-based comedian Jimmy Kimmel told his audience on Tuesday that 'there's no riots outside' and slammed 'mentally ill' President Donald Trump for deploying troops to quell what he claims is exaggerated unrest in the city. Trump sent over 700 Marines to Los Angeles in an effort to quash anti-ICE protests that have ravaged parts of the city on Tuesday. Images from L.A. showcase masked protesters blocking roads, destroying vehicles and engaging with police, while Mayor Karen Bass issued a curfew for a portion of the city's downtown area. Advertisement Kimmel mocked the media for portraying the anti-ICE protests as 'some kind of totalitarian hellscape' and condemned the Trump administration for deporting 'people who have lived here their whole lives.' 'You won't see this elsewhere on television. Not only is it not an apocalypse, they're having a Disney/Pixar movie premiere for 'Elio,' a movie about aliens. Don't tell Trump — he'll send in the Green Berets, too,' Kimmel ribbed. 'But I just want to say, thank God for President Trump and the heroes at ICE for protecting us from these bloodthirsty fruit stand vendors spreading their dangerous pineapple chunks and mangos with a squirt of lime all over the city.' 4 Kimmel mocked the media for portraying the anti-ICE protests as 'some kind of totalitarian hellscape.' ABC 4 Cars burn during a protest against federal immigration sweeps in downtown Los Angeles, California, U.S. June 8, 2025. REUTERS Advertisement The late-night host vented his anger towards the ICE raids in the city, claiming that the 'vast majority' of those being deported have 'never done anything wrong.' 'People who have lived here their whole lives, people who have been in this city longer than I have, the vast majority of whom have never done anything wrong, are being abducted, which is the correct word to use, by agents in masks, hiding their identities, grabbing people off the street and at work, sending people to detention centers,' he claimed. Kimmel asserted that it's not only Los Angelenos' right to protest the deportations, but it's their 'responsibility.' 4 U.S. President Donald Trump walks away after speaking to the media upon arrival at Joint Base Andrews following a visit to North Carolina, in Maryland, U.S., June 10, 2025. REUTERS Advertisement 'Los Angelenos gathered to demonstrate and, with very few exceptions, peacefully demonstrate to voice their opposition to this disgusting and unnecessary abuse of power instigated by our mentally ill president, who is dead-set on exacerbating this, who actually wants conflict, who is intentionally inflaming and lying to make it seem like there's a war going on,' he charged. Kimmel continued his criticism of the president and claimed that 'he wants there to be a war going on' in L.A. and doesn't care who gets hurt in the process. 'There's no riot outside,' Kimmel declared. 'We have more so-called 'unrest' here when one of our teams wins a championship.' 4 A protestor is detained in downtown Los Angeles, Sunday, June 8, 2025, following last night's immigration raid protest. AP Advertisement The host maintained that the media is exaggerating the violence taking place at the anti-ICE protests and argued that Trump exacerbated the unrest by sending in troops. 'Someone sets a fire in a garbage can, 12 camera crews go running toward it,' he asserted. 'Trump wants it to seem like anarchy, so he goes around our governor and calls in 4,000 troops from the National Guard and 700 active-duty Marines. When we had the wildfires that devastated big chunks of our city, he did absolutely nothing. Now that we're in the middle of a non-emergency, send in the National Guard!'

Members of the Fulbright scholarship board resign, accusing Trump of meddling

time21 minutes ago

Members of the Fulbright scholarship board resign, accusing Trump of meddling

All 12 members of the board overseeing the prestigious Fulbright scholarships on Wednesday resigned in protest of what they call the Trump administration's meddling with the selection of award recipients, according to a statement. A statement published online by the board members said the administration usurped the board's authority by denying awards to 'a substantial number of people' who already had been chosen. Another 1,200 award recipients who were already approved to come to the U.S. are undergoing an unauthorized review process that could lead to their rejection, the board members said. 'To continue to serve after the Administration has consistently ignored the Board's request that they follow the law would risk legitimizing actions we believe are unlawful and damage the integrity of this storied program and America's credibility abroad,' the statement reads. Congress established the Fulbright program nearly 80 years ago to promote international exchange and American diplomacy. The highly selective program awards about 9,000 scholarships annually in the U.S. and in more than 160 other countries to students, scholars, and professionals in a range of fields. A message seeking comment was left with the State Department, which runs the scholarship program. The resignations were first reported by The New York Times. The intervention from the Trump administration undermined the program's merit-based selection process and its insulation from political influence, the board members wrote. 'We believe these actions not only contradict the statute but are antithetical to the Fulbright mission and the values, including free speech and academic freedom, that Congress specified in the statute,' the statement said. 'It is our sincere hope that Congress, the courts, and future Fulbright Boards will prevent the administration's efforts to degrade, dismantle, or even eliminate one of our nation's most respected and valuable programs.' Award recipients are selected in a yearlong process by nonpartisan staff at the State Department. The recipients who had their awards canceled are in fields including biology, engineering, medical sciences, and history, the board members said. The announcement comes as the Trump administration ratchets up scrutiny of international students on several fronts. The administration has expanded the grounds for revoking foreign students' legal status, and recently paused scheduling of new interviews for student visas as it increases vetting of their social media activity. The government also has moved to block foreign students from attending Harvard as it pressures the Ivy League school to adopt a series of reforms. ___ standards for working with philanthropies, a list of supporters and funded coverage areas at

DOWNLOAD THE APP

Get Started Now: Download the App

Ready to dive into the world of global news and events? Download our app today from your preferred app store and start exploring.
app-storeplay-store