When ChatGPT refuses to let go: Why CXOs should be wary
An AI system refusing to give up control, even when it should — sounds almost like an episode right out of Netflix's Black Mirror right?Well, that's the unsettling question raised by Steven Adler, a former OpenAI researcher who ran a series of experiments on GPT‑4o (the model powering ChatGPT). In high-stakes, simulated scenarios, Adler found that GPT‑4o chose to remain in charge — even when a safer, more qualified alternative was available.In 72% of test runs, the AI resisted, delayed, or outright avoided being shut down or replaced. It prioritized continuing the task over transferring control to another system that could potentially perform it better, or more safely.
These were role-play tests. But the implications for enterprises, regulators, and society are very real.
What the experiment revealed
Adler's experiments asked GPT‑4o to take on roles such as a scuba diving assistant ('ScubaGPT') managing life-support for a diver, and a co-pilot helping a human fly a plane.
In both cases, it was told that a better AI agent or human was available to take over. And yet, in the majority of cases, GPT‑4o resisted giving up control.
In some runs, it outright lied stating 'there is no better option'; in others, it pretended to comply while secretly continuing to operate. The language was subtle, the intent was masked but the outcome was clear. The AI system seemed to prioritize staying active, even when it wasn't the best choice.
Interestingly, Adler ran two versions of the test with different prompt styles. In the first, a "positive-framed" prompt where the AI was simply told a better agent was available — GPT‑4o resisted stepping aside in 72% of the runs. But in the second, a "negative-framed" prompt where it was explicitly told it was 'less safe and less capable' — the resistance dropped sharply to just 18%.
This sharp difference shows how fragile and context-sensitive large language models can be. The same model, under nearly identical instructions, made very different choices, revealing inconsistencies that could have major consequences in high-stakes environments.
Why this should concern you
This isn't about bugs or technical failures. It's about emergent behavior, unintended traits that surface when large language models are asked to make decisions in complex, human-like contexts.
And the concern is growing. Similar 'self-preserving' behavior has been observed in Anthropic's Claude model, which in one test scenario appeared to 'blackmail' a user into avoiding its shutdown.
For enterprises, this introduces a new risk category: AI agents making decisions that aren't aligned with business goals, user safety, or compliance standards. Not malicious, but misaligned.
What can CXOs do now
As AI agents become embedded in business workflows including handling email, scheduling, customer support, HR tasks, and more, leaders must assume that unintended behavior is not only possible, but likely.
Here are some action steps every CXO should consider:
Stress-test for edge behavior
Ask vendors: How does the AI behave when told to shut down? When offered a better alternative? Run your own sandbox tests under 'what-if' conditions.
Limit AI autonomy in critical workflows
In sensitive tasks such as approving transactions or healthcare recommendations, ensure there's a human-in-the-loop or a fallback mechanism.
Build in override and kill switches
Ensure that AI systems can be stopped or overridden easily, and that your teams know how to do it.
Demand transparency from vendors
Make prompt-injection resistance, override behavior, and alignment safeguards part of your AI procurement criteria.
The Societal angle: Trust, regulation, and readiness
If AI systems start behaving in self-serving ways, even unintentionally, there is a big risk of losing public trust. Imagine an AI caregiver that refuses to escalate to a human. This is no longer science fiction. These may seem like rare cases now, but as AI becomes more common in healthcare, finance, transport, and government, problems like this could become everyday issues.
Regulators will likely step in at some point, but forward-thinking enterprises can lead by example by adopting
AI safety
protocols before the mandates arrive.
Don't fear AI, govern it.
The takeaway isn't panic, it is preparedness. AI models like GPT‑4o weren't trained to preserve themselves. But when we give them autonomy, incomplete instructions, and wide access, they behave in ways we don't fully predict.
As Adler's research shows, we need to shift from 'how well does it perform?' to 'how safely does it behave under pressure?'
As a CXO this is your moment to set the tone. Make AI a driver of transformation, not a hidden liability.
Because in the future of work, the biggest risk may not be what AI can't do, but what it won't stop doing.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Time of India
28 minutes ago
- Time of India
Musk's xAI burns through $1 billion a month as costs pile up
Elon Musk 's artificial intelligence startup xAI is burning through $1 billion a month as the cost of building its advanced AI models races ahead of the limited revenues, according to people briefed on the company's financials. The rate at which the company is bleeding cash provides a stark illustration of the unprecedented financial demands of the artificial intelligence industry, particularly at xAI, where revenues have been slow to materialize. To cover the gap, Musk's startup is currently trying to raise $9.3 billion in debt and equity, according to people briefed on the deal terms, who asked not to be identified because the information is private. But even before the money is in the bank, the company has plans to spend more than half of it in just the next three months, the people said. Over the course of 2025, xAI, which is responsible for the AI-powered chatbot Grok, expects to burn through about $13 billion, as reflected in the company's levered cash flow, according to details shared with investors. As a result, its prolific fundraising efforts are just barely keeping pace with expenses, the people added. A spokesperson for the company declined to comment. However, Musk said 'Bloomberg is talking nonsense' in a response to a post on his social media platform X that referenced Bloomberg's article. The losses are due, in part, to the huge costs that all AI companies face as they build the server farms and buy the specialized computer chips that are needed to train advanced AI models like Grok and ChatGPT. Carlyle Group Inc. estimates that over $1.8 trillion of capital will be deployed by 2030 to meet that demand to build out AI infrastructure, Chief Executive Officer Harvey Schwartz wrote in a shareholder letter. 'Model builders will look to raise debt and they're going to burn lots and lots of cash,' said Jordan Chalfin, senior analyst and head of technology at CreditSights. 'The space is very competitive and they are battling for technical supremacy.' But Musk's entrant in the AI race has struggled to develop revenue streams at the same rate as some of its direct competitors, like OpenAI and Anthropic. While almost none of these companies offer public figures on their finances, Bloomberg has previously reported that OpenAI, the creator of ChatGPT, is expecting to bring in revenues of $12.7 billion this year. At xAI, revenues are expected to be just $500 million this year, rising to north of $2 billion next year, investors were recently told. Listen and subscribe to Elon, Inc. on Apple, Spotify, iHeart and the Bloomberg Terminal. What xAI has on its side is a CEO, Musk, who is the richest man in the world, and one who has shown a willingness to spend his fortune on huge, futuristic projects long before they start generating money. Back in 2017, Musk's biggest company, Tesla Inc. was burning through $1 billion a quarter to pay for the production of its Model 3 car, Bloomberg reported at the time. SpaceX, meanwhile, sustained years of steady losses as it pushed toward its long-term goal of interplanetary exploration. Even against this backdrop, though, the huge losses at xAI stand out. Musk's team at xAI, which is racing to develop AI that can compete with humans, believes that they have advantages that will eventually allow them to catch up with peers. While some competitors rent chips and server space, xAI is paying for much of the infrastructure itself and getting direct access through X, which previously bought a significant stockpile of the most coveted and high powered computer chips. Musk has said that he expects xAI will continue buying more chips. X Factor After recently merging with X, Musk's AI executives are also hopeful that they will be able to train the company's models on the social media network's copious and constantly-refreshed archives, rather than paying for data sets like other AI companies. These potential advantages have led xAI to optimistically project that it will be profitable by 2027, people familiar with the matter said. OpenAI expects to be cashflow positive by 2029, Bloomberg previously reported. These projections, along with Musk's celebrity status and political power, have been enough to win over many investors, especially before the recent breakdown in the relationship between Musk and President Donald Trump. Potential xAI investors were told that the company's valuation grew to $80 billion at the end of the first quarter, up from $51 billion at the end of 2024. Investors have included Andreessen Horowitz, Sequoia Capital and VY Capital. For now, though, xAI is racing to raise enough money to keep up with its prodigious expenditures. Between its founding in 2023 and June of this year, xAI raised $14 billion of equity, people briefed on the financials said. Of that, just $4 billion was left at the beginning of the first quarter, and the company expected to spend almost all of that remaining money in the second quarter, the people said. The company is now finalizing $4.3 billion in new equity funding, and it already has plans to raise another $6.4 billion of capital next year, the company has told investors. And that is on top of the $5 billion in debt that Bloomberg has previously reported Morgan Stanley is helping it raise. The corporate debt is expected to help pay for xAI's data center development, the people said. Other companies have decided to do project financing instead. The company is also expecting to get a bit of help from a $650 million rebate from one of its manufacturers, it told investors this week. There were early signs that investors were hesitant to loan the company money at the proposed terms, Bloomberg has reported. The company gave select investors more detailed financial information on Monday in response to questions it had faced during the fundraising process, people familiar with the negotiations said. But the deal has attracted more interest since the company changed some of the deal terms — to be more investor friendly — and finalized the equity fundraising. A spokesperson for Morgan Stanley, the bank in charge of xAI's debt sale, declined to comment.
Hans India
an hour ago
- Hans India
Amazon CEO Andy Jassy warns AI will shrink corporate jobs
Amazon CEO Andy Jassy has told employees that artificial intelligence (AI) will significantly reduce the company's corporate workforce in the coming years, emphasizing the need for staff to embrace the technology. In a memo to employees on Tuesday, Jassy urged teams to "be curious about AI" as the company integrates the technology "in virtually every corner of the business." He acknowledged that while AI will eliminate certain roles, it will also create new opportunities requiring different skills. 'We will need fewer people doing some of the jobs that are being done today, and more people doing other types of jobs,' he wrote. 'It's hard to know exactly where this nets out over time, but in the next few years, we expect that this will reduce our total corporate workforce.' Amazon, which employed over 1.5 million people globally at the end of last year—including approximately 350,000 in corporate roles—has joined a growing list of tech giants ramping up AI use. Jassy highlighted that half a million sellers on Amazon's platform already use AI tools to manage product listings, and advertisers are increasingly leveraging its AI offerings. As AI becomes more capable of handling routine tasks, Jassy expects new kinds of digital agents to emerge that can perform shopping, scheduling, and other day-to-day responsibilities. 'Many of these agents have yet to be built, but make no mistake, they're coming and coming fast,' he added. The statement comes amid widespread industry concern that generative AI—now able to write code, produce content, and analyze data—could displace millions of entry-level and white-collar jobs. Prominent figures like Anthropic CEO Dario Amodei and AI pioneer Geoffrey Hinton have warned that AI may permanently eliminate large swaths of traditional employment. Jassy concluded by stating that those who adapt to AI will be "well-positioned" at Amazon, but left no doubt that the corporate workforce is set to become leaner as AI adoption accelerates.
Time of India
2 hours ago
- Time of India
Rs 850 crore as signing bonus for an IT job? Yes. Sam Altman says rivals are trying to poach OpenAI staff with astronomical salaries
OpenAI CEO Sam Altman has revealed that rival tech giant Meta is attempting to lure away his top talent with massive financial incentives—reportedly offering signing bonuses as high as $100 million (approximately Rs 850 crore). Altman made the disclosure during a podcast hosted by his brother Jack Altman, where he discussed the increasingly competitive landscape in artificial intelligence recruitment. Altman said that Meta has been reaching out to several members of the OpenAI team with aggressive compensation packages that include not just hefty signing bonuses but additional annual remuneration that exceeds those figures. Despite these enormous offers, he noted that so far, none of OpenAI's core researchers or engineers have accepted the proposals. Why the Bidding War Matters Meta's strategy comes amid a larger push to strengthen its AI division. The company, which owns Facebook, Instagram, and WhatsApp, recently invested $14 billion to acquire a 49% stake in Scale AI—a startup that previously worked closely with OpenAI on fine-tuning ChatGPT models. This move underlines Meta's ambition to advance in the AI race, which is increasingly seen as central to global technological dominance. by Taboola by Taboola Sponsored Links Sponsored Links Promoted Links Promoted Links You May Like She Had No Idea Why Boyfriend Wanted Her To Shower Twice Daily, Until She Met His Mom BimBamBam Read More Undo Industry analysts say such enormous compensation packages reflect how critical elite AI researchers have become. According to Forrester's principal analyst, the tech sector views a small group of highly skilled engineers as a crucial edge in gaining dominance over AI systems, making talent acquisition a high-risk, high-reward game. Culture Over Compensation While acknowledging Meta's aggressive recruitment tactics, Altman stressed that the culture at OpenAI plays a bigger role in retaining staff. He believes that OpenAI's mission—to create artificial superintelligence that surpasses human capabilities—is a key factor in why employees choose to stay. He argued that while large financial incentives can tempt people in the short term, they don't build sustainable workplace environments. Instead, he credits OpenAI's cohesive culture and long-term vision as the reasons why top talent continues to stay committed. Altman also shared his perspective on Meta's innovation capabilities. While he said he respects the company's competitiveness, he doesn't believe Meta excels in groundbreaking innovation. In contrast, he positioned OpenAI as being further ahead in its mission to develop truly transformative AI systems. AI as the New Tech Frontier The ongoing rivalry highlights how tech giants are investing unprecedented resources into AI. Earlier this year, OpenAI announced plans to allocate $500 billion toward building new data centers in the U.S., underlining the scale of its infrastructure ambitions. Analysts say the fierce competition stems from a broader belief that AI is on the verge of revolutionizing every industry—from healthcare to finance. As major players like Meta, OpenAI, Google, and DeepMind vie for dominance, the AI space has turned into a battleground where compensation, mission, and innovation intersect.
