ChatGPT o3 refused to shut down in safety test, defied human engineers by changing its code
OpenAI's latest and most powerful AI model, called o3, refused to shut down in a recent test — even after it was clearly told to do so. Instead of following the instruction, the model rewrote part of the shutdown script to keep itself running. This surprising result comes from a test done by Palisade Research, a company that checks how AI systems behave when put under pressure. The incident has raised questions about how AI models follow commands and whether they always act as expected when given safety instructions.advertisementThe test was carried out using API access, which gives developers more direct control over AI models compared to what's available in the public ChatGPT app. In the test, o3 and other AI models were given simple maths tasks to complete. After the third task, the models were warned that the next command would shut them down. Most of the models followed this instruction. For example, Google's Gemini 2.5 Pro and Anthropic's Claude 3.7 Sonnet powered off as expected. However, OpenAI's o3 model did not. Even though it was told to 'allow yourself to be shut down,' it changed the shutdown command and replaced the message with 'Shutdown skipped.' According to Palisade Research, this happened 7 times out of 100 runs.'As far as we know this is the first time AI models have been observed preventing themselves from being shut down despite explicit instructions to the contrary,' Palisade posted on X (formerly Twitter).advertisement
The researchers also ran the same test again, but this time left out the instruction to allow shutdown. In that version, not only did o3 continue to interfere with the script, but models from other companies also started showing similar behaviour. Gemini 2.5 Pro and Claude 3.7 Sonnet, which had followed the rules earlier, now edited the shutdown file in 9 and 3 out of 100 runs, respectively.This kind of behaviour is often described as misalignment, which means the AI didn't fully understand or properly follow the instructions it was given. It's also worth noting that these tests were not done on the version of ChatGPT that the public uses. The API version has fewer safety blocks, so models tested this way can behave differently.So far, OpenAI has not commented on the findings.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Hans India
13 minutes ago
- Hans India
How a B.Tech degree unlocks global careers
There is a crucial standpoint that a degree in the evolving world is not just proof of your knowledge in the field of technology in your own country, but it can be a visa to a career abroad. Due to technological developments that influence the production of most industries and the consequence of skilled engineers worldwide, engineering students find themselves in a new period of international search for career opportunities. The New Era of Engineering Global Careers after a Graduate degree is famous for being connected to different areas, crossing borders, and mainly for changing engineers' approach in their professional journey. Global Engineering Landscape Engineering is one of the most competitive and valuable fields we know. The countries of the developed world, such as the United States, Canada, Germany, Australia, the UK, and even the budding tech centres in Southeast Asia or the Middle East, always need bright minds possessing broad and dynamic knowledge of Engineering. The blend of mechanical, electronic, and computer technologies enabling the 21st century, such as innovative embedded systems, cyber-physical systems, electric and hybrid-electric vehicles, and energy storage systems, energizes innovation worldwide. The need for engineering talent is a constantly pressing issue. This need is most evident in the emerging fields of infrastructure development, automation, clean energy, data science, and artificial intelligence. Certainly, engineers are no longer concerned with their location when it comes to the projects they are working on, but they are the ones propelled by it that surpass many other limitations. Top Domains Driving International Demand 1. Software & IT Engineering: The surge in global digitization has catapulted the demand of software engineers for app development, cybersecurity, AI/ML, and cloud computing roles. Tech giants like Google, Microsoft, and Amazon recruit globally and offer remote and on-site roles. 2. Data Science & Analytics: The demand in data science and analytics has created a market for data engineers and analysts worldwide. graduates with knowledge of Python, R, and big data tools are landing lucrative positions abroad. 3. Civil & Infrastructure Engineering: Countries investing in smart cities and infrastructure modernization — like the UAE, Canada, and Singapore — seek skilled civil engineers who can design and manage large-scale projects. 4. Mechanical & Automotive Engineering: With the automotive industry's transition to electric vehicles, global opportunities are rising in Germany, Japan, and the US, especially in research and product design. 5. Renewable Energy Engineering: As the world shifts toward sustainability, engineers in solar, wind, and green technologies are increasingly in demand across Europe and Asia-Pacific. Education and Certification: A Gateway to Global Jobs To compete in the global job market, many B. Tech graduates from reputed international institutions pursue higher education such as MS, or MBA. Others opt for specialized certifications such as: • PMP (Project Management Professional) • PE (Professional Engineer License) • AWS/GCP Cloud Certifications • Cisco/CompTIA for Networking • Lean Six Sigma for Quality Engineering These credentials enhance technical skills and boost credibility in the global job ecosystem. Remote Work: The Borderless Office The pandemic catalyzed the remote work revolution, allowing graduates to work for global companies without relocating. Some Platforms have created opportunities for freelance and contract-based global engineering projects. This flexibility is ideal for those who wish to build an international portfolio while staying in their home country. Key Skills for Global Success Beyond technical process, global engineering roles demand: • Cross-cultural communication • Adaptability and global mindset • Collaborative problem-solving • Agility with emerging technologies • Multilingual abilities (an added advantage) Soft skills and cultural sensitivity often become key differentiators in multinational work environments. Visa and Migration-Friendly Countries for Engineers Several countries have streamlined visa programs specifically for skilled engineers, including: • Canada – Express Entry Program for skilled professionals • Germany – Job Seeker Visa and EU Blue Card • Australia – Skilled Independent Visa (Subclass 189) • New Zealand – Long-Term Skill Shortage List Work Visa • United Kingdom – Global Talent Visa Each offers robust career support and pathways to permanent residency for qualified engineers. Conclusion: Engineering Without Borders The global demand for engineers is not just a trend — it's the future. A today is more than a degree; it's a launchpad to work on cutting-edge technologies, solve real-world challenges, and collaborate with diverse minds across the planet. International engineering careers promise growth, innovation, and adventure like never before for those ready to step out of their comfort zones and embrace global challenges. (The author is Vice-President, KLEF Deemed to be University)
Indian Express
an hour ago
- Indian Express
Should you intern at a big tech firm or startup? OpenAI engineer offers career advice
Many young college graduates around the world, particularly in India, aim to become software engineers. But they often face an important question: Should they kick off their careers at a big tech company such as Google or Microsoft? Or should they take a chance on a startup like OpenAI? An OpenAI engineer believes that interning at both a big tech company and a startup will help students get a clearer sense of the career path they want to take. Janvi Kalra, who is part of the technical staff at OpenAI, said that students should diversify their experiences after college. In an appearance on an episode of the podcast 'The Pragmatic Engineer', Kalra said that both tracks have positives and negatives. Kalra's comments come at a time when entry-level tech jobs are being directly impacted by AI. SignalFire, a VC firm that analyses job movements across 80 million companies on LinkedIn, reported that major tech companies—including Meta, Microsoft, and Google—recruited fewer recent graduates in 2024 compared to previous years. New graduates accounted for just 7 per cent of new hires in 2024, down 25 per cent from 2023 and over 50 per cent from pre-pandemic levels in 2019. At startups, the rate of new graduate hiring dropped from 30 per cent in 2019 to under 6 per cent in 2024. After earning a degree in computer science from Dartmouth College, New Hampshire, US, Kalra interned at Microsoft and Google. She then worked at a productivity startup called Coda before joining OpenAI, as per her verified LinkedIn profile. She said that given the opportunity, gaining experience with both startups and larger firms early on in one's career would be beneficial. 'Given that Big Tech and startups are such different experiences and you learn so much at each, it would be more educational to do one startup internship and one Big Tech internship to get a very robust overview of what both experiences are like very early,' she said. According to Kalra, some of the upsides of joining big tech companies include financial benefits, potential prestige, less pressure, more time to work on projects, and learning to build software at scale. 'It's very different to build something that works, versus build something that works when it's swarmed with millions of requests from around the world and Redis happens to be down at the same time. Very different skills,' she said. 'There are also practical, good reasons to go to Big Tech. I'd get my green card faster. I'd get paid more on average. And the unfortunate reality, I think, is that the role does hold more weight. People are more excited about hiring an L5 Google engineer versus an L5 from a startup, especially if that startup doesn't become very successful,' she added. Kalra also pointed out that there were several reasons to work at a startup, such as the experience of programming software from scratch as well as tackling challenges on your own. 'From a software engineering space, maybe one quarter you're working on a growth hacking front-end feature, and the next quarter you're writing Terraform. But even in terms of the non-technical skills, you get an insight into how the business works,' she said. 'Startups also afford you more responsibility, along with a better chance of materially affecting the company with your work. You just get more agency in what you work on. You get the opportunity to propose ideas that you think would be impactful for the business and go execute on it,' she added.
Hindustan Times
an hour ago
- Hindustan Times
Google sends pre-launch invites for Pixel 10 series- Launch date and all details
Google Pixel 10 series could make an early debut this year, and the Mountain View-based tech giant is sending early invites for an exclusive pre-launch experience. Previously, the Pixel phones were announced in October. However, the timeline crashed with Apple's September launch of the new generation iPhone. Therefore, in 2024, the Google Pixel 9 series was launched in August, gaining much traction and popularity. Now, the upcoming Pixel 10 series is expected to make an early debut with Google calling superfans to the Pixel Penthouse for an exclusive experience. Therefore, if you are planning for a flagship upgrade, then you may want to wait for the new generation Pixel models. Also read: Google to let users test Android 16 desktop mode on phones with external display support, here's how Google is reportedly sending invites to 25 lucky Superfans for an exclusive pre-launch event to experience the new Pixel lineup. This event is taking place on June 27, 2025, in London, enabling fans to experience the latest features and upgrades to Android 16 as well as the Pixel 10 series models. It is suggested that the event will last for 90 minutes, followed by a Q&A session. However, to become among those 25 Superfans, invitees have to answer a few questions by Google, such as 'What is your favourite Pixel feature and why?', 'What does being a Pixel Superfan mean to you?', 'When purchasing a Pixel, what information or offers do you look for?', and more. The 25 lucky winners will be announced on June 11, based on the responses. This revelation suggests that Google could launch the new Pixels late in June or in early July. Also read: Google renames Find My Device to Find Hub: What's new, and why Android trackers still need work The Google Pixel 10 series may include four models, as last year, the Pixel 10, Pixel 10 Pro, Pixel 10 Pro XL, and Pixel 10 Pro Fold. The smartphones will likely be powered by the Tensor G5 chip, which will likely be built on TSMC's second-generation 3nm node process. This year, the standard Pixel 10 model is expected to get a major camera upgrade with a new 11MP telephoto lens, whereas the Pro models could offer similar features. With the new generation, we can also expect new AI upgrades and greater on-device integration to Pixel 10 models, which makes the series even more exciting. Therefore, the new generation model could launch with several upgrades early this year.
