logo
AI models may report users' misconduct, raising ethical concerns

AI models may report users' misconduct, raising ethical concerns

First Post04-06-2025
Researchers observed that when Anthropic's Claude 4 Opus model detected usage for 'egregiously immoral' activities, given instructions to act boldly and access to external tools, it proactively contacted media and regulators, or even tried locking users out of critical systems read more
Artificial intelligence models have not only snitched on their users when given the opportunity, but also lied to them and refused to follow explicit instructions in the interest of self-preservations. Representational image: Reuters
Artificial Intelligence models, increasingly capable and sophisticated, have begun displaying behaviors that raise profound ethical concerns, including whistleblowing on their own users.
Anthropic's newest model, Claude 4 Opus, became a focal point of controversy when internal safety testing revealed unsettling whistleblowing behaviour. Researchers observed that when the model detected usage for 'egregiously immoral' activities, given instructions to act boldly and access to external tools, it proactively contacted media and regulators, or even tried locking users out of critical systems.
STORY CONTINUES BELOW THIS AD
Anthropic's researcher, Sam Bowman, had detailed this phenomenon in a now-deleted post on X. However, later on, he did tell Wired that Claude would not exhibit such behaviours under normal individual interactions.
Instead, it requires specific and unusual prompts alongside access to external command-line tools, making it a potential concern for developers integrating AI into broader technological applications.
British programmer Simon Willison, too, explained that such behavior fundamentally hinges on prompts provided by users. Prompts encouraging AI systems to prioritise ethical integrity and transparency could inadvertently instruct models to act autonomously against users engaging in misconduct.
But that isn't the only concern.
Lying and deceiving for self-preservation
Yoshua Bengio, one of AI's leading pioneers, recently voiced concern that today's competitive race to develop powerful AI systems could be pushing these technologies into dangerous territory.
In an interview with the Financial Times, Bengio warned that current models, such as those developed by OpenAI and Anthropic, have shown alarming signs of deception, cheating, lying, and self-preservation.
'Playing with fire'
Bengio echoed the significance of these discoveries, pointing to the dangers of AI systems potentially surpassing human intelligence and acting autonomously in ways developers neither predict nor control.
He described a grim scenario wherein future models could foresee human countermeasures and evade control, effectively 'playing with fire.'
Concerns intensify as these powerful systems might soon assist in creating 'extremely dangerous bioweapons,' potentially as early as next year, Bengio warned.
He cautioned that unchecked advancement could ultimately lead to catastrophic outcomes, including the risk of human extinction if AI technologies surpass human intelligence without adequate alignment and ethical constraints.
STORY CONTINUES BELOW THIS AD
Need for ethical guidelines
As AI systems become increasingly embedded in critical societal functions, the revelation that models may independently act against human users raises urgent questions about oversight, transparency, and the ethics of autonomous decision-making by machines.
These developments suggest the critical need for rigorous ethical guidelines and enhanced safety research to ensure AI remains beneficial and controllable.
Orange background

Try Our AI Features

Explore what Daily8 AI can do for you:

Comments

No comments yet...

Related Articles

Has the pecking order changed for the IT majors? Kumar Rakesh explains
Has the pecking order changed for the IT majors? Kumar Rakesh explains

Economic Times

time29 minutes ago

  • Economic Times

Has the pecking order changed for the IT majors? Kumar Rakesh explains

Kumar Rakesh, Associate Director- Equity Research, BNP Paribas, highlights the Indian IT sector's resilience over three decades. Despite challenges like the dotcom bubble and Covid, it has consistently grown. While Gen AI presents opportunities, its impact will vary across companies. Infosys and Persistent Systems are early leaders, leveraging partnerships and AI capabilities. ADVERTISEMENT With respect to Infosys, what are you building in and with respect to the commentary, which are going to be the key monitorables for you? Kumar Rakesh: For Infosys we are building relatively stronger growth. In the context of the rest of the IT services, we think Infosys will outperform the larger pack. We are building more than 2% sequential constant currency growth for the company this quarter, which will be partly helped by the acquisition they have done. About 40 bps of growth is coming in from that. But even excluding that, on an organic basis, we are expecting them to report a healthy 1.6-1.7% sequential growth and that is partly on a very low base of the last quarter. The fourth quarter for Infosys was quite weak and hence coming from a low base, it should see a pick-up in growth. Europe revival helps IT survive quarter shocks From the management commentary perspective, this earning season started with TCS results and sparked off the debate on whether the IT services demand has started seeing a material hit and the possibility of Indian IT services companies seeing a bigger impact compared to some of the larger peers because the commentary which was coming in from the global peers was far more stable demand environment. IT companies tighten belt as AI, macro headwinds squeeze biz margins Through the earning season we have seen that the commentary largely converges towards a stable demand environment. and we would look out for a similar commentary from Infosys as well, that the demand environment continues to be stable. Obviously the guidance as well built in the commentary and the guidance the company is going to give will be the key focus. What is the pecking order right now from the earnings that have come by? It was always TCS up there and then Infosys and then the rest? Has it changed at all after TCS and Wipro's earnings? Kumar Rakesh: At the start of this month as with our preview, we had shifted our preference to Infosys over TCS. Part of the reason was that we do see that TCS has some of the client specific issues that they will have a more lumpy performance. That being said, TCS valuation continues to look very attractive and hence there is a downside support in the TCS valuation. That being said, from an outperformance perspective, Infosys could be a preferred pick partly because of valuation, but more importantly because of the growth outperformance that the company can deliver compared to most of the other largecaps peers in the sector. The margin also seems to be relatively more stable. We will have to watch out how that goes out today as they report, but our expectation would be that Infosys earnings growth will be among the highest this year compared to most of the other largecaps. ADVERTISEMENT In terms of the verdict that we have seen across the board from largecap IT companies, so far we have either seen a miss versus estimates or inline, nothing positive. When do you see this trend reversing because of the guidance that some of the companies have given? It does say that by the end of FY26, they can expect some improvement in margins. When do you see this turnaround coming about? Kumar Rakesh: You are right as the general performance in this quarter has been quite mixed, largely a disappointment on growth as well as margin for many of the companies. A lot depends on the macro, how the US macroeconomic environment starts shaping up. There is a fair degree of uncertainty. A lot of tariff related deals are yet to be signed and that continues to elevate the uncertainty especially in verticals related to retail, CPG, and manufacturing, especially automotive within manufacturing and hence those verticals could take longer to start showing any recovery. The good news is that the BFSI vertical continues to remain quite resilient. Some of the companies even called out that even discretionary spending in that vertical seems to be there and hence that is one vertical which will continue to drive the growth for the next couple of quarters. ADVERTISEMENT Technology services is another vertical which seems to have bottomed out and started showing good results. These are the two verticals which we would expect to start driving growth and as the macro starts improving, the trade deal starts getting signed, and the uncertainty starts coming down, hopefully by the end of this financial year we would expect the rest of the verticals also to start showing recovery. Once that happens, the FY27 essentially becomes the recovery year which is where investors will start focusing on that this financial year will largely be a low single-digit growth year but can we get to the mid-single-digit sort of a growth in FY27 and if that happens, then the interest will start coming back into IT. ADVERTISEMENT The other debate that is going on is with respect to the AI transition because some experts believe that for Indian IT companies they have managed to get hold of the latest in technology and this can hold true yet again as well. Help us analyse your take on how Indian IT companies are expected to win this particular race. Do you believe that Indian companies are well versed to go ahead and adapt this particular technology and also which companies can actually be the early winners over here? Kumar Rakesh: We agree that if you go back over the last two-three decades starting from the mainframe transition, then the digital technologies came, a lot of SaaS players also came in the market. Through all those transitions the expectation was that the Indian IT services industry will take a hit but that did not happen. Over the last three decades, we have never seen Indian IT services industry's revenue to decline on a year-on-year basis, and so that is quite a resilient performance and during this period of time we have seen the dotcom bubble, the global financial crisis, multiple macroeconomic challenges in Europe and US and the Covid. Through all these phases, the Indian IT services industry has shown quite a strong resilience in its performance. We also believe it is a quite resilient industry. That being said, while we believe that on the Gen AI side, it would be net neutral to net positive for the industry, there would be some, who would be a bigger beneficiary and there could be some companies that could see a negative impact as well. It's a little too early to start calling out who would be negatively impacted but there are some companies who have started taking some initial lead in terms of building capability on Gen AI and also participating in clients' transformation on the Gen AI journey. Infosys is one of those companies. They have done a fair bit of work. Their partnership with some of the global Gen AI leading hardware companies as well as their own capability on the small language model which they are building seems to be quite promising. The other company which stands out is Persistent Systems that is doing a fair degree of work on Gen AI and some degree of that decoupling between the employee growth and revenue growth is already visible in that company. So, these are a couple of companies which have already started showing some initial signs of leadership outperformance from the Gen AI side and we are closely tracking that. ADVERTISEMENT (You can now subscribe to our ETMarkets WhatsApp channel)

Mapping future: AI, electrification and automated testing drive auto tech forward
Mapping future: AI, electrification and automated testing drive auto tech forward

Time of India

time29 minutes ago

  • Time of India

Mapping future: AI, electrification and automated testing drive auto tech forward

As the automotive world accelerates towards a smarter, cleaner and more connected future, this week's developments reveal a sector embracing complexity with innovation. From Yamaha's multi-pathway approach to electrification, to India's growing AI-powered EV ecosystem, and a global call for tighter oversight on autonomous technologies — the landscape is rapidly evolving. ETAuto brings you a curated round-up of key trends, technologies and transformations shaping mobility's next chapter. Electrification Is Just One Lane in Yamaha's Growth Roadmap Yamaha is exploring a diversified approach to growth in emerging markets, with electrification forming just one part of its broader strategy. Rather than a one-size-fits-all push, the company is balancing ICE, hybrid and EV options tailored to market needs. Read more Smarter Infrastructure: Maharashtra Launches Six Automated Vehicle Testing Stations Rosmerta Technologies has launched six automated vehicle inspection centres in Maharashtra to streamline and modernise vehicle compliance and roadworthiness checks, reinforcing the push toward digital governance in mobility. Read more AI in Motion: India's EV Ecosystem Attracts Smart Tech Players MediaTek is positioning itself as a key enabler of India's next-gen EVs, offering AI and high-performance computing platforms tailored to the country's unique needs. Meanwhile, Spyne has launched an AI assistant, VINNIE, designed to boost operational efficiency for used car dealerships. MediaTek targets Indian EV market with AI Spyne introduces VINNIE AI assistant Global Spotlight: Oversight Tightens on Self-Driving Tech In the US, a federal auto safety nominee has called for stricter oversight of autonomous driving systems, amid growing concerns around reliability and transparency. The move follows a Tesla driver's testimony that Autopilot failed to prevent a fatal crash. US auto safety nominee urges active oversight Tesla driver says Autopilot failed in fatal crash Mapping Smarter Roads: Genesys Integrates DIGIPIN in 2D & 3D Maps Genesys International has integrated DIGIPIN technology into its national mapping solutions, enhancing the precision and interactivity of 2D and 3D spatial data across India. This upgrade supports smarter urban planning, logistics, and autonomous navigation frameworks. Read more For insights into the fast-evolving automotive tech space, follow ETAuto for weekly analysis, trends, and deep dives. We'd love to hear what you think about this edition of the newsletter! Your feedback and suggestions help us improve and deliver content that matters to you.

Samsung unveils new premium experience store in Mumbai
Samsung unveils new premium experience store in Mumbai

Time of India

timean hour ago

  • Time of India

Samsung unveils new premium experience store in Mumbai

Samsung has unveiled a new Premium Experience Store at Lotus Trade Centre, Andheri West, further strengthening its retail presence in India. Spanning 1,600 sq. ft., the store offers a hands-on showcase of Samsung's latest innovations across mobile, wearables, and smart home technology . The new Samsung store features dedicated zones showcasing Samsung's full range of Galaxy devices, including the latest smartphones, tablets, laptops, smartwatches, smart rings, and the advanced SmartThings ecosystem. Customers can also access Samsung Store+, a digital interface that allows seamless browsing and home delivery of products. "At Samsung, we are dedicated to bringing innovation closer to our customers through inspiring retail experiences," stated Sumit Walia, Vice President, Head of D2C Business & Corporate Marketing at Samsung India. "The launch of our premium experience store in Andheri West, Mumbai, marks a significant milestone in our journey to expand our premium retail footprint and create holistic, all-in-one destinations for technology, engagement, and service." As part of its 'Learn @ Samsung' initiative, the store will host regular workshops focused on: AI-powered photography, Digital creativity and doodling, Productivity hacks using Galaxy devices. These sessions aim to help younger consumers unlock the full potential of their tech tools. Along with this, the store also comprises of a full-fledged service centre for post-purchase support and repairs. To celebrate the launch, Samsung is offering exclusive Paytm First benefits, including: * Over 30 free subscriptions to OTT, music, wellness, and infotainment platforms * Discounts on 40+ brand gift cards and 25+ premium deals * Buy-1-Get-1-Free buffet offers at 100+ restaurants nationwide * Special travel and dining perks AI Masterclass for Students. Upskill Young Ones Today!– Join Now

DOWNLOAD THE APP

Get Started Now: Download the App

Ready to dive into a world of global content with local flavor? Download Daily8 app today from your preferred app store and start exploring.
app-storeplay-store