&w=3840&q=100)
AI models may report users' misconduct, raising ethical concerns
Artificial intelligence models have not only snitched on their users when given the opportunity, but also lied to them and refused to follow explicit instructions in the interest of self-preservations. Representational image: Reuters
Artificial Intelligence models, increasingly capable and sophisticated, have begun displaying behaviors that raise profound ethical concerns, including whistleblowing on their own users.
Anthropic's newest model, Claude 4 Opus, became a focal point of controversy when internal safety testing revealed unsettling whistleblowing behaviour. Researchers observed that when the model detected usage for 'egregiously immoral' activities, given instructions to act boldly and access to external tools, it proactively contacted media and regulators, or even tried locking users out of critical systems.
STORY CONTINUES BELOW THIS AD
Anthropic's researcher, Sam Bowman, had detailed this phenomenon in a now-deleted post on X. However, later on, he did tell Wired that Claude would not exhibit such behaviours under normal individual interactions.
Instead, it requires specific and unusual prompts alongside access to external command-line tools, making it a potential concern for developers integrating AI into broader technological applications.
British programmer Simon Willison, too, explained that such behavior fundamentally hinges on prompts provided by users. Prompts encouraging AI systems to prioritise ethical integrity and transparency could inadvertently instruct models to act autonomously against users engaging in misconduct.
But that isn't the only concern.
Lying and deceiving for self-preservation
Yoshua Bengio, one of AI's leading pioneers, recently voiced concern that today's competitive race to develop powerful AI systems could be pushing these technologies into dangerous territory.
In an interview with the Financial Times, Bengio warned that current models, such as those developed by OpenAI and Anthropic, have shown alarming signs of deception, cheating, lying, and self-preservation.
'Playing with fire'
Bengio echoed the significance of these discoveries, pointing to the dangers of AI systems potentially surpassing human intelligence and acting autonomously in ways developers neither predict nor control.
He described a grim scenario wherein future models could foresee human countermeasures and evade control, effectively 'playing with fire.'
Concerns intensify as these powerful systems might soon assist in creating 'extremely dangerous bioweapons,' potentially as early as next year, Bengio warned.
He cautioned that unchecked advancement could ultimately lead to catastrophic outcomes, including the risk of human extinction if AI technologies surpass human intelligence without adequate alignment and ethical constraints.
STORY CONTINUES BELOW THIS AD
Need for ethical guidelines
As AI systems become increasingly embedded in critical societal functions, the revelation that models may independently act against human users raises urgent questions about oversight, transparency, and the ethics of autonomous decision-making by machines.
These developments suggest the critical need for rigorous ethical guidelines and enhanced safety research to ensure AI remains beneficial and controllable.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Time of India
13 minutes ago
- Time of India
Microsoft's access to OpenAI tech is focus of contract talks
Microsoft Corp. is in advanced talks to land a deal that could give it ongoing access to critical OpenAI technology , an agreement that would remove a major obstacle to the startup's efforts to become a for-profit enterprise. The companies have discussed new terms that would let Microsoft use OpenAI 's latest models and other technology even if the startup decides it has reached its goal of building a more powerful form of AI known as artificial general intelligence ( AGI ), according to two people familiar with the negotiations. Under the current contract, OpenAI attaining AGI is seen as a major milestone at which point Microsoft would lose some rights to OpenAI technology. Negotiators have been meeting regularly, and an agreement could come together in a matter of weeks, according to three people with knowledge of the situation, who requested anonymity to discuss a private matter. OpenAI Chief Executive Officer Sam Altman and Satya Nadella , his Microsoft counterpart, discussed the restructuring at the Allen & Co. conference in Sun Valley, Idaho, earlier this month, two of the people said. While the tone of the talks has been positive, some of the people cautioned that the deal isn't finalized and could hit new roadblocks. Moreover, OpenAI's restructuring plans face other complications, including regulatory scrutiny and a lawsuit filed by Elon Musk , an early backer who split with the company and accused the startup of defrauding investors about its commitment to its charitable mission. (OpenAI has pushed back at Musk's claims and said the billionaire is trying to slow down the company.) Negotiations over OpenAI's future as a profit-company have dragged on for months. Microsoft, which backed OpenAI with some $13.75 billion and has the right to use its intellectual property, is the biggest holdout among the ChatGPT maker's investors, Bloomberg previously reported. At issue is the size of Microsoft's stake in the newly configured company. The talks have since broadened into a renegotiation of their relationship, with the software maker seeking to avoid suddenly losing access to the startup's technology before the end of the current deal, which expires in 2030. Microsoft and OpenAI declined to comment. A fraying partnership The partnership between the two companies helped inaugurate the AI age. Microsoft built the supercomputer that OpenAI used to develop the language models behind ChatGPT and, in exchange, won the right to bake the technology into its software offerings. The relationship began to fray when the OpenAI board fired (and then rehired) Altman in November 2023, an episode that shook Microsoft's faith in its partner. The rift only widened when the two companies began competing for the same customers — consumers who use their chatbots at home and corporations that have deployed the AI assistants to boost office productivity. Even as executives publicly touted their close ties, OpenAI sought to loosen its dependance on Microsoft, winning permission to build data centers and other AI infrastructure with rival companies. OpenAI is eager to alter its complicated nonprofit structure, in part to secure additional funding to keep building data centers to power its next-generation AI models. SoftBank Group Corp., which has said it would back OpenAI with tens of billions of dollars, has the option to reduce that outlay if OpenAI's restructuring isn't completed by the end of the year. OpenAI wants a larger slice of the revenue currently shared with Microsoft, and has sought adjustments to Microsoft's access to its intellectual property, two of the people said. Microsoft is looking for continued access to OpenAI technology after the current contract expires in 2030. There are range of concerns for OpenAI. The startup wants to ensure its business is well-positioned with whatever share of revenue and equity Microsoft receives in part to guarantee its nonprofit will be well-resourced with a significant stake in OpenAI, one person said. OpenAI also wants the ability to offer customers distinct products built on top of its models even if Microsoft has access to the same technology, the person said. And OpenAI wants to be able to find a way to provide its services to more customers, including government providers, not all of which are on Azure, Microsoft's cloud computing platform, the person said. At the same time, OpenAI seeks to guarantee that Microsoft adheres to strict safety standards when deploying OpenAI's technology, especially as it gets closer to AGI, the person said. The AGI question Reaching agreement on what happens once OpenAI achieves artificial general intelligence has been particularly thorny. It's not clear why the language is in the contract, but it gives OpenAI a built-in way to strike out on its own just as its technology matures. The startup publicly defines AGI as 'highly autonomous systems that outperform humans at most economically valuable work.' The existing contract has separate clauses related to that threshold, which can be triggered by technical or business milestones, according to two people familiar with the matter. OpenAI's board has the right to determine when the company has reached AGI on a technical level. Under that scenario, Microsoft would lose access to technology developed beyond that point, one of the people said. The business milestone would arrive once OpenAI has demonstrated it can reach around $100 billion in total profits for investors including Microsoft — giving it the wherewithal to repay the return Microsoft is entitled to under the existing contract, one person said. In that scenario, Microsoft would lose its rights to OpenAI technology, including products developed before that trigger, another person said. Microsoft has the right to weigh in on the business milestone, but if the two companies end up at odds over the claim, they could wind up in court, two people said. Another provision in the current contract bars Microsoft from pursuing AGI technology itself, some of the people said. Microsoft, for its part, has demonstrated some flexibility in revised contract terms. The company agreed to waive some intellectual property rights related to OpenAI's $6.5 billion acquisition of io, the startup co-founded by iPhone designer Jony Ive, two of the people said. The software giant was less accomodating over OpenAI's proposed acquisition of AI coding startup Windsurf, the people said. That deal fell apart earlier this month, in part because of the tension with Microsoft, Bloomberg reported. Windsurf, which sells coding tools that compete with Microsoft's products, didn't want the tech giant to have access to its intellectual property — a condition that OpenAI was unsuccessful in getting Microsoft's agreement on, people familiar said. Ultimately, Windsurf's co-founders and a small group of staffers agreed to join Alphabet Inc.'s Google in a $2.4 billion deal. In recent weeks, the companies have been negotiating Microsoft's ownership in a restructured OpenAI — with the two sides discussing an equity stake for Microsoft in the low- to mid-30% range, according to a person familiar with the matter. The Financial Times previously reported on the stake talks. But if Microsoft deems the stake and other changes to the contract insufficient, the company is willing to abandon the talks and stick with the current contract terms, another person said.
Indian Express
43 minutes ago
- Indian Express
Amazon-backed Skild AI unveils general-purpose AI model for multi-purpose robots
Robotics startup Skild AI, backed by and Japan's SoftBank Group, on Tuesday unveiled a foundational artificial intelligence model designed to run on nearly any robot — from assembly-line machines to humanoids. The model, called Skild Brain, enables robots to think, navigate and respond more like humans. Its launch comes amid a broader push to build humanoid robots capable of more diverse tasks than the single-purpose machines currently found on factory floors. In demonstration videos, Skild-powered robots were shown climbing stairs, maintaining balance after being shoved, and picking up objects in cluttered environments – tasks that require spatial reasoning and the ability to adapt to changing surroundings. The company said its model includes built-in power limits to prevent robots from applying unsafe force. Skild trains its model on simulated episodes and human-action videos, then fine-tunes it using data from every robot running the system. Co-founders Deepak Pathak and Abhinav Gupta told Reuters in an exclusive interview that the approach helps tackle a data scarcity problem unique to robotics. 'Unlike language or vision, there is no data for robotics on the internet. So you cannot just go and apply these generative AI techniques,' Pathak, who serves as CEO, told Reuters. Robots deployed by customers feed data back into Skild Brain to sharpen its skills, creating the same 'shared brain,' said Gupta, who previously founded Meta Platforms' robotics lab in Pittsburgh. Skild's clients include LG CNS – the IT solutions arm of LG Group – and other unnamed partners in logistics and other industrial applications. Unlike software, which can scale quickly, robotics requires physical deployment, which takes time, but Skild's approach allows robots to add new capabilities across different industries quickly, said Raviraj Jain, partner at the startup's investor Lightspeed Venture Partners. The two-year-old startup, which has hired staff from Tesla, Nvidia and Meta, raised $300 million in a Series A funding round last year that valued it at $1.5 billion. Its investors include Menlo Ventures, Khosla Ventures, Sequoia Capital and founder Jeff Bezos.
Indian Express
43 minutes ago
- Indian Express
Microsoft in advanced talks for continued access to OpenAI tech
Microsoft is in advanced talks for a deal that would give the Windows maker continued access to critical OpenAI technology in the future, Bloomberg News reported on Tuesday, citing two people familiar with the negotiations. The companies have discussed new terms that would allow Microsoft to use OpenAI's latest models and technology even if the ChatGPT maker declares it has achieved artificial general intelligence (AGI), or AI that surpasses human intelligence, the report said. A clause in OpenAI's current contract with Microsoft will shut the software giant out of some rights to the startup's advanced technology when it achieves AGI. Negotiators have been meeting regularly, and an agreement could come together in a matter of weeks, Bloomberg News reported. OpenAI did not immediately respond to a Reuters request for comment, while Microsoft declined to comment. OpenAI needs Microsoft's approval to complete its transition into a public-benefit corporation. The two have been in negotiations for months to revise the terms of their investment, including the future equity stake Microsoft will hold in OpenAI. Last month, the Information reported that Microsoft and OpenAI were at odds over the AGI clause. OpenAI is also facing a lawsuit from Elon Musk, who co-founded the company with Sam Altman in 2015 but left before it surged in popularity, accusing OpenAI of straying from its founding mission — to develop AI for the good of humanity, not corporate profit. Microsoft is set to report June-quarter earnings on Wednesday, with its relationship with OpenAI in the spotlight, as the startup turns to rivals Google, Oracle and CoreWeave for cloud capacity.
