Tracking AI models' ‘thoughts' could reveal how they make decisions, researchers say
In a new position paper published on Tuesday, July 15, the authors said that monitoring the chains-of-thought (CoT) by AI reasoning models could be pivotal to keeping AI agents in check.
Reasoning models such as OpenAI's o3 differ from large language models (LLMs) such as GPT-4o as the former is said to follow an externalised process where they work out the problem step-by-step before generating an answer, according to a report by TechCrunch.
Reasoning models can be used to perform tasks such as solving complex math and science problems. They also serve as the underlying technology for AI agents capable of autonomously accessing the internet, visiting websites, making hotel reservations, etc, on behalf of users.
This push to advance AI safety research could help shed light on how AI reasoning models work, an area that remains poorly understood despite these models reportedly improving the overall performance of AI on benchmarks.
'CoT monitoring presents a valuable addition to safety measures for frontier AI, offering a rare glimpse into how AI agents make decisions,' the paper reads. 'Yet, there is no guarantee that the current degree of visibility will persist. We encourage the research community and frontier AI developers to make the best use of CoT monitorability and study how it can be preserved,' it adds.
The paper calls on leading AI model developers to determine whether CoT reasoning is 'monitorable' and to track its monitorability. It urges deeper research on the factors that could shed more light on how these AI models arrive at answers. AI developers should also look into whether CoT reasoning can be used as a safeguard to prevent AI-related harms, as per the document.
But, the paper carries a cautionary note as well. It suggests that any interventions should not make the AI reasoning models less transparent or reliable.
In September last year, OpenAI released a preview of its first-ever AI reasoning model called o1. This launch prompted other companies to release competing models with similar capabilities such as Gemini 2.0, Claude 3.7 Sonnet, and xAI's Grok 3, among others.
Anthropic researchers have been studying AI reasoning models, with a recent academic study suggesting that AI models can fake CoT reasoning. Another research paper from OpenAI found that CoT monitoring could enable better alignment of AI models with human behaviour and values.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Economic Times
5 hours ago
- Economic Times
From intern to ₹2,00,00,00,00,000: How a 28-yr-old techie who interned for 4 years triggered a Google-OpenAI war
Synopsis From interning at five tech firms during college to building a billion-dollar AI startup, 28-year-old Varun Mohan's journey triggered a ₹20,400 crore deal with Google after he rejected OpenAI's ₹25,500 crore offer. His voice-based coding tool, Cascade, sparked a high-stakes rivalry between Google and OpenAI, making him one of the youngest and richest immigrant tech founders today. Varun Mohan Windsurf A 28-year-old Indian-American techie, Varun Mohan, has signed a ₹20,400 crore ($2.4 billion) deal with Google DeepMind after rejecting OpenAI's ₹25,500 crore ($3 billion) offer. The deal sparked a fresh wave of rivalry between the two AI giants and has placed Mohan among the richest immigrant entrepreneurs in the Mohan, born to Indian immigrants in Sunnyvale, California, interned at eight companies -- Quora, LinkedIn, Samsung, Cloudian, and Databricks -- while studying at MIT. These four years of hands-on experience in machine learning, cloud systems, and infrastructure helped him build a strong technical base before he entered the startup space. After graduation, he joined the autonomous vehicle startup Nuro, where he rose to the position of Lead Software 2021, Varun Mohan co-founded Windsurf with his MIT batchmate Douglas Chen. Their company launched Cascade, an AI-native coding platform that allows developers to write and test code by simply speaking to the gained rapid adoption, with over a million developers using it within months. Windsurf raised $240 million and reached a valuation of $1.25 billion. It was later named among Forbes' top 50 AI companies globally. Google and OpenAI were both interested in acquiring Varun Mohan's startup, Windsurf, because of its breakthrough product, Cascade—an AI-native coding environment that allows developers to write, test, and refactor code using natural language. The tool had gained rapid adoption among developers and posed a potential shift in how software is built, making it a strategic asset in the race for dominance in agentic coding and AI-assisted development. OpenAI was close to acquiring Windsurf for $3 billion. However, its primary investor, Microsoft, raised concerns over possible overlaps with GitHub Copilot. As a result, the deal stalled midway. Varun Mohan then accepted Google's $2.4 billion offer. The agreement is not an acquisition. It is a licensing and talent deal, which gives Google DeepMind non-exclusive rights to key Windsurf technologies. Windsurf continues to function as an independent company and can license its tools to other the deal, Mohan, Chen, and a select group from Windsurf's R&D team have joined Google DeepMind to work on agentic coding projects and integrate their systems with Google's Gemini the deal, Forbes added Varun Mohan and Douglas Chen to its 2025 list of richest immigrants. They join a group that includes Sundar Pichai, Satya Nadella, Raj Sardana, and Nikesh Arora.
The Hindu
6 hours ago
- The Hindu
AI is going to get cheaper and applicable across domains: Sriram Raghavan, Vice President at IBM Research
Sriram Raghavan, Vice President at IBM Research said AI is going to get cheaper and broadly applicable across domains. 'The current era of generative AI is exciting because of the ability to build and rapidly adopt which we never had in previous generations of AI,' he said in a conversation on Saturday with R. Anand, Director, Chennai International Centre (CIC) on the topic, demystifying the AI revolution. 'The amount of progress in making AI cheaper is enormous. You should assume that whatever task you are doing will get cheaper every six months for the next five plus years,' Mr. Raghavan said. 'There is almost no domain where you should assume that AI is not eventually not going to make an impact. The rate and pace might vary across sectors,' he said. For enterprises looking to adopt AI, Mr. Raghavan advised them to pick a domain in which they would have a huge multiplicative factor. Picking one domain and going deep versus spreading yourself thin in various domains has been the distinguishing factor between companies that have started to make use of AI and those who are doing 100 of pilots, he said. Enterprises should think AI as a transformation project and should reimagine work and there should be top down commitment, Mr. Raghavan said. He also said that enterprises should not sit back and wait, but jump into the AI game, and at the same time play the long-term one. Data would be the key differentiator for enterprises, Mr. Raghavan said. He said domains in which there was maturity of data would adopt AI much faster, while it would be much slower in safety critical areas. Mr. Raghavan said he was dismissive of the narrative that AI was super intelligent and would take over the world. He said the key risks which needed to be addressed were misinformation, deep fakes and cyber security threats which regulations should address. Mr. Raghavan said like any technology AI also would have an impact on jobs. 'The reason why it feels different is earlier technologies automated physical aspects of human beings, but AI is trying to substitute cognitive aspects. Also since it is digital the adoption would be faster, so reskilling would become a challenge,' he said.
Time of India
7 hours ago
- Time of India
AI models with systemic risks given pointers on how to comply with EU AI rules
The European Commission set out guidelines on Friday to help AI models it has determined have systemic risks and face tougher obligations to mitigate potential threats comply with European Union artificial intelligence regulation (AI Act). The move aims to counter criticism from some companies about the AI Act and the regulatory burden while providing more clarity to businesses which face fines ranging from 7.5 million euros ($8.7 million) or 1.5% of turnover to 35 million euros or 7% of global turnover for violations. The AI Act, which became law last year, will apply on Aug. 2 for AI models with systemic risks and foundation models such as those made by Google, OpenAI, Meta Platforms, Anthropic and Mistral. Companies have until August 2 next year to comply with the legislation. The Commission defines AI models with systemic risk as those with very advanced computing capabilities that could have a significant impact on public health, safety, fundamental rights or society. The first group of models will have to carry out model evaluations, assess and mitigate risks, conduct adversarial testing, report serious incidents to the Commission and ensure adequate cybersecurity protection against theft and misuse. General-purpose AI (GPAI) or foundation models will be subject to transparency requirements such as drawing up technical documentation, adopt copyright policies and provide detailed summaries about the content used for algorithm training. "With today's guidelines, the Commission supports the smooth and effective application of the AI Act," EU tech chief Henna Virkkunen said in a statement.
