The Trust Deficit In Artificial Intelligence

News18

3 days ago

Unlike traditional engineering systems, ML models rarely provide error bounds or correctness guarantees.
Artificial Intelligence (AI) and Machine Learning (ML) have experienced transformative growth over the past fifteen years, propelled by advancements such as deep learning, convolutional neural networks, transformer architectures, and the emergence of large language models (LLMs). These innovations have enabled machines to comprehend and generate language and images in previously unimaginable ways. The transformer architecture, introduced in 2017, has revolutionised image analysis and synthesis, as well as natural language processing, by incorporating attention mechanisms that capture long-range context. This development has led to LLMs like ChatGPT, Llama, and Gemini, capable of language translation, summarisation, and content generation. Often trained on extensive internet-scale data, these models are adaptable to new tasks through prompt tuning. These technologies now power applications in diverse domains including healthcare, law, education, creative arts, business, public services, and autonomous systems. Numerous Indian agencies and startups are also leveraging AI systems using local context for various applications.
Despite their remarkable capabilities, these models have significant limitations, particularly in terms of reliability and fairness. Unlike traditional engineering systems, ML models rarely provide error bounds or correctness guarantees. Their accuracy is typically validated only on data similar to the training set, not on real-world, dynamic environments. The assumption that deployment data will resemble training data frequently proves incorrect. Real-world data changes over time and circumstances (known as distribution shift), and detecting or adapting to these changes is challenging, especially post-deployment.
In many applications, especially those involving unstructured data like images or text, defining the space of possible inputs is difficult. This makes probabilistic guarantees impossible and external validation challenging. Additionally, generative models are trained for coherence rather than factual accuracy, leading them to 'hallucinate" plausible-sounding but false information. Such failures are hard to detect, particularly for non-expert users, posing risks in high-stakes and safety-critical fields like medicine or autonomous systems.
Moreover, the internal representations used by ML systems differ significantly from human cognition, as illustrated by adversarial attacks where minor, imperceptible changes to input can cause confident misclassifications, revealing the brittleness of these systems under slight perturbations. The disparity between human and machine failure points complicates the ethical deployment of AI systems in critical applications.
Regarding fairness, AI systems often inherit or amplify social biases embedded in the data. Even if sensitive attributes like caste, gender, or religion are not explicitly included, models can learn proxy variables that lead to discriminatory outcomes, known as disparate impact. In diverse societies like India, where large sections of the population are digitally underrepresented, this issue is particularly problematic. Attempts to mitigate bias through data preprocessing or algorithmic corrections often reduce model accuracy without guaranteeing fairness. Research indicates that under realistic conditions, it is mathematically impossible to ensure equal fairness for all groups simultaneously.
In conclusion, while AI holds immense potential for enhancing productivity, inclusion, and creativity, its application in public-facing scenarios demands extreme caution. These systems are best utilised where critical thinking can moderate their outputs, such as in exploratory research or informed personal use. Broader deployment requires rigorous monitoring, context-aware evaluation, and strong institutional oversight to ensure these powerful tools do not become sources of harm or inequality.
The author is Head of Department of Computer Science, and Centre for Digitalisation, AI and Society at Ashoka University. Views expressed in the above piece are personal and solely that of the author. They do not necessarily reflect News18's views.
view comments
First Published:
July 16, 2025, 13:43 IST
Disclaimer: Comments reflect users' views, not News18's. Please keep discussions respectful and constructive. Abusive, defamatory, or illegal comments will be removed. News18 may disable any comment at its discretion. By posting, you agree to our Terms of Use and Privacy Policy.