How an AI bot cracked the tough NEET exam, beating most human scores

Business Standard

15 hours ago

The results of India's competitive medical entrance examination — the National Eligibility cum Entrance Test (NEET) 2025 — have recently been announced, but it wasn't just human aspirants in the race this year. As students tackled the high-stakes paper, Allen, one of the country's leading coaching institutes, quite literally put its AI-powered bot 'Allie' to the test.
As a result, while Allen's top-performing student, Tanmay Jagga, secured an All India Rank (AIR) of 74 with 650 out of 720 marks, Allie outshone him by some margin with 678 marks, a score likely to fetch a candidate a top-10 rank and a seat at the prestigious All India Institute of Medical Sciences (AIIMS), Delhi, according to the coaching institute.
Thankfully, the bot has not been developed to directly compete with students but to build trust in the AI-driven complimentary support system offered to students for academic problem-solving. "We are pedagogically and result-wise quite sound but there was always a scope for improvement. We realised that apart from in-class learning, a lot happens outside of class where students practice, have doubts, and get stuck," said Ankit Khurana, Allen's chief product officer on the company's motivation behind the bot. "In this journey, students face several challenges but teachers can't be a support for every student. So, we thought our digital platform could be a companion for students. With that philosophy, we created 'Allie'."
According to data shared by the institute's chief technology officer Saurabh Tandon, over 90,000 students used the bot to solve nearly 600,000 queries last month. Out of these, roughly 80 per cent of doubts were immediately responded to while the rest were transferred to a human teacher for resolution.
Allie, who somewhat resembles ChatGPT in appearance, answers the questions within seconds but for complex questions, such as those comprising images, it can take up to three minutes, Tandon said. The bot was initially launched in December 2024 and was trained for biology subjects but by March 2025, it covered almost the entire syllabus for the NEET examination, barring some topics like organic chemistry. That omission could explain why Allie got two questions wrong and didn't attempt eight questions in the NEET exam.
As for the underlying technology, the company said it relies on a proprietary verifier model and a dynamic routing system. While a proprietary verifier model refers to the use of targeted strategies to verify answers by cross-referencing them with curated study content, including theory and Q&A (question and answers) archives, the dynamic routing system sends queries to a suitable language model based on the problem type (image-based, text-based, or reasoning-based question), and other metrics.
Going forward, the company plans to train the bot with the NEET examination syllabus in its entirety while also assisting students preparing for other examinations such as the Joint Entrance Examination (JEE) for admission to undergraduate engineering programs. In the ecosystem of Allen, around 5 lakh students prepare for JEE and NEET examinations as paid subscribers.
"The interaction with Allie has grown roughly six times in the last six months, that's something which we did not expect. The amount of interaction that happened was unprecedented. As we improve the technology, we will do the same for JEE examination. We will also keep on refining the bot for NEET in terms of the quality of answers and conversation," Khurana said.