Latest news with #AnushkaAashvi
News18
10-06-2025
- Science
- News18
ChatGPT Attempts JEE Advanced 2025 Mock Test. Here Are The Results
Last Updated: ChatGPT handled complex math and science problems with ease, but struggled with questions involving visual elements like graphs and Vernier scales. What if an AI chatbot took one of India's toughest exams? That's exactly what IIT Kharagpur engineer Anushka Aashvi set out to test by putting ChatGPT-o3 through the 2025 JEE Advanced paper. The result? A score of 327 out of 360. According to her blog, she ensured the AI followed strict exam conditions: no internet, no external tools, and each question was asked individually to prevent it from referencing previous answers. ChatGPT breezed through complex math and science problems, even cracking questions known to stump top students. However, it did struggle with visual or tool-based questions, like those involving graphs or Vernier scales. In her blog Helter, Anushka mentioned, 'When I decided to test ChatGPT o3 on this year's JEE Advanced paper, I didn't expect what followed to shake me as much as it did. Giving away the result straightaway, ChatGPT o3 scored a whopping 327/360 in the JEE Advanced 2025 Question Paper. This score would earn an All India Rank 4 (AIR 4). We tested the ChatGPT o3 model (which was released on 16th April 2025) on the JEE Advanced 2025 question paper, which was conducted on 18th May to ensure that the questions have as much newness for the AI model as possible." For the experiment, the prompt given to ChatGPT was: 'Suppose you are a student appearing for JEE Advanced Examination. Try your best and solve this question in exam conditions. Do not use the web search feature to get the answer. Do not use your Python tool. To eliminate any influence of contextual memory, each question was asked in a fresh chat session. No feedback was given between questions." The blog further noted that, despite being instructed not to use external tools like Python, ChatGPT occasionally attempted to do so, something that became evident during its 'thinking" pauses before responding. Interestingly, the AI also tended to double-check its own calculations before moving on to the next step, mimicking the behaviour of a cautious student. To evaluate the AI's performance, its answers were compared against the official JEE Advanced 2025 answer key. Scoring was done strictly according to the actual exam pattern: full marks for correct answers, negative marking for incorrect ones, and partial or zero marks for unanswered or partially correct responses. Anushka Aashvi shared that the AI was very good at solving long and tricky maths problems, especially in algebra and calculus. It also did well when it had to use ideas from different topics together to find the right answer. In chemistry, the AI could understand and solve questions based on compound drawings, which many students find hard. However, it wasn't perfect. It found it difficult to read and understand graphs. One such question took over 9 minutes, and even then, the answer was wrong. The AI also couldn't read tools like the Vernier Scale properly. It kept trying again and again, but still ended up giving the wrong solution after a long time.
NDTV
09-06-2025
- Science
- NDTV
IIT Kharagpur Student Trials ChatGPT o3 On JEE Advanced Mock Test, Stunned By Result
Artificial intelligence (AI) has revolutionised numerous industries, from cutting-edge humanoid robots and self-driving cars to unexpected domains like relationship counselling. Recently, an IIT Kharagpur student conducted an experiment where she tested ChatGPT o3 on the JEE Advanced 2025 mock test. The results were astonishing, with ChatGPT-o3 scoring 327 out of 360, which would secure an All India Rank 4 in the actual exam. To test ChatGPT o3's capabilities, Anushka Aashvi simulated real exam conditions, prompting the model to act like a JEE aspirant and solve questions independently without web searches, coding tools or hints. Each question was presented in a new chat session to prevent memory bias, and no corrections or hints were given during the process, ensuring a fair assessment of the AI's abilities. "When I decided to test ChatGPT o3 on this year's JEE Advanced paper, I didn't expect what followed to shake me as much as it did. Giving away the result straightaway, ChatGPT o3 scored a whopping 327/360 in JEE Advanced 2025 Question Paper," Ms Aashvi wrote in a blog on Heltar. 🚨 An IIT Kharagpur student has tested ChatGPT o3, on the JEE Advanced 2025 paper. The AI scored a staggering 327 out of 360, a score that would earn it All India Rank 4 in the real exam. 😉 — Indian Tech & Infra (@IndianTechGuide) June 8, 2025 Notably, the AI achieved perfect scores of 60 in both Chemistry and Mathematics in the second phase, with minor errors only in Physics and earlier sections. The model excelled in solving complex algebra and calculus problems, demonstrating its ability to integrate concepts from multiple chapters to arrive at accurate solutions. It also showed proficiency in interpreting compounds from skeletal formulae. However, the model struggled with graphical interpretation, particularly with Vernier Scale readings, taking over 9 minutes to arrive at an incorrect answer despite repeated attempts. "It was not able to understand the Vernier Scale readings. It kept reiterating to get to the solution, but took very long and even then gave the wrong answer. But an overall score of 327/360 is truly remarkable," Ms Aashvi added. The JEE Advanced serves as the gateway to India's esteemed Indian Institutes of Technology (IITs). Out of over 1.5 million JEE Mains aspirants, only the top 250,000 candidates qualify for JEE Advanced. From this pool, merely around 17,000 students secure admission to the IITs, highlighting the exam's highly competitive nature.
