29-04-2025
AI breakthrough: AlphaGeometry 2 and Symbolic AI outperform Maths Olympiad gold medallists
The world was astonished a year ago, but it is now shocked. Last year, AlphaGeometry, an AI problem solver developed by Google DeepMind, astonished the world by placing second in the International Mathematical Olympiad (IMO). The DeepMind team now claims that the performance of their improved system, AlphaGeometry 2, has surpassed that of the typical gold medallist. The findings are detailed in a preprint available on the arXiv service.
International Mathematical Olympiad
The International Mathematical Olympiad (IMO) is the world's most prominent mathematics competition. The inaugural competition, which took place in Romania in 1959 among seven Soviet Bloc nations, began to grow swiftly, reaching 50 nations in 1989 and surpassing 100 countries for the first time in 2009. The competition has always aimed to help school-age mathematicians improve their problem-solving abilities.
In India, the Homi Bhabha Centre for Science Education (HBCSE) organises the Mathematical Olympiad Programme on behalf of the National Board for Higher Mathematics (NBHM) of the Department of Atomic Energy (DAE), Government of India. The Indian team to compete in the international competition is chosen using a broad-based Indian Olympiad Qualifier in Mathematics (IOQM). For additional information, click here.
Questions are picked from four topic areas: algebra, combinatorics, geometry, and number theory, with no necessity or expectation that students can utilise calculus.
The competition consists of six problems. The tournament lasts two days and consists of three problems per day; each day, participants get four and a half hours to complete three questions. Each problem is worth 7 points, with a maximum of 42 points.
AI in the race
In 2024, the IMO was hosted in Bath, United Kingdom, with 609 high school students from 108 countries participating. Chinese student Haojia Shi finished first in the individual rankings with a perfect score- 42 points.
In the country rankings, the United States team came out on top, and China came in second. The human problem-solvers won 58 gold medals, 123 silver and 145 bronze. One of the event's highlights was the presence of two unofficial contestants: AlphaGeometry 2 and AlphaProof, both artificial intelligence algorithms built by Google DeepMind.
The two programs were able to solve four out of six tasks. Mathematician and Fields Medallist Timothy Gowers, a past IMO gold medallist, and mathematician Joseph K. Myers, another previous IMO gold medallist, evaluated the two AI systems' solutions using the same criteria as the human competitors. According to these standards, the programs received an excellent 28 points out of a potential 42 points, equivalent to a silver medal.
This means that the AI came close to earning a gold medal, which was granted for a score of 29 points or higher. Furthermore, just 60 pupils achieved higher completion scores. Furthermore, AlphaGeometry 2 solved the geometry problem correctly in just 19 seconds. Meanwhile, AlphaProof solved one number theory and two algebra problems, including one that only five human participants could figure out.
Training the tools
Training an AI requires a large quantity of data. AlphaProof's training was restricted by the amount of mathematical material accessible in a formal mathematical language. The DeepMind researchers then used the Gemini AI tool to translate millions of problems on the Internet that people have solved step-by-step in natural language into the Lean programming language, allowing the proof assistant to learn about them.
Using this huge data, AlphaProof was taught using reinforcement learning, as AI systems were taught to master chess, shogi, and Go previously. Reinforcement Learning (RL) is similar to instructing a dog. The dog (agent) learns tricks by performing actions (such as sitting). If it sits appropriately, it gets a treat (reward); otherwise, no treat. Over time, it learns which acts result in rewards.
Similarly, RL systems learn using trial and error to maximise rewards in tasks such as gaming or robotics. AlphaProof repeatedly competes with itself and improves step by step; if the process does not result in a win, it is penalised and learns to explore alternative techniques.
What works for number theory or algebra does not work in geometry, necessitating a new methodology. As a result, DeepMind created AlphaGeometry, a unique AI system designed to solve geometry difficulties.
The experts initially created an exhaustive list of geometric 'premises,' or basic building pieces of geometry, such as a triangle having three sides, and so on. Just as the Architect studies the design, AlphaGeometry's deduction engine algorithm evaluates the 'problems'. It picks the appropriate blocks (premises) assembled step by step to build the home (proof).
The AI was able to manipulate the geometric objects around a 2D plane, like adding a fourth point and converting a triangle into a quadrilateral or moving a point to change the triangle's height. The 'proof' is complete when all components fit together correctly, and the home is sturdy. Unlike trial-and-error learning (RL), this is equivalent to following an instruction manual with unlimited LEGO parts.
Going for gold
The DeepMind team has now produced an improved version, AlphaGeometry 2, that trains the model with more data and accelerates the process. The AI system is now able to solve linear equations.
With the upgrade, the AI was recently proved capable of answering 84% of all geometry problems set in IMOs during the last 25 years, compared to 54% for the previous version of AlphaGeometry. Future developments in AlphaGeometry will include dealing with mathematical problems containing inequalities and nonlinear equations, which will be necessary to solve geometry completely.
A team of researchers, including IIIT Hyderabad's Ponnurangam Kumaraguru, has made a breakthrough with their 'Symbolic AI', outperforming AlphaGeometry's capabilities. Furthermore, the hybrid symbolic AI, which complemented Wu's technique with AlphaGeometry, outperformed human gold medallists on IMO geometry problems.
This Symbolic AI system solves geometry problems by combining algebraic methods—primarily Wu's Method—with synthetic approaches, such as the deduction engine algorithm. The heart of this technique is 'Wu's method,' which is analogous to systematically completing a gigantic jigsaw puzzle.
Consider solving a jigsaw puzzle; it is challenging to complete if some elements (such as variables and equations in geometry) are concealed behind clutter. Thus, initial decluttering is valuable, such as sorting the puzzle pieces by colour/edge.
Wu's approach rearranges geometric equations into a more organised hierarchy. We can answer one equation at a time, just as we would place corner pieces in a jigsaw puzzle first, then use the results to simplify the next. Wu's Method simplifies complex geometry into a step-by-step assembly line.
'With very low computational requirements, this performs comparably to an IMO silver medalist. When combined with AlphaGeometry, the hybrid system successfully solves 27 out of 30 IMO problems,' according to Mr. Kumaraguru. 'This system is remarkably efficient — on most consumer laptops, with no access to a GPU, it can solve these problems within a few seconds. It also requires no learning or training phase.'
China's endeavours are not far behind. TongGeometry, a system for proposing and solving Euclidean geometry problems that bridge numerical and spatial reasoning, was developed by scholars at the Beijing Institute for General Artificial Intelligence (BIGAI) and the Institute for Artificial Intelligence, Peking University, Beijing. It has solved all International Mathematical Olympiad geometry problems for the last 30 years, outperforming gold medallists for the first time. 'Their work, including their analysis on novel problem generation, is quite interesting,' says Mr. Kumaraguru. However, he declined to express 'informed opinions' on their work due to the lack of publicly available material.
Are mathematicians redundant?
Are we nearing the point when mathematicians are obsolete? Gowers concurs: 'I would guess that we are still a breakthrough or two short of that.'
While the AlphaProof outperformed the humans, it took almost 60 hours to answer the problem, when the humans were only given 41/2 hours. If human competitors had been given that time for each task, they would surely have scored higher.
Another prerequisite is that problems were manually translated into the proof assistant Lean, meaning humans did the auto formalisation. In contrast, the AI program did the necessary mathematics.
Autoformalization converts ambiguous human language into precise logical or mathematical assertions that computers can reason about. For example, the sentence 'If it rains, then the ground gets wet' in plain English has to be translated into Propositional Logic: Rain → WetGround, which is symbolic form: First-Order Logic: ∀x (rain(x) → wetground(x)). Humans could do their own auto formalisation.
Furthermore, in the previous IMO, DeepMind's algorithms did not even attempt to solve combinatorial issues since they were difficult to transfer into programming languages like Lean. Alphageometry could only work with Euclidean plane geometry problems in 2D. Another significant inherent difficulty with AI is 'hallucinations', which are nonsensical or erroneous assertions that might occur, especially when dealing with intricate thinking.
(T.V. Venkateswaran is a science communicator and visiting faculty member at the Indian Institute of Science Education and Research, Mohali.)
