If it can weather some challenges, AI can supercharge forecasting

The Hindu

23-04-2025

Like it or not, it's clear: every year, India must face down intense heat waves and erratic but also often intense bursts of rainfall. In a bid to find as many ways out of the consequences — or at least their ability to surprise governments — as possible, the country has turned to artificial intelligence (AI) for help with modelling and early warnings.
Traditional weather forecasting uses numerical weather prediction (NWP) models. Such models begin with physics equations that simulate atmospheric behaviour using the principles of fluid dynamics and thermodynamics. They process observational data from weather stations and satellites, including temperature and wind speed, and perform their complex and time-consuming calculations on supercomputers.
AI-based models start with the data instead. AI algorithms can 'learn' the relationships between some inputs and an output — e.g. a given set of wind, temperature, and humidity conditions on one hand and the formation of a cyclone on the other — or extract spatial and temporal patterns from large datasets. And they do this without prior knowledge of the underlying earth system processes. This makes AI particularly useful for applications that lack a complete theory.
For example, an AI model can explore hidden links between various earth system variables, such as air temperature, pressure and humidity or ocean temperature, salinity, and currents, to uncover cause-effect relationships existing physics-based models don't capture. AI models can also factor in a wider range of input variables, whereas physics-based models use input variables that experts have traditionally considered to be relevant.
The Indian government joined the new international race to build such models when it announced 'Mission Mausam' in September 2024 with an allocation of ₹2,000 crore over two years. Its stated goals are to exponentially enhance the country's weather and climate observations and to better understand modelling and forecasting for more accurate and timely services.
The Mission aims to do this by, inter alia, developing better earth system models and data-driven methods using AI. The Ministry of Earth Sciences has set up a dedicated AI and machine-learning (ML) centre to develop and test different techniques and models AI to improve short-range rain forecasts, develop high-resolution urban meteorological datasets, and explore these technologies for nowcasting rainfall and snow using data from Doppler radars.
Indian researchers are also making forays in the use of AI for weather prediction. For example, groups at the DST Centre of Excellence in Climate Modelling (CECM) at IIT-Delhi; the Indraprastha Institute of Information Technology, New Delhi; the Massachusetts Institute of Technology in the US; and the Japan Agency for Marine Earth Science and Technology have together developed a ML model to predict monsoon rainfall. The model uses data from 1901 to 2001 related to the Indian summer monsoon, and accounts for the influences of systems like the El Nino (a climate pattern that emerges due to unusual warming of surface waters in the eastern Pacific Ocean) and the Indian Ocean Dipole (IOD).
According to the team, this model performs better than current physical models to predict monsoon in the country, with a forecast success rate of 61.9% for the test period of 2002-2022. The team said it can also predict rains months in advance subject to the availability of El Nino and IOD data; can be updated based on how the El Nino and IOD data evolve; can better capture nonlinear relationships in the monsoon drivers' data; and is less computationally intensive.
Challenges are only beginning
That said, these are early years and the path ahead is challenging, both in India and abroad.
Weather systems are inherently nonlinear and chaotic, so sophisticated models are required to capture their dynamic nature, IIT-Delhi associate professor Tanmoy Chakraborty said. AI models in particular require large, high-quality datasets for the models to train on first. But these datasets are hampered by problems like sensor error, inconsistent formats, and the data being spatially and temporally inconsistent.
Satish Regonda, associate professor in the departments of civil engineering and climate studies at IIT-Hyderabad, said AI/ML models typically require large amounts of data — especially at finer spatial and temporal resolutions — because as weather processes are dominated by randomness. The more data there is, the better it is to find signs of order in the chaos.
Moreover, neither AI models nor the experts that built them are generally able to explain how they were able to make certain predictions. This is why in a February 2025 paper in NatureCommunications, researchers from institutes in France, Germany, Greece, Italy, the Netherlands, and Spain wrote that operational challenges in using AI/ML for weather and climate prediction include 'the complexity of AI outputs, which hinder interpretation by non-experts.'
The scepticism stems from 'the near impossibility of explaining the reasons for good or bad performance,' Regonda added. Traditional weather models provide an intuitive understanding of the underlying processes through their equations, and the framework allows the analysis of model errors and corrections. Nonetheless, efforts are now in place to develop hybrid approaches by combining AI/ML with physics-based modelling for weather forecasting, according to Regonda.
The two bigger problems
In India, many weather forecasters don't use or run weather models that require high computing power and high-quality data; instead they use the information thus generated from other agencies, including the India Meteorological Department (IMD), the US National Oceanic and Atmospheric Administration (NOAA), the European Center for Medium Weather-range Forecasting (ECMWF), and private firms — or a combination of data produced by multiple models. Then they overlay their local knowledge, including movement of clouds and past scenarios. Regonda said these forecasters competed with each other although, 'given the growing interest in AI/ML and as finer resolution data becomes increasingly [better] available, and because of high-intensity and short-duration rainfall events, I think AI/ML models will be used extensively in the near future in India.'
The two principal challenges with the use of AI/ML for predicting what is also increasingly erratic weather are (i) the availability of sufficient data and (ii) the right human resources, and experts differ on which of the two is a bigger hurdle.
Saroj Kanta Mishra, a professor at CECM in IIT Delhi and the leader of the team that built the monsoon model, said it was human resources, especially at the interface between AI and predicting weather and climate. 'Climate science is not fundamentally an independent discipline and draws scientists from physics, mathematics, certain engineering branches such as mechanical and civil engineering, and computer science,' according to Mishra. 'It is, however, not common for many scientists from these disciplines to come into climate science as it falls somewhere between core natural sciences or core engineering disciplines.'
'For scientists working on climate science, when one does not have the AI/ML expertise required for climate science, it is like a black box, and very superficial in nature,' he continued. 'Similarly, for hardcore data, core AI/ML scientists don't have an adequate background in climate science. So the scope of doing deep research and making groundbreaking progress is highly unlikely in the present situation.'
Chakraborty agreed. 'Many powerful AI models, specifically generative AI models, operate as black boxes, hindering the understanding of prediction mechanisms and limiting trust in their outputs,' he said.
'Black box' here refers to the inscrutability of the relationships between an AI model's inputs and outputs. That is, when an AI model accepts certain inputs and produces a particular output, how the inputs and output are connected is not clear.
Critical mass
Climate is a very complex phenomenon and its prediction in India has been a challenge for decades, Mishra added. 'The physical systems driving India's climate are challenging, and AI/ML could solve problems that humans find difficult.'
According to Chakraborty, 'India's diverse topography and climate zones demand regionally tailored models, increasing development complexity.' This is further compounded by inadequate sensor networks and gaps in the meteorological infrastructure, particularly in remote regions. The end result is sparse and inconsistent data, leading to subpar model accuracy.
Further, the Indian monsoon's complex dynamics and interannual variability present a significant challenge for long-range and short-range forecasting, Chakraborty added.
However, Mishra didn't agree that the paucity of data for use in AI/ML models is a major problem 'as there has been a 10-fold increase in observational data in India over the years.' The need for more data and more computing power 'is a never-satiable demand' that can't be achieved overnight, he added.
Instead, he said India needs — and can attain — is the development of a sophisticated model tailored to solve the country's problems. 'If we get the right talent together, it can be done in very less time,' Mishra said. 'For this, active collaborations between the climate scientists and AI/ML scientists are essential, and that will happen if we can keep them under one roof, for example setting up an institute exclusively for applications of AI/ML with a mission to solve the pressing issues the country is facing today. Such an initiative could bind these experts together and groundbreaking research could be done.'
Chakraborty echoed him and said: 'A critical shortage of professionals with expertise in both meteorology and machine learning hinders the development and deployment of advanced models.' This includes data scientists with a good understanding of the physics of the atmosphere. While more data is being collected and better, there are still challenges in data accessibility, standardisation, and integration from diverse sources, he said, especially of historical data and real-time data.
Modelling a changing future
However, Madhavan Nair Rajeevan, former Secretary of the Ministry of Earth Sciences, expressed belief in the reverse: that human resources and expertise in working on ML-based weather modelling are not challenges per se in India whereas the availability of long-term data of high quality is.
'We should ensure we compile good, reliable data sets for ML-based applications. But we will need a lot of computing resources with graphics processing unit (GPU)-based computers,' he said. While conventional home computers use central processing units (CPUs), computers that use GPUs instead are adept at performing multiple computations in parallel, and thus more powerful. 'In India, we have enough expertise to work with ML for weather-modelling.'
In his tenure at the Ministry, Nair had initiated a centre for excellence in AI/ML at the Indian Institute for Tropical Meteorology (IITM), Pune, and supported several research projects for weather and climate modelling. 'Hopefully in the next one to two years, some good results will come out,' he added.
Worldwide as well, scientists are trying to overcome challenges in using ML for climate science. At the 2024 Heidelberg Laureate Forum in Germany, scientists pointed out that while they have been able to apply ML in weather forecasting with good success, they haven't been able to do so so readily to problems in climate science.
'An ML model trained to predict good weather today is not very useful in a much warmer future world with a different state of the atmosphere,' the forum heard. The atmosphere is also chaotic and the resulting random fluctuations interfere with the average climate change signal. Thus, it is easier to predict the mean future climate but modelling its full variability is very difficult.
One notable emerging enterprise in this regard is hybrid climate modelling, in which scientists combine the physics-based climate models that solve differential equations with the tools of ML.
AI/ML and extreme weather
For all these challenges, some scientists believe AI/ML models can be particularly useful to predict extreme weather events such as heat waves, droughts, torrential rainfall, and floods. 'AI has emerged as a transformative tool for the detection, forecasting, analysis of extreme events, and generation of worst-case events, and promises advances in attribution studies, explanation, and communication of risk,' the February 2025 paper in Nature Communications read. It added that the abilities of ML, and deep learning in particular, together with computer vision techniques are advancing the detection and localisation of events.
That said, 'accurately predicting and modeling extreme weather events, e.g. cyclones, heat waves, and cloud bursts, is crucial but challenging due to their localised and rapid development,' Chakraborty said.
The Nature Communications paper also expressed caution about challenges in data management issues, such as handling dynamic datasets, biases, and high dimensionality, i.e. datasets with a large number of covariate variables, and which render computations as well as extracting useful information from the analysis very difficult. AI models also struggle with unclear statistical definitions of what is 'extreme', the paper noted.
Another challenge is 'trustworthiness concerns' that arise from the complexity and interpretability of ML models, the difficulty of generalising across different contexts, and the quantification of uncertainty. Nair agreed, saying, 'Though ML is a powerful tool, it should be used carefully, with stringent verification processes.'
T.V. Padma is a science journalist in New Delhi.