What Happens When LLM's Run Out Of Useful Data?

Forbes

25-06-2025

There is one obvious solution to a looming shortage of written content: have LLMs generate more of it.
By SAP Insights Team
Most of us feel like we're drowning in data. And yet, in the world of generative AI, a looming data shortage is keeping some researchers up at night.
A 2024 report from the nonprofit watchdog Epoch AI projected that large language models (LLMs) could run out of fresh, human-generated training data as soon as 2026. Earlier this year, the ubiquitous Elon Musk declared that 'the cumulative sum of human knowledge has been exhausted in AI training,' and that the doomsday scenario envisioned by some AI researchers 'happened basically last year.'
GenAI is unquestionably a technology whose breakthroughs in power and sophistication have generally relied on ever-larger datasets to train on. Beneath the flurry of investment, adoption, and general GenAI activity, a quiet concern has surfaced: What if the fuel driving all this progress is running low?
There is one obvious solution to a looming shortage of written content: have LLMs generate more of it.
The role of synthetic data in large language models
Synthetic data is computer-generated information that has the same statistical properties and patterns as real data but doesn't include real-world records. Amazon recently had success using this method with LLM-generated pairs of questions and answers to fine-tune a customer service model. Because the task was narrow and the outputs were easily reviewed by human beings, the additional training on synthetic data helped the model get better at responding accurately to customer inquiries, even in scenarios it hadn't seen before.
Another use case for synthetic data is for businesses using proprietary data to train bespoke LLMs—whether building them from scratch or, more commonly, layering retrieval-augmented generation (RAG) atop a commercial foundation model. In many such cases, the proprietary data involved is tightly structured, such as with historical transaction records formatted like spreadsheets with dates, locations, and dollar amounts. In contexts like these, LLM-generated synthetic data is often indistinguishable from the real thing and just as effective for training.
But in less narrowly defined training scenarios, specifically the development of those big commercial models RAG relies on, the risks of training on synthetic data are real.
The most widely cited danger has the dramatic name 'model collapse.' In a 2024 study published in Nature, researchers showed that when models are repeatedly trained on synthetic data generated by other models, they gradually lose diversity and accuracy, drifting further from the true distribution of real-world data until they can no longer produce reliably useful output.
Mohan Shekar, SAP's AI and quantum adoption lead for cloud-based ERP, likens the process to 'model incest.' With every successive iteration, a model trained on its own output will tend to reinforce biases and flaws that may at first have been barely noticeable, until those minor defects become debilitating deformities.
Long before reaching these extreme states, models trained with synthetic data have also been shown to exhibit a dullness and predictability reflecting their lack of fresh input. Such models may still have their uses, especially for mundane work and applications, but as Shekar puts it, 'If you're trying to innovate—really innovate—[a synthetic-data–trained model] won't get you there. It's just remixing what you already had.'
Some researchers, including OpenAI CEO Sam Altman, have long argued that innovation in how models are trained may soon start to matter more than what they're trained on. The next wave of breakthroughs, the thinking goes, may come from rethinking the architecture and logic of training itself and then applying those new ideas.
Yaad Oren, head of research and innovation at SAP, is confident that such a shift is underway. Recent advances in training methods already mean 'you can shrink the amount of data needed to build a robust product,' he says.
One of those recent advances is multimodal training: building models that learn not just from text but also from video, audio, and other inputs. These models can effectively multiply one dataset by another, combining different types of information to create new datasets.
Oren gives the example of voice recognition in cars during a rainstorm. For car manufacturers trying to train an LLM to understand and follow spoken natural-language instructions from a driver, rain in the background presents a hurdle. One unwieldy solution, says Oren, would be to 'record millions of hours of people talking in the rain,' he says, to familiarize the model with the soundwaves produced by a person asking for directions in a torrential downpour.
More elegant and practical, though, is to combine an existing dataset of human speech with existing datasets of 'different rain and weather sounds,' he says. The result is a model that can decipher speech across a full range of meteorological backdrops—without ever having encountered the combination firsthand.
Even more promising is the potential impact of quantum computing on model training. 'What quantum brings in,' says Shekar, 'is a way to look at all the possible options that exist within your datasets and derive patterns, connections, and possibilities that were not visible before.'
Quantum computing could even increase the total supply of usable data by accessing the vast, underutilized oceans of so-called unstructured data, says Shekar. 'Instead of needing 50 labeled images to train a model,' he says, 'you might be able to throw in 5,000 unlabeled ones and still get a more accurate result.'
That could be a very big deal indeed. AI engineers have long had the same feelings about unstructured data that physicists have about dark matter: an exquisite blend of awe, annoyance, and yearning. If quantum computing finally unlocks it, especially in tandem with multimodal learning and other innovations, today's fears of a data drought might recede.
A version of this story appears on SAP.com