The easy phase of AI progress is ending. For several years, leading labs improved their models by feeding them more books, articles, code, Reddit threads and Wikipedia pages. Research estimates that, if the trends it tracked continue, models will be trained on datasets approaching the full stock of public human text between 2026 and 2032. That limit falls on the first and broadest stage of training, where a model learns from public text.
That will not stop progress, but more public text no longer promises the next jump in ability. So the labs that keep improving will get more value from the data and models they already have. That means better training, better reasoning and better checks on whether answers are right.
For better training, labs are adding material written by AI. Training too heavily on machine-written text risks a gradual loss of quality, called model collapse. So the aim is to mix the two, with human text still the anchor. A 2025 paper found that AI-rephrased text on its own trained models no faster than natural web text. But a mix of one-third rephrased text and two-thirds web text reached the same result five to ten times faster in larger training runs. That stretches scarce human text further.
Reasoning can improve without adding any new text to that first stage of training. OpenAI trained its o1 model by rewarding better answers, which lets the model check itself, backtrack and work longer on harder problems. On an Olympiad qualifying exam, OpenAI’s GPT-4o model solved 13% of problems, while o1 solved 83%.
Checking answers is harder to scale. Reuters reporting found that AI companies rely on physicians, financial analysts, accountants, historians and scientists, because general reviewers are no longer enough for advanced tasks. A model that can already handle parts of junior work cannot keep improving through junior review alone.
Meta’s move to pay nearly $15 billion for a stake in Scale AI points the same way. Scale AI’s business is data labelled by people and the systems that test models. Data and computing power limit AI, and so does the supply of people who can judge whether a model is right.
