Large Language Models Are Not Truly Learning

TL;DR

Large language models do not reliably keep learning on their own once put to work, a paper by Yann LeCun argues, so their learning depends heavily on people.

  • Toddlers learn with no outside team
  • The paper proposes a three-part design
  • LeCun's company raised $1.03 billion

The AI systems known as large language models can write, summarise, generate code and answer questions with impressive fluency. Yet they do not learn the way living things do. That is the argument of a paper by Yann LeCun, Meta’s former AI chief, and his colleagues. Once a model is put to work, it does not reliably keep learning on its own from experience.

In the paper’s account, a model’s learning depends heavily on people working behind the scenes. They gather the data, clean it and organise it. Then they train the model, test how well it performs and step in when its performance starts to break down. The model may look intelligent, but much of its learning happens outside the machine.

A toddler works differently. It watches, touches, drops, breaks, falls, rests and comes back with a better grasp of what happened. No outside team decides what it should pay attention to next. The paper argues that living things also switch between ways of learning as the situation changes and sense when they are going wrong. AI systems built this way do not do these things reliably on their own.

The difference holds even at rest. In animals, learning does not stop when action stops. The brain keeps working through the day’s experience in the background, strengthening what was useful and weakening what was not. Language models do not go to sleep and wake up wiser. They stay largely fixed until people retrain them.

So the paper proposes a design in three parts. One part would learn by watching the world and a second by acting in it. A third would decide when to watch, when to act and when to reflect on what happened. The goal is a machine that keeps learning without constant retraining by people.

LeCun is already building on that view. His company, Advanced Machine Intelligence, has raised $1.03 billion for an alternative approach to AI, built around reasoning, planning and internal models of how the world works. The money does not mean the paper’s design has been built. It does show serious investors betting that language models are not the final form of AI.

Lead smarter with AI in just 5 minutes a week

Join 50K+ leaders getting our AI For Leaders newsletter every Wednesday: deep analysis, frameworks, and strategies.

No spam ever • Unsubscribe anytime