An AI model that reads and writes text, called a large language model, starts out as a tangle of noise. Picture a bowl of spaghetti in which every strand is a connection wired to another strand at random. Feed that tangle a sentence and it answers with gibberish. Whatever it eventually knows has to be trained into it from nothing.
That makes it an odd kind of software, because ordinary software is a list of instructions a person wrote. When a user asks for one type of thing, it does a specific thing for a set number of steps, then stops. A large language model has no such list. No line of code spells out what to do with a question about taxes or a request for a poem. Inside it sits a neural network.
A neural network is loosely modeled on a brain. Over a lifetime, certain patterns of firing among the brain’s neurons get reinforced. A pattern that leads to something rewarding fires more easily next time. One that leads to pain, like a hand pulling back from a hot stove, gets weakened. Nobody designs this. Years of repeated reinforcement build it into the habits and instincts a person carries into adulthood.
Artificial neural networks copy that process, minus the biology. In a new network, none of the randomly wired connections, called parameters, has ever been reinforced or weakened. The largest systems are reported to have more than a trillion of them, far too many for any engineer to set by hand.
So the connections get set through training. In the first stage, called pretraining, engineers feed the network a massive amount of ordinary text from the internet in small pieces. The network sees one piece and guesses what comes next, close to random at first. That guess is checked against the next word in the original text. The connections behind a good guess get strengthened and those behind a bad guess get weakened, the same reinforcement that shapes a brain.
Repeated across billions of pieces of text, that one exercise teaches the network grammar, facts and the patterns that make an answer sound real. Yet the finished model has never received a single instruction, only a long series of corrections.
