Type a question into an AI assistant such as ChatGPT or Claude, and a clear answer appears in seconds. From outside, the exchange looks like magic. Behind it sits a long, traceable sequence of steps that starts with data, before any training.
Engineers gather enormous amounts of text from books, webpages, code, articles, forums and reference material. Teams then strip out duplicates, low-quality content and broken formatting, because a model trained on noisy text repeats the noise back to users.
A language model never reads the way a person does, so clean text has to become numbers. Each sentence is broken into small pieces, such as whole words, parts of words or punctuation marks. An unusual word such as “unbelievable” splits into several pieces. Each piece then becomes a list of numbers.
Most modern language models are built on one design, called the transformer. Inside it, one part looks at the surrounding words and decides which ones shape each word’s meaning. That part is called attention. Take a sentence about a programmer who fixed a server because the machine kept failing. Attention links “the machine” back to “the server.”
Before training, the model holds only random internal numbers and no knowledge. Training gives it one task, which is to predict the next piece of text. Fed “the capital of Japan is,” an untrained model answers “London.” A calculation measures how wrong that guess is. Training then traces which of those numbers caused the mistake and nudges each one slightly toward a better answer. Engineers repeat that cycle across huge amounts of data for weeks or months, on thousands of specialized chips. This first stage of training builds most of the model’s knowledge.
The result behaves like a strong autocomplete engine. It continues whatever text it is given without following instructions or answering questions, so a second stage turns it into an assistant. First, human reviewers build collections of questions paired with ideal answers, and training on those pairs teaches the model to answer. Then reviewers and automated systems compare two answers and reward the stronger one, weighing usefulness, accuracy and safety. OpenAI documented this two-step process for the models it named InstructGPT.
No one programmed the model separately to write Python code, explain physics or translate French. Each ability came from the same repeated task of predicting the next piece of text and getting slightly closer to correct each time.
