LANDMARK · 7 MIN · UNDERSTAND
Predicting the next token
After this landmark, you can explain why the same prompt gives different answers, what the temperature setting really changes, and why a model has no plan for the sentence it is writing.
At each step a language model produces a probability for every token in its vocabulary — tens of thousands of numbers saying how likely each chunk is to come next. Something then picks one, that token is appended, and the whole process runs again with the longer text as input. There is no draft, no outline, and no internal sentence waiting to be written out. The apparent plan in a well-structured answer is a consequence of each next token being chosen in a context that already contains the beginning of a well-structured answer. Two things follow. First, variability is normal: if the pick is random rather than always-the-most-likely, the same prompt yields different text. Second, an early token constrains everything after it. Once the model has committed to an opening claim, continuing coherently is easier than contradicting itself — which is one mechanism by which a confident wrong start becomes a confident wrong answer.
This is the canonical concept. It stays the same across learner lenses so personalization never changes the underlying facts.
◌ EVERYDAY LENS
What this looks like for you
Getting a different answer the second time isn’t a malfunction — sampling is deliberately random. If you want the same result twice, say so, or reuse the output you liked rather than regenerating and hoping.
MAKE A DECISION
You’re building a step that extracts invoice totals from documents into a structured field. What decoding setting is most appropriate?
CARRY THISSend the same factual question to an assistant three times in fresh conversations. Note where the three answers differ — those are the places the text was least determined by the evidence.