TRAIL
How a model actually works
Tokens, prediction, attention, context, and cost — the mechanics under everything else.
This is the trail for the moment you stop wanting analogies. Not “it’s like autocomplete” but what is actually happening: what the model reads, how it chooses each word, what it can hold at once, and what you are paying for.
By the end you’ll be able to explain why a model miscounts letters, why the same prompt gives different answers, why long conversations drift, and why a confident answer carries no information about whether it is right. These are the mechanics behind failures you have already met.
Begin trail →- 1UnderstandTokens
Start at the input. Models don’t read letters or words — they read chunks from a fixed vocabulary, and that single design choice explains a whole family of odd failures.
- 2UnderstandPredicting the next token
Then the output, one token at a time, sampled from a probability distribution. This is why the same prompt gives different answers and why an early wrong turn hardens into a confident wrong answer.
- 3UnderstandInside a transformer
The mechanism in the middle: attention, which lets every token be read in the light of the others. It’s also why parameter count tells you about capacity rather than quality.
- 4UnderstandContext as working memory
What the model knows about your situation is what is in the window right now. Treating that window as a designed budget is where most real quality improvements come from.
- 5UnderstandWhat happens at inference
What running the model actually costs, in two phases with very different behaviour. This is where speed and price come from — and which levers genuinely move them.
- 6UnderstandReasoning and extended thinking
The same next-token machine, told to spend more of it before answering. Extra “thinking” genuinely helps on multi-step problems — and the steps it shows are generated text, not a proof, which is the catch that makes the next habit matter.
- 7UnderstandUncertainty and confidence
The closing habit: a model’s tone is a style setting, not a measurement. Getting a usable confidence signal takes structure from outside the model.