TRAIL
How models are made
Pretraining, alignment, size, openness, and the data underneath — what shapes a model before you ever use it.
Every model you use arrived with a history: what it was trained on, what was done to it afterwards to make it an assistant, how it was shrunk to run affordably, and what its makers will tell you about any of that.
By the end you’ll be able to say why a model has a knowledge cutoff, why two products built on similar models behave so differently, when fine-tuning is the right answer and when it is an expensive detour, and what questions to put to a vendor about training data and documentation.
Begin trail →- 1UnderstandHow models are trained
Pretraining: one long pass over a fixed corpus assembled at a point in time. The cutoff, the inherited biases, and the confident gaps all start here.
- 2UnderstandFine-tuning and alignment
A pretrained model doesn’t answer questions — it continues text. Everything that makes it an assistant, including its refusals and its tone, happens in this stage.
- 3UnderstandSize, distillation, and quantisation
How the same capability gets delivered at very different sizes and costs, and what distillation and quantisation actually trade away.
- 4UnderstandOpen and closed models
Downloadable weights or an API endpoint — the choice that decides where your data goes, whether the model can change under you, and what you can still run in five years.
- 5UnderstandData provenance
Where the training data came from and whether anyone agreed to it — and, at the other end, whether you can tell what a piece of content is.
- 6UnderstandModel cards and documentation
The document that should answer the previous five steps. Reading one well is mostly noticing which sections aren’t there.