← Concept IndexDEFINITION WHY IT MATTERS COMMONLY CONFUSED WITH
Pretraining
Also called: the big training run, self-supervised learning
The large, expensive first phase where a model learns language and world patterns by predicting missing or next tokens across a huge body of text — before any task-specific tuning.
It's where the model's broad ability and most of its knowledge come from, and where the training-data cutoff and many baked-in biases originate. Almost everything a model 'knows' was set here, once.
Learning from your prompts. Pretraining happened before you ever used the model; your conversation doesn't update these weights.