LANDMARK · 8 MIN · UNDERSTAND
How models are trained
After this landmark, you can explain what pretraining does, why models have a knowledge cutoff, and how to reason about the limits their training data imposes.
↗
This concept is shared, but the Everyday lens is less central here.
You can still explore it. We’re showing the shared explanation and a related practical view without hiding the knowledge.
Pretraining takes an enormous body of text and repeatedly asks the model to predict a hidden or next token, nudging its parameters whenever it is wrong. Run at sufficient scale, this produces a system with broad linguistic competence and a great deal of absorbed world knowledge, without anyone specifying any of it as a rule. Three consequences matter for using the result well. It has a cutoff: the corpus was assembled at a point in time, and nothing after it is present. It is a fixed artefact: the weights do not change as you talk to it, so nothing you type is learned, and nothing it gets wrong is corrected by you telling it so. And it inherits its corpus: the languages, viewpoints, domains, and errors that were abundant in the data are the ones the model handles fluently, while thin coverage produces confident, plausible, wrong output rather than an admission of ignorance.
This is the canonical concept. It stays the same across learner lenses so personalization never changes the underlying facts.
◌ EVERYDAY LENS
What this looks like for you
Anything recent — a news event, a price, a rule change, a new product — is outside what the model absorbed unless it can search. Correcting it in conversation fixes the rest of that conversation and nothing beyond it.
MAKE A DECISION
A model gives an outdated answer about a regulation. You correct it, and it acknowledges the correction gracefully. What has changed?
CARRY THISAsk an assistant about something in your field that changed in the last year. Note whether it flagged its own cutoff, or answered confidently from stale knowledge.