How multimodal models work
Images, audio, and video become tokens too. That’s why one model can handle them — and why it fails at them in specific ways.
EXPLORE THE ATLAS
Switch your lens to change examples and recommendations. The underlying concepts stay shared and trustworthy.
WHAT THIS ISA landmark is one idea you can learn in about five minutes, ending in a decision you make. Rings group them by how far along you are, not by topic. See the legend
Images, audio, and video become tokens too. That’s why one model can handle them — and why it fails at them in specific ways.
Models are stateless. Every “it remembers me” feature is a store outside the model, re-inserted into the window each time.
A language model produces one token at a time from a probability distribution. Everything else it appears to do sits on top of that.
Some models will “think” before answering — producing intermediate steps, or spending extra computation, to do better on hard problems. The steps help, but they are generated text, not a proof.
A model is billions of numbers with no built-in explanation of itself. Interpretability is the research effort to read those internals — and it's why “ask it why” is not the same as knowing why.
Models don’t read letters or words. They read tokens — and that choice explains a surprising number of their failures.
A model’s tone carries no information about whether it is right. Getting a usable uncertainty signal takes deliberate work.