← Concept IndexDEFINITION WHY IT MATTERS COMMONLY CONFUSED WITH
Caching and reuse
Also called: response caching, prompt caching, memoization
Storing the result of work already done so identical or similar requests don't pay for it again — from reusing a repeated prompt prefix to returning a saved answer for a common question outright.
Much production traffic is repetitive. Reusing unchanged work cuts cost and latency at once, and a source-change check that skips the model entirely when nothing changed is often the single biggest saving available.
Storing user data. Caching is about not redoing computation; it doesn't require retaining personal content, and a cache that holds sensitive inputs is its own privacy problem to design around.