LANDMARK · 6 MIN · BUILD
Cost controls
After this landmark, you can name the cost levers in an AI system (caching, model tiering, prompt length, request volume), before a surprise bill forces the conversation.
You can still explore it. We’re showing the shared explanation and a related practical view without hiding the knowledge.
Unlike most software, where marginal cost per user action is close to zero, every AI request costs real money proportional to input and output size, and that cost multiplies directly with usage. A feature that’s cheap in a demo can become an expensive line item at scale, especially if it makes a large model call on every page load or reprocesses the same content repeatedly.
The main levers are caching (don’t recompute what hasn’t changed), model tiering (use a cheaper model for easy cases, escalate to a stronger one only when needed), and trimming prompt and context size to what’s actually necessary; all decided at design time, not discovered on the first surprising invoice.
This is the canonical concept. It stays the same across learner lenses so personalization never changes the underlying facts.
What this looks like for you
You mostly see this as pricing tiers and rate limits on AI tools; those exist because inference isn’t free at the provider’s end either, and it shapes what’s offered for free versus paid.
MAKE A DECISION
A support chatbot resends the full conversation history to the model with every new message, and costs have grown far faster than user growth. What’s the most direct fix?
CARRY THISFor one AI feature you build or use heavily, estimate its cost at 10x current usage. Does anything in the design (unbounded context, no caching, always using the biggest model) make that scaling worse than it needs to be?