An AI feature that costs almost nothing to demonstrate can cost a great deal to run. The demo sends a few requests to a model and gets back something impressive. Production sends those requests at volume, in loops, with retries, against rate limits, every hour of every day. The unit economics that looked trivial in the prototype become the line item that decides whether the feature is viable. Most teams discover this after they have shipped, which is the most expensive moment to discover it.

The reason is structural, not accidental. The cost of running AI in production is dominated by inference, the work of actually answering each request, rather than by anything that happened during the prototype. And the newer the pattern, the worse the gap. An agentic feature, one where the model plans, calls tools, and works through a task in several steps, consumes many times more tokens per task than a single chatbot reply. Industry estimates put that multiple in the range of five to thirty times. A feature that felt free in a one-shot demo can become the most expensive thing the product does, simply by being built the way the demo was.

Why the demo lies

A demo is a controlled environment optimised, usually without anyone deciding to, for looking good. It runs once, on a clean input, with a human watching. Nothing about it resembles production.

In production the same feature runs continuously, on inputs no one anticipated, with no human watching most of the time. If the feature is agentic, a single user request can fan out into a dozen model calls as the system reasons, retries, and corrects itself. Each of those calls costs tokens, and tokens are the meter the whole thing runs on. The demo showed one call. Production runs the loop.

The other thing the demo hides is failure under load. In production, calls to a model provider fail, and they fail in a particular way. Reliability data from 2026 shows that a large share of production AI errors are not bad outputs at all. They are rate limits, the provider telling the application it is sending requests faster than its allocation permits. The model was never the bottleneck. The capacity ceiling was, and a demo never hits it.

Cost per token is the wrong number

When teams do think about AI cost, they usually think in cost per token, because that is what the provider's pricing page shows. It is the wrong unit to design around.

The number that matters is cost per successful outcome. A cheap model that needs three attempts and a round of human correction to produce a usable result can be more expensive, in total, than a costlier model that gets it right the first time. Token price is an input. Outcome cost is the thing that lands on the invoice, and it depends on how many calls the task actually takes, how often it has to be retried, and how much human time it consumes when it goes wrong.

Designing around cost per outcome changes the engineering. It makes a strong case for routing, sending routine work to a smaller, cheaper model and reserving the expensive frontier model for the requests that genuinely need it. It makes caching valuable, because a result that can be reused does not need to be regenerated. And it puts hard limits into the architecture, token budgets and circuit breakers that stop a runaway agentic loop before it bills its way through a month of margin in an afternoon. None of these are exotic. They are the difference between a feature that is viable at scale and one that is quietly bleeding money no one is watching.

What to build in from the start

These concerns belong in the architecture from the first day the feature is scoped, not retrofitted after the first alarming invoice. The studio treats AI features the way it treats any production system that has to be operated rather than merely shipped, which is the work of Product Build and the discipline that carries into Product Stewardship.

In practice that means a few things are non-negotiable. The application should be model-agnostic, able to route between models and swap providers, rather than welded to one vendor's endpoint. It should cache aggressively wherever results can be reused. It should enforce token budgets and fail safely when they are hit, so that a loop that misbehaves stops rather than spends. And it should expect rate limits as a normal operating condition, with backpressure and retry logic that degrades gracefully instead of throwing errors at users. The studio's analytics and monitoring stack, the same layered approach used to run platforms like T-Stats Solutions, applies directly here. A feature whose cost and failure modes are not instrumented is a feature being operated blind, and AI features fail in ways that are invisible until someone reads the bill.

There is a strategic reason to insist on model-agnostic architecture beyond cost control. The current economics of frontier AI are not stable. The providers running the largest models are, by their own reporting, pricing inference below what it costs them to serve. That cannot continue indefinitely, which means today's per-token prices should be treated as a floor that may rise, not a permanent condition to build a business model on. A product welded to one provider at today's prices has taken on a risk it may not have priced. A product that can route and switch has kept its options open.

What this means for your roadmap

The practical implication is that an AI feature is not done when it produces a good result in a demo. That is the moment it has earned the right to be engineered. The questions that decide viability come after the demo works: how many model calls does this take per task at the scale we expect, what does it cost per successful outcome rather than per token, what happens when the provider rate-limits us, and what stops a loop that misbehaves.

A team that can answer those questions before it ships has a feature it can operate. A team that ships first and asks later has a feature that works until it is popular, which is the worst possible time for the economics to fall apart. The demo is the easy part, and its ease is exactly what makes it misleading. The work that determines whether an AI feature survives contact with real usage is the work that happens after the impressive part is already done.