A feature that calls a model on every user action has a cost curve that a feature calling a database does not. We now show clients that curve before we build anything, not after the first invoice surprises them.
Cost scales with usage, not with effort
A conventional feature costs roughly the same to run whether it is used ten times a day or ten thousand. A model-backed feature's running cost scales with usage directly, and usage is exactly what a successful product wants more of. Success and cost grow together, which is a different kind of risk than teams are used to managing.
The three numbers we now put in a proposal
- Cost per call, at current model pricing, for the specific operation being designed.
- Expected calls per user per month, estimated from the workflow, not guessed.
- The multiplier if the product succeeds — what the bill looks like at ten times today's usage, because that is the scenario nobody plans for and everybody wants.
Where the savings actually come from
Not from a cheaper model, usually. From calling the model less: caching results that do not change between requests, routing simple cases to a rules-based check before the model is invoked at all, and batching where the workflow allows a delay of a few seconds. The model call is often the last resort in the design, not the first.
If nobody has multiplied the per-call cost by expected volume before the sprint starts, cost is not a design constraint, it is a discovery the client makes on next month's invoice.
A case where this changed the design
A support-triage feature was originally scoped to classify every incoming message with a model call. The volume made that expensive at the client's growth targets. We moved to a cheap keyword filter first, escalating to the model only for messages the filter could not confidently place — roughly a third of the volume, at a fraction of the cost, with no measurable drop in accuracy on the cases that mattered.
The uncomfortable conversation this avoids
The alternative to having this conversation early is having it after launch, when the client is looking at a bill three times what they expected and asking whether the feature is worth keeping. We would rather have the conversation while the answer is still a design choice.