Model choice comes up in nearly every AI-feature kickoff, usually early and usually with more weight than it deserves. Swapping a model is a config change. The decisions that are expensive to change later are the ones that get skipped in that same meeting.
What actually costs to change
- The schema the model must return. Once a feature and its consumers depend on a particular shape, changing it touches every caller, not just the model call.
- What happens on a low-confidence answer. Silently showing an uncertain answer as fact, versus flagging it, versus falling back to a human, are three different product experiences that need deciding once, not discovered in production.
- Where the ground truth lives. If the model should answer from live data rather than memorised knowledge, the retrieval and freshness guarantees are the actual engineering, and they don't move when you switch providers.
A model swap we did mid-project
We changed the underlying model for a support-triage feature partway through a build, for cost reasons, and the change touched one configuration value and a handful of prompt-format details. The schema, the validation, the fallback behaviour and the logging were untouched because none of that was model-specific to begin with.
If swapping the model is hard, the design put intelligence in the wrong layer.
The conversation we try to have instead
Rather than "which model", we ask what the feature should do when the model is confidently wrong, because it will be, at a rate nobody can promise down to zero. That answer shapes the schema, the UI and the review process, and it's the same answer regardless of provider.
Cost as a design input, not an afterthought
We put a per-operation cost figure on the same page as the acceptance criteria for any model-backed feature, something we started doing as a habit earlier this year. Model choice affects that number, but so does prompt length, caching, and how often the feature is actually invoked — usually more than the model line item does.
What we tell clients who ask first
We answer the model question honestly and briefly, then move the conversation to the failure path, because that's where the six weeks of engineering actually go.