Retrospectives used to end after velocity, blockers and morale. We added a fourth line — actual model-usage cost against what was planned — and it has been an uncomfortable, useful addition more often than a quiet formality.
Why it wasn't there before
Cost tracking for model-backed features lived on a separate dashboard, checked separately, by a different set of eyes than the ones running the retrospective. That split meant a sprint could be judged a success on velocity and quality while quietly drifting on cost, with nobody in the room that mattered actually looking at the number until a monthly review caught it later than it should have.
What the question actually surfaces
Not just whether cost matched the estimate, but why, in the same conversation as everything else that went right or wrong that sprint. A feature that used more model calls than planned because of an edge case discovered mid-sprint is a different finding from one that drifted because nobody was watching, and the retrospective is where that distinction actually gets made rather than left to a spreadsheet to imply.
What changed in practice
- Cost surprises get discussed with the same team that built the feature, while the context is fresh, rather than reaching an engineer weeks later as a line item they have to reconstruct the reasoning for.
- Cost decisions become visible to the whole team, not just whoever owns the dashboard, which has caught more than one case of a reasonable-seeming choice in one feature quietly setting a bad precedent for the next.
- A cost overrun gets the same treatment as a missed sprint goal — a cause, not a blame, written down so the next sprint's planning accounts for it.
The resistance we expected and didn't get
We thought engineers might treat a cost line in retrospective as an accusation, the way some teams treat velocity targets. It has mostly landed the opposite way, because it's framed as one more piece of information about how the sprint actually went, alongside blockers and bugs, rather than a target to be hit.
Cost is a property of the work, the same as quality and speed. Reviewing two of the three and not the third eventually costs someone a difficult conversation with a client.
What we'd tell a team adding this for the first time
Keep it to one question, asked plainly, at the end alongside the others. The value is in it becoming routine and low-stakes, not in turning it into its own separate ceremony that people start dreading or gaming.