An automated pricing suggestion feature passed every code review and every test we had for it, and still made a bad recommendation in a specific recurring scenario for months before anyone caught it, because nobody had reviewed its decisions, only its code.
The distinction that matters
Code review asks: does this do what it was written to do. That question was answered correctly — the code implemented its logic faithfully. The question nobody was asking was: is the logic itself still the right logic, given how it's actually being used. Those are different reviews, and only one of them was happening.
Where the gap showed up
The feature suggested price adjustments based on recent sales velocity. For a small, specific category of seasonal products, velocity data from the off-season was misleading the model into suggesting price cuts right before demand was about to pick up on its own. The code was working exactly as designed; the design had a blind spot nobody had gone looking for.
What we added: a decision review, separate from code review
- A regular sample of actual outputs, reviewed by someone who understands the domain, not the code — in this case, someone who understood seasonal pricing, not the pricing engine's internals.
- Outcomes tracked against the decision, not just logged — did the suggested price change correlate with the result it predicted, checked periodically rather than assumed.
- A channel for the people affected by the decisions to flag a pattern, distinct from a bug report, because "this keeps recommending something odd for this one category" doesn't read like a bug and wouldn't have been filed as one.
Why this is a different job from testing
A test suite checks that the code does what the last person who wrote a test thought it should do. It cannot catch a case where the original thinking was incomplete, because the test was written from the same incomplete thinking. Reviewing decisions requires someone with domain judgement looking at outcomes, not someone with engineering judgement looking at code.
A feature can be entirely correct and entirely wrong at the same time, if the code matches the design and the design missed a case.
What changed for the seasonal category
We added a seasonal adjustment factor to the velocity calculation, informed by the domain reviewer rather than discovered by the engineering team, because the fix required knowing something about the business that the code review process was never going to surface.
What we do differently on new AI features now
Domain review of a sample of real decisions is scheduled from launch, on a cadence, rather than added after the first time a quiet blind spot costs someone money.