Bug fixing has an obvious shortcut and it is usually the wrong one: read the report, guess at the cause, patch it, ship. Coding agents make that shortcut faster and no less wrong.
The order we now enforce
A bug report becomes a failing test before it becomes a fix. The test encodes the actual reported behaviour — the input that broke, the output that was expected — and it fails against the current code. Only then does anyone, agent or engineer, write the fix.
Why this order matters more with agents
A model asked to "fix this bug" from a vague description will often produce a plausible-looking patch that changes behaviour near the reported symptom without actually addressing the reported input. It passes a glance. It does not pass the test, because there is no test yet to fail against — which is exactly the point of writing one first.
- The test must fail before the fix. If it passes immediately, the reproduction is wrong, not the code.
- The test must use the reported data, not a simplified version of it. Simplification is where root causes go missing.
- The fix is judged against the test, not against whether the original symptom seems to have gone away in manual checking.
What this caught for us
On one project this surfaced that a "checkout total is wrong" report was actually two unrelated bugs — a rounding error in one currency path and a stale cache in another — that happened to produce a similar-looking symptom. A model given the vague description alone fixed the cache issue and left the rounding error live, because the rounding error only showed up with the exact numbers from the report.
A bug report describes a symptom. A failing test is the only artefact that proves you understood the cause.
The cost is smaller than it looks
Writing the reproduction first takes perhaps ten extra minutes on a typical bug. It has, on every project where we have tracked it, reduced the rate of reopened tickets — bugs marked fixed that come back a week later under a slightly different description.
Where we still use judgement
Not every report is worth a permanent test — a one-off data correction does not need a regression suite entry. But anything touching business logic that a customer will hit again gets this treatment, without exception, model-assisted or not.