A diff that includes both the implementation and its tests looks complete. Whether it actually is complete is a separate question, and it is the one we have started asking explicitly in review.
Tests that match the code are not evidence
When the same session produces the implementation and the tests, the tests tend to describe what the code does rather than what it is supposed to do. They pass, reliably, against a bug just as readily as against correct behaviour, because both were written from the same understanding at the same moment.
The question we now ask in review
Not "do the tests pass" but "what test would a careful engineer have written for this ticket, and is it present". This forces the reviewer back to the acceptance criteria rather than the diff, which is where the actual verification has to happen.
- The unhappy path — an empty list, a missing field, a duplicate submission. Generated tests skew towards the case the code was written to handle.
- The boundary — one under, one over, one exactly at the limit. Often present for numbers, often absent for dates and time zones.
- The concurrent case — two requests for the same resource at once. Rarely generated, frequently the actual bug in production.
A specific example
A discount-code redemption feature arrived with fourteen generated tests, all green, all asserting a single-request flow. Nothing tested two requests redeeming the same single-use code within the same second. That was, unsurprisingly, the first bug a real user found.
A green test suite generated alongside the code it tests confirms the code is internally consistent. It does not confirm the code is correct.
What we ask engineers to add by hand
One test per acceptance criterion, written from the ticket rather than the implementation, before the generated tests are trusted at all. This is a small addition in volume and the part of the suite we actually rely on when something breaks later.
The broader point
Coverage percentage was never a good proxy for confidence, and generated tests make that gap wider rather than closing it, because they are extremely good at producing coverage and only incidentally good at finding out whether the software does the right thing.