A client reported that a manager could see another manager's team's invoices. The code that checked permissions was correct. The test data that had validated it had exactly one manager in the system, so the check had never been exercised against a second one.
The pattern, once you're looking for it
We went back through a year of permissions defects across several projects and found the same shape repeatedly: not a wrong rule, but a rule that had only ever been tested against a lonely account with nobody else in its role to be wrongly shown.
Why this keeps happening
Test fixtures tend to grow one record at a time, added by whoever needed data for the feature they were building that week. Nobody sits down and says "we need at least two of every role, with overlapping and non-overlapping data, to properly test isolation" — it's not a natural thing to think of until the isolation has already failed once.
What we added to fixtures as a standing rule
- At least two accounts per role, with data that partially overlaps and partially doesn't, so a query that's too permissive has something to leak and a query that's too strict has something to wrongly hide.
- One account with no data at all, because an empty state is where off-by-one and null-handling bugs concentrate.
- A test that logs in as each account and asserts what it cannot see, not just what it can — the missing half of most permissions test suites we've reviewed.
Why "cannot see" is the harder half to write
Asserting a manager can see their own team's invoices is a natural test to write; asserting they cannot see someone else's takes deliberately setting up a second manager's data, and it is exactly the step that gets skipped under time pressure, which is precisely the step that would have caught this defect.
A permissions suite that only ever logs in as one user per role has tested that the feature works. It has not tested that the feature is safe.
What it cost the client
No data was actually exposed beyond the one report during testing before we caught it in review, but the fix and the fixture rebuild took the better part of a day — considerably more than the test we should have had from the start.
Where we apply this now
Every new project's fixture generation includes at least two populated accounts per role from day one, as a template rather than a retrofit, because retrofitting fixtures after the fact is exactly the work nobody schedules.