The client had never run a sale bigger than a normal Saturday. Marketing wanted ten times normal traffic for four hours. Nobody could tell us with any confidence whether the platform would survive an hour of that, let alone four.
Starting from what we didn't know
We did not know which part would fail first, so we refused to guess and instead built a load profile from the actual access logs of the busiest Saturday on record, scaled it up, and pointed it at a full copy of production data in an isolated environment.
The obvious suspect was fine
Everyone assumed the database would be the constraint. It wasn't, not close — connection pooling and the existing read replicas absorbed ten times load without much drama, because that part of the system had been sized correctly from an earlier project.
The dependency that nearly wasn't
The one thing that fell over was a third-party stock-availability service the checkout flow called synchronously, on every add-to-basket. At ten times load, its response time crept from around 200ms to several seconds, and the checkout flow waited for it, patiently, on every request, compounding the whole page.
- We made the call asynchronous and served the last known stock figure while a fresh one loaded in the background, accepting a small, disclosed staleness window.
- We put a hard timeout on it with a sensible default rather than an indefinite wait, because a slow yes is still worse than a fast maybe.
- We told the vendor what load to expect, which turned out to matter — their infrastructure could handle it, but only if provisioned ahead of time rather than autoscaled on the day.
What we tested that a checklist wouldn't have caught
We ran the load test with the promotional discount codes active, not just plain traffic, because the discount calculation touched a different code path than a normal purchase and nobody had exercised it under load before.
The failure that matters is rarely the one everybody was already worried about. It's the dependency nobody put on the list because it had never been a problem before.
The sale itself
Traffic peaked at just over nine times the normal Saturday. Checkout latency stayed within the range we'd tested. The stock service's provisioned capacity held. The client's main comment afterwards was that it felt uneventful, which for a launch of that size is the correct outcome.
What we kept
The load-testing environment and profile are now a standing asset, rerun before every subsequent promotional event rather than rebuilt from scratch, because the second time was a day's work instead of three weeks'.