The marketing date was fixed and public. The platform had never seen more than a modest steady load. Two weeks to make it survive an order of magnitude more.
Measure before touching anything
The first two days went into instrumentation rather than optimisation: request traces, slow query logs, and a load test that replayed real traffic shapes rather than a flat curve. Guessing at bottlenecks wastes the time you do not have.
What was actually slow
Not the application code. Three things: a category query without the right index, an image pipeline resizing on request, and a session store that became the single point of contention under concurrency.
The changes, in order of payoff
- Index and query rewrite on the two hot paths.
- Pre-generated image variants, served from a CDN.
- Read replicas for catalogue traffic, writes untouched.
- Aggressive caching on anything anonymous, with a clear invalidation rule.
The rehearsal
Three days before, we ran the load test at twice the projected peak, in production, at night, with the client watching the dashboard. It found one more problem — a queue worker that fell behind and never caught up — which was worth the whole exercise.
Nobody wants a rehearsal until the first launch they did without one.
Outcome
Peak traffic arrived at roughly eighteen times the previous high. Error rate stayed flat, checkout latency moved by less than a fifth of a second, and the on-call phone did not ring.