Symphony Apps Development logo — teal interlocking S monogram beside the studio wordmark
All writing

The restore drill we run on someone else's system

Auditing an inherited codebase, we start by trying to break it in the one way most teams never actually test.

Category
Engineering
Reading time
5 min
Published
17 Mar 2026
Topics
Engineering, Quality, Process

When we're asked to audit a system we didn't build, the first thing we do is not read the code. It's try to restore a backup, because it tells us more about the state of the system than an afternoon of reading ever would.

Why this is the first move

A restore test is cheap to run and brutally honest about its result: either the backup restores to a working system or it doesn't, and there's no code review opinion that can soften a failed restore. It also tends to surface, incidentally, most of the other things worth knowing about a codebase — undocumented environment variables, a migration that was never actually run in production, credentials that only one departed employee had access to.

What we typically find

  • Backups exist and have never been restored. The most common finding by a wide margin. Backups are usually configured correctly and tested never.
  • A restore that "mostly" works, missing a queue's contents or a cache that was assumed to be rebuildable but isn't, discovered only because we tried.
  • Restore instructions that describe an earlier version of the system, accurate eighteen months ago and quietly wrong since, because nobody updates a runbook they haven't needed.

What a clean result tells us

Occasionally the drill goes fine, and that's informative too — it usually correlates with a team that has good habits elsewhere: current documentation, sensible test coverage, an on-call process that's actually been used. We haven't found an exception to that correlation yet.

If nobody has restored the backup, you don't have a backup. You have a file that has never been asked to do its job.

Why we lead the audit with this

Clients occasionally expect us to start with a code walkthrough, and are surprised when day one is a restore drill instead. It sets the tone for the rest of the engagement — we're looking for what's actually true about the system, not what the documentation claims is true, and the fastest way to establish the gap between those two is to test something concrete.

What we recommend afterwards

A quarterly restore test, scheduled and owned by a named person, logged with what it caught. Not glamorous, and consistently the single highest-value recommendation we make in these audits, measured by how often clients tell us later that it caught something before it became an incident.

Tell us what you’re trying to ship

A first call is thirty minutes and costs nothing. Bring the problem, not a spec — working out what to build is the part we are good at.

Or email office@symphonyapps.ro. We reply within one business day, in English or Romanian.