For a while, prompts in our projects lived as string literals scattered through the code that called them, edited by whoever was debugging a specific complaint that week. Nobody reviewed a prompt change the way they would review a change to a database query, even though both can silently change what users see.
The problem this caused
A prompt tweaked to fix one complaint quietly changed behaviour for every other case that passed through it. Because the change lived in a string with no test attached, nobody noticed until a different complaint arrived about the thing that used to work.
What we do now
Prompts live in their own files, versioned in the repository like any other source. Each one has a small set of example inputs and expected properties of the output — not exact-match assertions, since model output varies, but structural and factual checks: does the output cite a source it was given, does it stay inside the fixed taxonomy of allowed answers, does it decline correctly when the input is out of scope.
Reviewing a prompt change looks like reviewing code
- What examples changed behaviour, and is that intended.
- What examples that used to pass now fail, and why.
- Whether the change was scoped to the actual complaint, or widened to "improve" unrelated behaviour along the way — the same discipline we apply to generated diffs elsewhere.
The quiet benefit
Versioning made prompts diffable, which made their history legible. Six months into a project, we can now see exactly which prompt was live when a specific piece of user-reported bad output was generated, instead of guessing from memory whether anyone had touched it recently.
A prompt is a piece of the system's behaviour, not a comment. It deserves the same review a query or a validation rule gets, because it fails the same way they do — silently, until someone notices the output is wrong.
Where we still fall short
Prompts interact with the specific model version behind them, and a model update can change output even when the prompt is untouched. We do not yet have a good automated way to catch that class of regression before a user does, and it is the next thing on the list.