Bug reports arrive in every shape: a screenshot with no words, a paragraph of frustration with no reproduction steps, a support ticket forwarded three times with context lost at each hop. We use a model to turn these into a consistent shape before an engineer looks at them, and it has changed how quickly things get fixed.
What we do not ask the model to do
Diagnose the bug. A model reading a vague report and guessing at a root cause produces a plausible-sounding theory that is frequently wrong, and a wrong theory stated confidently wastes more engineering time than an honest "unclear" would.
What we ask it to do instead
Turn an unstructured report into a structured one: what the user was trying to do, what they expected, what happened instead, and what information is missing that would let someone reproduce it. The output is a better-formed question, not an answer.
The fields that matter most
- Reproduction steps, extracted or explicitly flagged as absent. A report with no steps is routed differently from one with clear steps, rather than sitting in the same queue looking equally actionable.
- Environment details, pulled from whatever metadata came with the report, so nobody has to go back and ask the user what device they were using.
- Severity, proposed conservatively. The model is told to under-rate rather than over-rate ambiguous cases, because a report escalated unnecessarily costs more attention than one that waits an extra hour for a human to confirm.
Where this earns its keep
Triage volume, not diagnosis quality. A support queue that used to take a person twenty minutes per report to read, categorise and route now takes them two minutes to confirm a pre-filled structure, with the model's confidence in its own extraction shown plainly so a triager knows which fields to double-check and which to trust.
The valuable output of this pipeline is not "here is the bug". It is "here is exactly what is missing before anyone can find the bug", which is the question a tired support agent forwarding a ticket at the end of a shift rarely has the time to ask themselves.
The failure mode we watch for
A model confidently filling in a reproduction step that the user never actually stated, inferred from a similar past report. This looked helpful and was occasionally fabricated. We now require the model to mark inferred fields distinctly from fields taken directly from the user's own words, so nobody mistakes a plausible guess for a fact the user reported.