The short answer
Changing a prompt or a model without an evaluation set means you cannot tell improvement from regression. The change feels better because you wrote it.
A modest test set of real cases, run automatically, plus a few production signals, is enough to make changes accountable.
Build the test set from real cases
- Collect actual inputs from production, including the ones that went wrong.
- Have someone who knows the domain state the correct output.
- Keep the awkward cases, because they are the point.
- Run the whole set on every prompt or model change.
- Add each new production failure to the set.
Point five is what makes the set improve over time. Every bug becomes a permanent regression test.
Production signals worth watching
| Signal | What it suggests |
|---|---|
| Correction rate | How often output is wrong enough to fix |
| Abandonment | Users giving up mid-task |
| Retry rate | Output not good enough first time |
| Escalation to a human | Feature not covering the case |
| Latency | Affects use more than teams expect |
Correction rate is the most honest single measure for anything that produces content a user then acts on.
Do not evaluate only the average
An overall score can improve while the feature gets worse for a particular segment. Break results down by case type, customer size or language, whichever matters for your product.
Regressions concentrated in one segment are easy to miss in an aggregate number and are exactly what customers notice.
Keep it cheap enough to run
- Small enough to run on every change without hesitation
- Automated, with a clear pass or fail
- Versioned alongside the code
- Owned by someone specific
- Reviewed periodically for cases that no longer matter
An evaluation suite that takes an hour and costs real money will be skipped under deadline, which is precisely when it was needed.