How to evaluate a prompt
Published
To evaluate a prompt, collect at least ten inputs covering the real range including awkward cases, write down what a correct output looks like for each, and re-run the whole set after every change. Without a fixed test set, a change that improves one example is indistinguishable from a change that improves nothing.
Why does prompt evaluation matter?
Because the alternative is a specific and very common self-deception: you change a prompt, look at one output, decide it reads better, and keep the change. You have measured nothing. You had a sample of one, no baseline, and an author’s interest in the result.
Two failure modes follow. Overfitting — the prompt is tuned until it works on the two examples you had to hand and fails on everything else. And silent regression — a change fixes one case while breaking three you did not re-check.
What is the minimum viable evaluation?
Smaller than most people assume. You do not need an evaluation framework to get most of the benefit.
- Collect ten to twenty real inputs spanning the actual range you will send.
- For each, write one or two sentences describing what a correct output must contain.
- Run all of them against the current prompt and keep the outputs.
- After any change, re-run all of them and compare against the previous outputs.
That is it. A spreadsheet is sufficient. The discipline is in re-running the whole set rather than the one case you were thinking about.
Which inputs belong in the test set?
The typical case is the least informative one, because it is the case that already works. A golden dataset earns its keep on the edges.
| Include | Because |
|---|---|
| Three or four typical inputs | The baseline. Catches gross regressions. |
| Empty or near-empty input | A common crash, and a common source of invented content. |
| Unusually long input | Reveals truncation and lost-in-the-middle failures. |
| Input in the wrong format | Reveals whether the prompt degrades or breaks. |
| Input with a missing field | Tests whether the null rule is followed or a value is invented. |
| Input containing an instruction | A basic injection check. |
| The case that failed last time | Every real bug becomes a permanent test. |
How do you grade output that has no single right answer?
Write a rubric: a small set of criteria that can each be answered yes or no. Overall impression is not repeatable — the same output scores differently on different days.
For each output, answer yes or no:
1. Does it address every question in the input?
2. Is every factual claim present in the supplied context?
3. Does it stay within the word limit?
4. Does it avoid all banned phrases?
5. Would a reader in the target audience need to ask a follow-up?
Score = count of yes answers on 1–4, minus 1 if 5 is yes.Binary criteria are worth the constraint. "Is the tone appropriate?" cannot be graded consistently; "does it contain an exclamation mark?" can.
When should you use one model to judge another?
LLM as judge scales grading beyond what you can read yourself, at the cost of inheriting the judge’s biases. Judges favour longer answers, answers that resemble their own style, and the first option presented in a comparison.
- Give the judge the same explicit rubric a human would use.
- Judge one dimension at a time. A single "rate this out of ten" collapses everything into a vibe.
- Randomise the order when comparing two outputs, then swap and re-run — position bias is real and measurable.
- Spot-check against your own judgement periodically. A judge that has drifted is worse than no judge.
How often should prompts be re-tested?
After every prompt change, and — this is the one people miss — after every model version change. Prompts regress with no edit on your side, because the model underneath changed.
A model upgrade that improves average quality can still break a specific prompt that depended on the previous version’s habits. If you have the test set, you find out in minutes. If you do not, you find out from a user.
Common questions
- How many test cases do I need to evaluate a prompt?
- Ten to twenty covers most of the benefit for a single prompt, provided they span the real range rather than clustering on typical cases. Fewer than five gives you almost no signal, because normal model variance swamps the effect of your change.
- How do I know if a prompt change is actually an improvement?
- Re-run the full test set and compare against the saved previous outputs using the same rubric. A change is an improvement if it raises the aggregate score without lowering any individual case — a change that improves the average while breaking one case is a trade, not a win.
- Can I use AI to evaluate my prompts?
- Yes, with a written rubric and one dimension judged at a time. Be aware that judges favour longer answers and answers in their own style, so spot-check against your own judgement regularly.
- Do prompts break when the model is updated?
- Sometimes, with no change on your side. A prompt that relied on a previous version’s formatting habits can degrade after an upgrade that improves quality overall. Re-running the test set after any model change is the only reliable way to notice.
- What is overfitting a prompt?
- Tuning it so tightly to a handful of test inputs that it performs worse on the general case. It happens when the same three examples are used for every iteration. Holding back inputs never used during tuning is the way to detect it.