Golden dataset
A golden dataset is a fixed set of inputs with known-good outputs, used as the benchmark for judging whether a prompt change is an improvement.
Golden datasets should include the awkward cases, not just the typical ones — empty input, unusually long input, input in the wrong format. Those are where prompt changes silently regress.
Related terms
- Prompt evaluation
- Prompt evaluation is testing a prompt against a fixed set of inputs and grading criteria to find out whether a change actually improved it.
- Regression
- A regression is a case that used to work and stopped working after a change to a prompt, a model version, or a template.
More on evaluation
See the full glossary, read the guides, or put it into practice in the prompt builder.