Regression
A regression is a case that used to work and stopped working after a change to a prompt, a model version, or a template.
Prompts regress in ways code does not: a model version upgrade can change behaviour without any edit on your side. Keeping the test set and re-running it after every model change is the only way to notice.
Related terms
- Golden dataset
- A golden dataset is a fixed set of inputs with known-good outputs, used as the benchmark for judging whether a prompt change is an improvement.
- Prompt evaluation
- Prompt evaluation is testing a prompt against a fixed set of inputs and grading criteria to find out whether a change actually improved it.
More on evaluation
See the full glossary, read the guides, or put it into practice in the prompt builder.