Top-p (nucleus sampling)
Top-p is a sampling setting that restricts the model to the smallest set of next tokens whose combined probability exceeds a threshold, discarding the rest.
Top-p and temperature both control variety and are usually not adjusted together. A top-p of 0.9 means the model only ever samples from the tokens making up the top 90% of probability mass, which cuts off unlikely continuations without flattening the distribution the way a very low temperature does.
Related terms
- Temperature
- Temperature is a setting between 0 and roughly 2 that controls how much randomness a model uses when choosing each next token.
- Determinism
- A process is deterministic when the same input always produces exactly the same output, with no randomness involved.
More on core concepts
See the full glossary, read the guides, or put it into practice in the prompt builder.