Token
A token is the unit a language model reads and writes — roughly four characters of English text, or about three quarters of a word.
Tokens are why pricing and context limits are quoted in tokens rather than words. Common words are usually one token; rare words, code, and non-Latin scripts split into more. Any character count divided by four is an estimate rather than a measurement, since every model tokenises slightly differently.
Related terms
- Context window
- A context window is the maximum amount of text, measured in tokens, that a model can consider at once — including the prompt, any supplied documents, and the reply it generates.
- Temperature
- Temperature is a setting between 0 and roughly 2 that controls how much randomness a model uses when choosing each next token.
More on core concepts
See the full glossary, read the guides, or put it into practice in the prompt builder.