Context window
A context window is the maximum amount of text, measured in tokens, that a model can consider at once — including the prompt, any supplied documents, and the reply it generates.
When a conversation exceeds the window, the earliest content is dropped, which is why long chats start "forgetting" things. Because the reply shares the same budget, a prompt that fills the window leaves no room for the answer.
Related terms
- Token
- A token is the unit a language model reads and writes — roughly four characters of English text, or about three quarters of a word.
- Lost in the middle
- Lost in the middle is the observed effect that models recall information placed at the start or end of a long context more reliably than information in the middle.
- Context stuffing
- Context stuffing is pasting large amounts of material into a prompt in the hope that the model will find the relevant part itself.
More on core concepts
See the full glossary, read the guides, or put it into practice in the prompt builder.