Context window
How much text an AI model can take into account at once, measured in tokens.
The context window is the amount of text a language model can look at in one go. Everything the model uses to produce an answer has to fit inside it: the hidden instructions from the app, your conversation so far, any documents you pasted or uploaded, and the reply it is writing. It is measured in tokens, the chunks of text models work with; in English a token is roughly three quarters of a word.
A model has no memory of its own between messages. When you chat, the app sends the whole conversation back to the model each turn, and that is how it 'remembers' what you said. Once the conversation or the documents get longer than the window, something has to go: older messages are dropped or summarized, and the model simply cannot see them anymore. That is why a very long chat can start to forget details from the beginning.
Context windows have grown a lot, from a few thousand tokens in early chatbots to hundreds of thousands, and in some models over a million, which is enough for several books. Bigger is not free, though: long inputs cost more, take longer, and models can pay less attention to details buried in the middle of a huge text. For large collections of documents, it is often better to retrieve only the relevant pieces (RAG) than to paste everything.
Keep in mind
- Everything the model sees at once: instructions, chat, files and reply.
- Measured in tokens; a token is roughly three quarters of an English word.
- Beyond the limit, older content is dropped or summarized.
- Bigger windows cost more, and the middle of very long inputs gets less attention.
