The context window
A chatbot works with a whiteboard. Your chat goes on it, word by word. What's on the board, it can see. When the board is full, the oldest part gets wiped. That's why a long chat can lose track.
First, what do you think?
You chat with a chatbot for an hour. Does it remember everything you said? Have a guess.
What is the context window?
The whole story in three sentences. While it answers, a chatbot can only see a limited amount of text. That's the context window. Anything outside it doesn't exist for the predictor.
Think of a whiteboard. Your first message goes on it. Then the chatbot's answer. Then your next message. The predictor looks at the whole board to predict its next word. When the board is full, something has to be wiped.
The context window is that whiteboard. It's all the text the model can see while it answers. One maker calls it the model's working memory.
- Everything counts. Your messages, its answers, a document you pasted, and the answer it's writing now.
- It's counted in tokens. Some models fit up to a million tokens. That's roughly 555,000 English words. Smaller windows exist too.
- It's not the training. What the model learned in training is separate. The whiteboard is only for this one conversation.
- A new chat is a clean board. Unless the app adds something itself, like a memory feature, the new chat starts empty.
A first small step: for a new topic, start a new chat. Give the board a clean start.
What the predictor does
The predictor can only use what's on the board. That's the rule.
An example. At the start of a long chat, you wrote that you're allergic to nuts. An hour later you ask for a recipe. If that first message is wiped off the board, the predictor doesn't see it. It may suggest hazelnuts. It didn't forget, like a person forgets. The message isn't in view anymore.
There's a second effect. Even before the board is full, a very full board makes it harder. One maker writes that as the number of tokens grows, accuracy and recall go down. More text on the board means more to sort through.
What happens when it's full depends on the app.
- Some chat apps wipe the oldest part first. First in, first out.
- Some apps summarise the old part, so it takes less space.
- Some stop and tell you the chat is too long.
The predictor only sees what's on the whiteboard. Important things belong near the bottom, where they're still in view.
Try it yourself
A short chat on a very small whiteboard. Add messages one by one, and watch the oldest one get wiped. For this exercise, every word counts as one token.
Everything runs in your browser. Nothing you type is sent anywhere.
Use the free version of any chatbot. Don't type anything personal.
Start a new chat and type:
Remember this code word: pineapple. I'll ask for it later.Then chat about something else for a while. After that, ask: "What was the code word?"
What to expect. In a short chat, it gets it right: the word is still on the board.
What to look for. Now start a new chat and ask the same question. It can't know. The new chat is a clean whiteboard.
In practice
Three people run into the whiteboard. The board explains what went wrong.
Case 1The essay that went off track
Fenna (17) works on an essay with a chatbot for two hours. At the start she gave clear rules from her teacher. By the end, the chatbot ignores them.
Repeat "follow the rules!" and get frustrated.
The rules were at the top of a very long chat. They may have been wiped, or got lost among everything else on the board.
Start a new chat. Paste the teacher's rules and her current draft together, and continue from there.
Everything important is on a clean board, close to the question.
Case 2The pasted document
Bas (31) pastes a long contract into a chatbot. He asks about a clause on page 40. The answer is vague.
Assume the chatbot can't read contracts.
A full board makes it harder to find one detail. More text means more to sort through.
Paste only the relevant pages, and say exactly what he's looking for.
Less on the board, and the important part is right in view.
Case 3"But I told you yesterday"
Hanneke (70) told a chatbot yesterday about her grandchildren. Today she opens a new chat and mentions "my granddaughter's party". The chatbot asks who she means.
Feel ignored.
Yesterday's chat was on another board. Today's board started empty.
Go back to yesterday's chat. Or start with a short summary: "I have two grandchildren, Lisa and Tim."
The predictor can only use what's on today's board. So she puts it there.
By situation
Working on homework in a long chat? Put the assignment and your rules in again when the chat gets long. Or start a new chat per assignment.
Don't paste a whole folder of documents when you need one detail. Paste the relevant part. Start a new chat per task, so old tasks don't fill the board.
A new chat doesn't know your earlier chats, unless the app has a memory feature. If something matters, say it again at the start.
Myths and mistakes
| Claim | True? | Why |
|---|---|---|
| A chatbot remembers everything you ever told it | No | It sees what's on the current board. Some apps add a memory feature, but that's an extra. |
| A bigger board is always better | Not always | More fits, but a very full board makes details harder to find. |
| The context window is what it learned | No | Learning was training. The board is only this conversation. |
| If it ignores your instruction, it's being difficult | No | The instruction may be off the board, or lost among lots of text. |
Related topics
Read first
- Tokensthe board is measured in tokens.
Next
- Training and the knowledge cutoffwhat the model learned before your chat.
Related
- A good prompthow to put what matters on the board.
Sources
- documentationAnthropic. Context windows.The context window is all the text a model can look at while it answers, a kind of working memory. The whole conversation counts. With more text, accuracy drops. Chat apps may drop the oldest part first.
- documentationGoogle AI for Developers. Understand and count tokens.For Gemini, a token is about 4 characters, and 100 tokens are about 60 to 80 English words. Each model has a maximum number of tokens, and the price depends partly on the tokens in and out.
- documentationAnthropic. Models overview.Each model has a training data cutoff: the date its training text stops. It also lists a reliable knowledge cutoff, and context windows in tokens.
Test
You get 5 questions about the whole page. Get 4 right, and you've mastered the topic.
About the tools in this course. Every tool is described in the same way, in alphabetical order, without a recommendation. Names, prices and features change often: the maker's own site is always right. This site was built with the help of Claude, one of the tools described here.