Context window
How much the model can hold in mind at once.
A model does not remember your conversation the way a person does. Each time it responds, it re-reads everything it has been given — your question, the documents you attached, the earlier turns — and that whole bundle has to fit inside a fixed budget called the context window, measured in tokens (roughly three-quarters of a word each).
When people say a model has a million-token context, they mean it can take in something like a few thousand pages in one go. That sounds like the end of the problem, and it is not, because attention thins out across a very long input: a fact buried in the middle of an enormous document is retrieved less reliably than the same fact near the beginning or end. Bigger windows also cost more and run slower, since the model pays for every token it reads.
The practical consequence is that feeding a model everything is rarely the best move. Selecting the right twenty pages usually beats supplying two thousand.
Why it matters here
Agencies tend to assume that a large context window means they can simply pour the policy library in and ask questions of it. That approach degrades quietly rather than failing loudly — the answer still looks fluent, it is just wrong more often. Any procurement that rests on a context-window number should be asked how retrieval quality was measured, not how big the window is.
The question to ask
How was this tested at the length of document we actually use, and what happened to accuracy at that length?
Go deeper
Reviewed 2026-09-20 · All decoders