Context Window
A LLM's working memory: how much text it can 'see' at once (conversation + instructions + documents). Anything that does not fit simply does not exist for it. A bigger window is not a better-read one: the Lost in the Middle study (Liu et al., 2023) shows models retrieve information placed at the start or end of the context more reliably than in the middle. Practitioners also talk about a dumb zone, the fill level past which answers get sloppy: a field rule of thumb rather than a measurement, and Dex Horthy (HumanLayer), who made the term popular, says he stops around 300,000 to 400,000 tokens on a model sold as 1 million. An advertised 1M-token window is not 1M usable tokens.
Strengths
- Starting a fresh conversation for each distinct task is often enough to get sharp answers back, since stacking messages teaches the model nothing
- Field rule: compact or start fresh well before the limit, forgetting and hallucinations climb as the window fills
Limitations
- Large windows cost more in tokens, check pricing
Best for
- Understanding why an agent loses track on long sessions