LLM Day-to-Day Degradation
7 mechanisms explaining LLM quality variance from day to day, even with frozen weights: temperature drift, provider A/B routing, cache contamination, context window pollution, tokenization noise, system prompt instability, and infrastructure variance. Each is diagnosable with specific techniques. The 'context window pollution' mechanism has been officialised by Anthropic as Context Rot, the progressive degradation of reasoning quality as the window fills up, with autocompacts run at 'the worst moment' producing incoherent summaries. For mechanism #4 (context window pollution), practitioners talk about a dumb zone: past a certain fill, often placed around 40% though that figure is not a measurement, attention dilutes and the model retrieves the start and end better than the middle, which the Lost in the Middle study (Liu et al., 2023) did measure. Practical consequence: starting a fresh conversation beats stacking messages in the same session, even when the window advertises 1M tokens.
Strengths
- 7 distinct mechanisms with diagnosis and fix, not a problem list, an action guide
- Applies to all LLMs, not just Claude, useful in a multi-model workflow
- Stable foundational knowledge, does not become obsolete with model updates
Limitations
- Advanced level, requires understanding provider infrastructure to apply certain diagnoses
- Few direct solutions: several mechanisms are outside end-user control
Best for
- Teams deploying Claude agents in production and observing unexplained regressions
- Claude Code power users diagnosing a quality drop between two identical sessions
- ML engineers evaluating model stability before integrating it into a critical pipeline