LLM Day-to-Day Degradation
7 mechanisms explaining LLM quality variance from day to day, even with frozen weights: temperature drift, provider A/B routing, cache contamination, context window pollution, tokenization noise, system prompt instability, and infrastructure variance. Each is diagnosable with specific techniques. The 'context window pollution' mechanism has been officialised by Anthropic as Context Rot, the progressive degradation of reasoning quality as the window fills up, with autocompacts run at 'the worst moment' producing incoherent summaries. Operational threshold for mechanism #4 (context window pollution): past ~40% window fill, degradation becomes measurable, attention dilutes on the middle of the prompt, the LLM 'remembers the start and the end' but hallucinates on the middle (dump zone). Practical consequence: starting a fresh conversation beats stacking messages in the same session, even when the window advertises 1M tokens.
Strengths
- 7 distinct mechanisms with diagnosis and fix, not a problem list, an action guide
- Applies to all LLMs, not just Claude, useful in a multi-model workflow
- Stable foundational knowledge, does not become obsolete with model updates
Limitations
- Advanced level, requires understanding provider infrastructure to apply certain diagnoses
- Few direct solutions: several mechanisms are outside end-user control
Best for
- Teams deploying Claude agents in production and observing unexplained regressions
- Claude Code power users diagnosing a quality drop between two identical sessions
- ML engineers evaluating model stability before integrating it into a critical pipeline