02. 10. 2026
AI, Development
If you’ve spent any real time with an AI coding agent, you know the feeling. The first twenty minutes are great – the agent finds the right file, makes the right edit, and explains itself clearly – this is amazing, what a time to be alive!
Just like when you’re stuck in a 3 hour meeting, at 1.5 hours in to it you can’t even remember when you got there, who you even are, and maybe even when you started working there.
The same goes for using LLMs. Everything seems great until maybe around the fortieth tool call some cracks start showing: It quietly stops being good. It forgets a constraint you stated very clearly, or it re-reads a file it already read. It starts suggesting fixes for a problem you solved half an hour ago. There’s no obvious problem, it just seems to be getting dumber and dumber as you go along. Weird, right?
Most people read that as “Hmm, today Claude seems more dull than usual.” But the truth is, this is a very common and predictable scenario, one that’s bound to happen.
Claude’s context window fills up fast, and performance degrades as it fills. Period. That’s how it works.
I spent a while writing up how these agents actually work for my team, mostly so we’d stop trading in “folklore” or “pseudoscience”. What follows is the part that turned out to matter most. In this article, my examples use Claude Code, because that’s what I run most of the time, and because it’s the best documented. But the same can be basically applied to Codex and OpenCode, too.
The commands and the limits can’t though, and I’ll get to that later.
The Preconception Is Wrong
It’s tempting to picture the context window as stacking, linear and predictable, like water in a bucket. Fine until full, then it overflows, and until that point everything inside is equally available. That picture is both comforting and wrong.
It behaves more like a whiteboard where, as you finish the available space, you start writing on top of the text that’s already there. The information is technically still there. The model just attends to it less reliably because it’s harder to reach and read.
Attention is strongest at the edges (of the context):
- The start holds the system prompt, your
CLAUDE.md, and your project rules, which are visible to every later computation - The end holds your current message and the most recent tool results, which are closest to where the model is making its predictions
- The middle degrades first, and degrades further as the window fills
This is measured, not vibes. The foundational paper is Liu et al., Lost in the Middle: How Language Models Use Long Contexts (2023). Chroma Research’s Context Rot (2025) found non-uniform degradation across 18 frontier models as input length grew, and it added a detail I find very useful in practice: Plausible but irrelevant content hurts more than obvious noise. A retrieved file that looks relevant but isn’t is worse than junk (so be careful!)
Two consequences follow, and as you imagine they have dire consequences:
First, (once again) less is more. A focused 60K-token context beats a cluttered 300K one, even though both “fit”. Second, position matters, a lot. Where something sits in the context window changes how reliably it gets used!
Which also means a bigger window is most definitely not the fix people hope it is. A 1M-token window gives you more room, not equal attention across that room. The same degradation at the middle exists at every size. You can actually say that this is “by design”. It’s a known trade-off that comes from using transformers (which is what evaluates tokens in a relationship-like way).
Context Grows Around Your Prompt
Here’s the trap, and it took me an embarrassingly long time to see it clearly. I’m so sorry to disappoint my fans 🙁
Suppose you state a constraint at the start of a session. Fifty tool calls later, your original instruction has tens of thousands of tokens sitting between it and the end of the window. It’s no longer towards the end, because the real “start” has tool definitions and context files, so your original instruction has drifted into the weak middle! And you didn’t move it, the model’s context just grew around it!
That’s precisely why the agent “forgot” something you said perfectly clearly, in writing, in the same conversation. While you thought it was getting dumber and unreliable.
There’s a corollary here that reads as nonsense until you think about ordering, though. A 65%-full window bloated with tool definitions can be worse than an 80%-full window that’s mostly conversation. Conversation sits below your task, in the strong recency zone. Tool schemas sit above it, pushing it down. (That one is a heuristic from the Context Engineering chapter of agenticoding.ai rather than a measured figure, but the mechanism is real.)
The practical fix is almost stupidly simple: Restate critical constraints as you go. If a requirement matters five steps into a task, say it again at step five. It feels like nagging (well, it is to be honest) but you need it. You’re moving the constraint back into the zone where it gets attended to. Not ideal, but it does work.
And when the agent ignores something you clearly said, suspect position before you conclude it can’t do the job! Sometimes doing a hand-off and starting a new session helps a lot.
What’s actually in there?
Before you type anything, this is roughly what’s loaded. The token counts are purely just to illustrate, taken from documentation examples, and yours will clearly be very different:
| What | Illustrative cost (tokens) |
|---|---|
| System prompt | 4,200 |
Project CLAUDE.md | 1,800 |
| Auto memory index | 680 |
| Skill descriptions (one-liners only) | 450 |
Personal global CLAUDE.md | 320 |
| Environment info | 280 |
| MCP tool names (schemas deferred) | 120 |
That’s a small startup bill. THEN you start working, and the real consumers show up. Per the docs, file reads dominate context usage, so one auth file might be 2,400 tokens. An npm test run, 1,200. A grep, 600. Do that forty times and you can see where the window went (spoiler: I don’t think the sun shines there.)
The cheap and the expensive mechanisms sort out like this:
| Feature | Cost |
|---|---|
CLAUDE.md | Paid on every single request |
| Skills | Description upfront, body only when invoked |
| MCP servers | Names upfront, schemas on demand |
| Subagents | Isolated from the main session |
| Hooks | Zero, unless the hook deliberately returns content |
That last row is the “sleeper”. Hooks are shell commands that harness runs on events, outside the model entirely, so they cost nothing in context! They’re the cheapest extensibility mechanism you have, and most people never touch them. Crazy!
When a session starts feeling sluggish or forgetful, /context shows you exactly what’s consuming the window, including which memory files actually loaded. That last part is invaluable when an instruction isn’t taking effect – it can really shine some light on why/what/when. Also, /doctor is the closest thing to a linter for your setup, flagging unused skills and servers against what they cost you.
Sadly, compaction is lossy
When you approach the limit, Claude Code does two things, in order. First, it clears older tool outputs first (the bulky results you’ve already acted on). If that isn’t enough, it summarizes the conversation (compaction).
Your requests and key code snippets are preserved. Detailed instructions from early in the conversation may be lost. That’s the trade, and it’s worth knowing before you rely on it.
What survives, thankfully, is the useful part:
| Mechanism | After compaction |
|---|---|
Project CLAUDE.md, auto memory | Re-injected from disk |
| The plan written in plan mode | Re-injected from disk |
| Files read or edited | Re-read, up to five, most recently modified first |
| Invoked skill bodies | Re-injected, capped at 5,000 tokens each and 25,000 total |
| Skill descriptions (the startup index) | Don’t reload. Only skills you actually invoked persist |
| Everything else in conversation | Summarized away |
There’s one consequence here worth more than the rest of the table put together. Files get re-injected. Conversation gets summarized away. So anything that must survive a long session belongs in a file on disk, not in chat.
That’s also why plan mode can save the day on non-trivial work. The plan is written to disk, so it comes back after compaction, unlike anything you simply “said”. The docs are honest about the other side of it, however: Plan mode adds overhead, and if you could describe the diff in one sentence, skip it (unless it’s comfortable for your colleagues to review it and understand the PR better).
If you ever see the error Autocompact is thrashing: the context refilled to the limit, compaction succeeded and then a file or tool output immediately refilled the window, several times running. Read the oversized file in chunks, run /compact keep only the plan and the diff, move the large-file work to a subagent, or just clear.
Honestly, repeated mid-task compaction is best read as a scoping signal. It usually means your tasks are too big, not that your window is too small, so maybe switch to a model with a bigger context, or split your task into waves for example.
So, how can we mitigate this and work around it?
There’s quite a few things, tricks and procedures that can be employed to work around it, but that’s worth a dedicated post about it. In the next episode of this saga (part 2) we’ll delve more into that, and we’ll see how it’s more a question of “getting used to it” and “best practices” than any real hard limit.
If you’re interested, you can find part 2 here.
Thank you for your time and interest!
These Solutions are Engineered by Humans
Did you find this article useful? Using LLM infrastructure to its fullest is still work in progress for most teams, and getting it right early can pay off. We’re always looking for people who enjoy building monitoring solutions at the intersection of AI infrastructure and Enterprise operations. Check out our open positions.
Marco Berlanda
UX Front-end engineer by day, UX wizard by night, and an Interaction design ninja all the time. Always on the hunt for those ‘wow, didn’t see that coming!’ solutions to problems.
Author
Latest posts by Marco Berlanda
18. 12. 2025
Development, Front-end, UI, Vue
API Contracts Don’t Protect Vue 3 Frontends… Integration Tests Do
30. 09. 2025
Development, DevOps, Kubernetes
A GitOps Path from Code to OpenShift Cluster