The Reality of Long-Running AI Development: The Context Wall and Running a Memory

Develop with an AI long enough and you always hit the same wall: the AI starts forgetting the beginning.

As a conversation grows long, the AI summarizes and compresses the earlier exchanges (a mechanism called compacting). It can't hold a long history verbatim, so the design makes sense. But summarizing drops details, useful as it is. The exact numbers in a spec, the reason behind a decision, a pitfall you once hit and decided to avoid: those thin out with every round of summarizing.

The context wall

The trouble is that once you keep working off a summarized conversation, you drift from the original spec without noticing.

The AI carries on plausibly, using whatever summarized context it has left. The result runs convincingly on its own but conflicts with the spec you set at the start. That gap is where most of the rework in a long session comes from. Even the feeling of "I just checked that, so it's fine" may be riding on a memory that summarizing has already thinned.

What we treat as truth: the spec and the plan

First, what counts as truth. Not the memory inside the conversation, but two documents: the spec and the plan.

• The spec decides what to build. It records the design intent and the contract we have to uphold: the shape of the exchange between screen and server, the definition of a database column, a function's inputs and outputs.
• The plan is the order we implement it in. Each stage's definition of "done" is written up front as numbers and commands ("this search returns zero rows," "this test passes"), in a form a machine can check.

The keystone is the Appendix at the end of the spec. Every value tied to a contract goes there, in one place, instead of being scattered through the prose. And we don't write it from guesswork. We record it as the result of checking the real thing directly, noting what we verified and as of when, as the single source of truth. The body and the plan never restate those values; they just point to it: "see Appendix A.1."

Why go this far? Because the same fact written in two places will eventually go stale in one of them, and the two will disagree. That is drift. Keep it in one place, and even when summarizing thins your memory, the appendix takes you back to the primary source. So before starting on implementation, we re-read the spec, the plan, and the appendix every time. The moment you feel "I know this, it's in memory" or "I checked this earlier" is exactly when you stop and pull the real thing again. A fact written in a file is always more reliable than your own summarized memory. For that matter, even the "here is the current state" you get handed at the start of a session can be slightly stale, so before branching off, refetch the real thing and then move.

Rebuild the context often

The second part is how you handle the context itself.

Compacting happens automatically, but you don't get to choose the summary. You can't control what stays and what drops. And the more context you carry, the more the model's attention scatters and the less of its real ability it can bring to bear. A swollen context is like trying to think at a cluttered desk.

So we develop while watching how much context is left, and before the automatic summary kicks in, at a clean stopping point, we clear it ourselves and start a fresh session. Clearing resets the context to blank, and the model, now travelling light, can perform at full strength.

It looks wasteful at first, but clearing and starting over is far easier to handle than leaving it to automatic summarizing. You decide where to draw the line and what to carry forward. Stacking short, cleanly separated sessions keeps more control, and better quality, than dragging one long session along under a hazy summary.

Hand off through files: across PCs and offices

Before clearing, we leave a handoff for the next session. We use two tools that write out session state.

One is remember, a Claude Code plugin. It bundles the current session's work log into units like the day's entry or the last few days and writes them out, so the next session can read them back. It works like short-term memory for quickly recovering how far you got.

The other is one we built ourselves, memory-write. It came out of a specific problem: when a session ends, the learnings and decisions you earned disappear, and if you pick the work up on another PC, none of it carries over. To keep long-lived facts as a permanent memory, we wrote our own plugin. What it does: classify a fact worth keeping by kind (the project's rules, the state of work in progress, past mistakes and their lessons), write it to a memory file, update an index, and commit it to git.

The crux is putting it on git. On a different PC, or in a different office, a git pull brings the same learnings straight to hand. Write down once that "doing it this way breaks here" or "that assumption was wrong," and the next session's AI won't step on the same trap, wherever it resumes. You can't carry a conversation's context with you, but the facts and handoffs you offloaded into files can be picked up from anywhere. The place changes; the work continues.

What you can delegate, and what you can't

Put it together and the line becomes clear.

The context wall doesn't go away. So we don't use the AI on the assumption that it "remembers everything forever." The facts worth remembering live outside the AI's memory, in the spec, the plan, and the memory store. What we hand to the AI is the work built on those facts. We delegate the work of moving things along; we don't delegate holding onto the facts. Only once you've settled that division can the work keep running for long hours without falling apart.

The AI doesn't remember everything by magic. We assume it forgets, and we put whatever we can't afford to lose on the outside. It's unglamorous, but it turned out to be the most practical discipline for long-running AI development.

The path to this way of running a memory is in How a Micro-SaaS Tech Stack Changed in a Single Year; how we assure quality is in Two-Track Review with Claude and Codex. Other posts on how we build are collected in the dev category.

PentaTrail is what we're building out of these habits. This has been a story about our own development, but the same idea, using AI to keep watch continuously, applies just as well to a company's attack surface.

See PentaTrail / CTEM