What you will learn How memory inside a conversation differs from memory across conversations, why long sessions quietly degrade, and what to write down — and never write down.
As we saw in the context window, the model "remembers" only because the whole conversation is re-read every turn. Close the window and nothing is left.
For one person that is an annoyance. For an organisation it is a root cause of failed adoption.
MIT NANDA's 2025 study of over 300 enterprise AI initiatives named it directly: the reason 95% of them showed no measurable financial impact was not model quality and not regulation, but the absence of learning and memory in the deployed systems. A tool you have to re-brief every morning never makes it into the workflow.
HyperTeamsgraph TD
A["Layer 1 — what it is looking at now<br/>this whole conversation"] --> B["Layer 2 — this conversation, compressed<br/>earlier turns summarised"]
B --> C["Layer 3 — the record across sessions<br/>written to files"]| Layer | Where it lives | When it disappears |
|---|---|---|
| What it sees now | the context window | when you close the window |
| This conversation, compressed | inside the context window | when you close the window |
| The record across sessions | files and storage | when you delete it |
Layers 2 and 3 are not the same thing. Layer 2 compresses so the current conversation can keep going; layer 3 writes outside so tomorrow can use it. Confuse them and you assume "it summarised, so it remembers" — then re-explain everything the next day.
When context gets pushed out covered what happens when the front falls off the window. But trouble starts before anything is pushed out.
As stale tool results, finished intermediate steps, and repeated re-reads of the same file pile up, attention to what actually matters gets diluted.
The industry calls this context rot. What makes it dangerous is that no error is raised. Nothing fails suddenly; answer quality slides down gradually and invisibly.
The remedy is to cut and hand off. A wrap-up followed by a fresh conversation almost always beats one very long one.
This is what keeps a long piece of work moving. The principle is simple.
Recent turns need detail; old ones only need their conclusion. Tools can do this automatically, but you should look at what the summary drops. If the reason behind a decision is cut, you will re-litigate that decision a step later.
To cross a session boundary it has to be in a file outside the conversation. Not an elaborate system — one text file is enough to start.
Write down
| What | Example |
|---|---|
| Background you had to re-explain | "Invoices arrive as PDFs in the finance mailbox" |
| Settled rules and their reason | "Over 1M won needs human approval — audit requirement" |
| Approaches that failed, and why | "Screen automation broke when the screen changed" |
| Roles and routes, not personal names | "Billing questions go to the finance team address" |
Never write down
The first line matters most. A credential written into memory once is read back into the context of every session that follows. The "where do the credentials live" question from tools are permissions applies here unchanged.
One fact per file. Once a file starts growing, nobody can tell which part is current. When a note turns out to be wrong, delete it rather than editing around it.
| What it does | Where |
|---|---|
| Capture a repeated procedure for reuse | capturing procedures as skills |
| Keep referring to files in a work folder | registering a work folder |
| Share material with the team | Drive |
Only within that conversation. The summary also lives in the context window, so it disappears with it. To use it tomorrow it has to be written to a file outside.
No. Accumulated notes consume context in every session, and a stale note becomes the basis for a wrong answer. Good memory is not large, it is maintained. Without a routine for deleting, it becomes debt within six months.
1. How does an in-conversation summary differ from a cross-session record?
The summary lives in the context window and disappears when it closes; the cross-session record lives in a file outside and persists until deleted. Confusing the two leads to assuming it remembers, then re-explaining tomorrow.
2. Why is context rot dangerous?
Because it raises no error. As stale tool results and finished intermediate steps accumulate, quality slides down gradually and invisibly. Quietly breaking an earlier rule, or answers drifting toward generalities, are the signs.
3. What must never go into memory?
Credentials — API keys, passwords, tokens. Once written they are read back into the context of every later session. Customer personal data does not go in either without a legal basis and a retention period.
With memory handled, it's time to sharpen the instruction side → The four parts of a good instruction
Yesterday: you spent 30 minutes explaining your invoice format and account codesToday: new conversation → explain it all againSigns to watch for: □ It quietly breaks a rule you set earlier □ It re-checks facts already confirmed □ Answers drift toward generalitiesRecent turns → kept verbatimEarlier turns → replaced by a summary✗ API keys, passwords, tokens → replayed into every later session✗ customer personal data → needs a legal basis and a retention period✗ what the repo or chat log already has → duplicates start disagreeing