The short answer
AI agent memory is a set of distinct mechanisms that determine what the agent sees right now, what it recalls from the live conversation, what persists for it across sessions, and what it retrieves from a knowledge base on demand. Confusing these layers — not lacking them — is the most common cause of production agent mistakes.
The four memory layers
The context window
Everything actually sent to the model in one call: instructions, recent conversation turns, and any retrieved results. Its size is finite, and everything it holds is counted and affects both cost and accuracy. A context window crowded with irrelevant material weakens focus even when the material itself is correct.
Short-term memory
Live conversation state: what the user just asked, what they just confirmed, where the flow paused. It typically ends with the session or after idle timeout, and that is intentional — it is not meant to hold durable facts.
Long-term memory
Information deliberately kept for use in later sessions: a user preference, a prior decision, a recurring note. It should be written by an explicit, deliberate decision — not accumulated automatically from every exchange.
Retrieved knowledge
Not memory about the user at all, but general knowledge the agent looks up on demand from an external knowledge base — the RAG pattern. Its advantage is staying current without rewriting the agent's instructions; it is simply replaced when the source updates.
Layer comparison table
| Layer | Lifespan | Example | Misuse risk |
|---|---|---|---|
| Context window | A single call only | The last five conversation turns | Overflow that distracts the agent |
| Short-term | Length of the session | "They just picked the monthly plan" | Mistakenly treated as permanent |
| Long-term | Across sessions, until deleted | "Prefers email over phone" | Stale fact shown as current |
| Retrieved knowledge | Until the source updates | Current return policy from the knowledge base | Retrieval from an outdated source |
What to remember, what not to
- Worth storing: relatively stable preferences, decisions the user confirmed, and process context reused often.
- Not worth storing: transient sensitive data, one-off details, or anything better retrieved on demand from a trusted source than kept in storage.
- Practical rule: if it will change soon or is rarely reused, do not store it — retrieve it when needed instead.
Identity and session scoping
Every stored memory must be tied to a clear identity and session, so one user's memory can never surface in another user's conversation. This is the single most important design constraint in any memory system, because a scoping mistake instantly becomes a privacy and trust failure, not just a technical bug.
Privacy and data minimisation
Data minimisation means keeping the smallest amount needed for the purpose, with a defined lifetime for each stored item and a way to update or delete it. Memory that never forgets is not a feature — it is a quietly accumulating liability.
Common memory failure modes
- Stale memory: a once-true fact shown as current without checking its age.
- Context overflow: stuffing the window with unnecessary material weakens responses and raises cost.
- Cross-user leakage: unclear identity scoping lets one person's memory appear for another.
- Memory-knowledge confusion: storing general facts as personal memory instead of retrieving them from an up-to-date knowledge base.