LLM Agents & Tool Use

agent memory

An LLM only 'remembers' what fits in its current context window, and on a long task that fills up fast. Agent memory is the scaffolding around the model that lets it store useful facts — what it has tried, what it learned, the user's preferences — and pull the relevant pieces back in later, so it does not keep forgetting and repeating itself across a long run.

In practice memory is split by lifespan. Short-term (working) memory is just the recent conversation kept in context. Long-term memory lives outside the model in a database or vector store; the agent writes notes there and, when it needs them, retrieves the relevant entries and pastes them back into the prompt. This is the same retrieval idea used in RAG, applied to the agent's own history rather than a document collection.

Memory is what separates an agent that learns within a task from one that starts every step blank. But stored memories can also be stale, wrong, or contradictory, so deciding what to keep, what to retrieve, and what to forget is itself a hard part of the design.

Also called
agent memory storeworking and long-term memory