Anthropic Shows How AI Agents Keep Memory: 13% to 48% Accuracy
In short
- Anthropic published how it builds memory for AI agents: transcripts, notes, wikis, and an offline step it calls dreaming.
- On a company wiki of about 350,000 files, accuracy rose from 13% to 48% over 128 dreaming sessions. Wiki plus search hit 61%, against 53% for search alone.
- The wiki plus search combo saved about 20,000 tokens per question. At volume, that is real money.
Your AI agent forgets everything the moment the session ends. Anthropic just showed how to fix that, with numbers. If you build agents for a business, this is a free blueprint.
What happened
On November 6, Anthropic’s engineering team published “Components of agent memory”. It splits memory into layers. Transcripts are a searchable log. Memory notes are things the agent writes down during or after a session. Wikis are linked pages built from those notes. On top sits a small injected context that loads into every new session.
Then comes dreaming. Between sessions, the agent reviews old transcripts, fixes wrong notes, adds new pages and decides what to preload. Sounds fancy. It works.
The numbers: on a database task, Opus 5.5 went from a 0.655 to a 0.838 mean reward with memory. On a company document wiki, accuracy climbed from 13% to 48% after 128 dreaming sessions. On multi-document merger questions the wiki scored 87%, search alone 73%.
It also shows what breaks. A wiki on one legal task grew to 1.77 times the size of the source documents and added little. When agents wrote to one shared table, facts got lost: 41 of 57 captured. One page per contract fixed it: 57 of 57.
How to use it
- Keep the top layer tiny. A short index that links to details. Long instruction files hurt adherence.
- Store only what the agent cannot see. Strategies and verified rules, not things the screen already shows.
- One page per writer. Never let two agents edit the same page.
- Run a nightly review. A second pass that checks notes against transcripts catches stale facts before they get trusted.
- Prompt weaker models explicitly. Tell them to read memory before answering. Sonnet 5 did it about 9 times in 10, Sonnet 4.6 under half the time.
Do the math: 20,000 tokens saved per question. Say your support agent handles 1,000 questions a month. That is 20 million tokens. At an assumed €3 per million input tokens, you save about €60 a month, around €720 a year. The price is my assumption, check your own rate. Dreaming costs compute too, so the real saving is smaller.
What it means
Memory is turning from a feature into a craft. The winners will not be the ones with the biggest context window. They will be the ones whose agents keep clean, correct notes. Cheap models with good memory can beat expensive ones with none, at least on some tasks.
One caution. These are Anthropic’s own benchmarks on its own models. Test on your data before you rebuild anything.
Ivan’s Take
Worth reading in full if you build agents. The bloat and shared-table lessons alone will save you a week of debugging. I would copy the one-page-per-item rule today for any inventory or customer notes agent. The benchmark gains I would treat as a ceiling, not a promise.
Source: Anthropic
If it doesn’t save you time or money, you won’t see it here
- Facts, not hypeEvery price and number is checked against the official source
- Always the real priceYou see what it costs before you click
- I say when it’s weakEven when it’s OpenAI or Google
