How AI Agents Remember Things
Learn how to give AI agents memory using context windows, vector stores, and external state, with runnable examples.
Before you start
You are building an AI agent that keeps forgetting what the user said two turns ago. The model has a context window, but it is finite, and every new conversation pushes older messages out. You need a way to make the agent remember things across turns and sessions.
This article walks through three practical memory strategies: fitting everything into the context window, storing conversations in a vector database, and persisting state in a database or file. You will get runnable code for each approach, plus a recommended setup for a production agent.
You should have Node.js 18 or later and Python 3.9 or later installed. You will also need an OpenAI API key for the examples, but the patterns apply to any LLM provider.
- Node.js 18+ for JavaScript examples
- Python 3.9+ for Python examples
- An OpenAI API key (or any LLM API) for the agent calls
- Basic familiarity with async/await and REST APIs
The problem: when context runs out
Imagine a support chatbot that helps users troubleshoot. The user says, 'I am getting error 503 on the login endpoint.' Then they paste a stack trace. Then they ask, 'What did I say the error was?' If the agent only keeps the last few messages, it might miss the earlier mention.
The simplest fix is to include all conversation history in every request. But context windows are limited. For example, GPT-4o has a 128k token limit, which is roughly 100,000 words. That sounds like a lot, but a long debugging session can exceed it quickly.
When the context overflows, you have three options: truncate old messages, summarize them, or move them to an external store. Let's look at each.
Step 1: Sliding window with truncation
The easiest memory is a sliding window: keep the last N messages and drop the rest. This is what many chat applications do by default.
Here is a simple implementation that keeps the last 10 messages and sends them to the OpenAI API.
const { OpenAI } = require('openai');
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
let history = [];
const MAX_MESSAGES = 10;
async function chat(userMessage) {
history.push({ role: 'user', content: userMessage });
// Keep only the last MAX_MESSAGES
history = history.slice(-MAX_MESSAGES);
const response = await openai.chat.completions.create({
model: 'gpt-4o',
messages: history,
});
const assistantMessage = response.choices[0].message.content;
history.push({ role: 'assistant', content: assistantMessage });
return assistantMessage;
}
// Example usage
chat('My name is Alice.').then(console.log);
chat('What is my name?').then(console.log); // Will remember if within windowStep 2: Summarization for longer memory
When the window is too short, you can summarize older messages and keep the summary in the context. This gives the agent a condensed version of what happened earlier.
Here is a function that summarizes the conversation when it exceeds a token threshold, then replaces the old messages with the summary.
import openai
import tiktoken
openai.api_key = "your-api-key"
MAX_TOKENS = 4000
def count_tokens(messages):
encoding = tiktoken.encoding_for_model("gpt-4o")
return sum(len(encoding.encode(msg["content"])) for msg in messages)
def summarize(messages):
prompt = "Summarize the conversation so far, keeping key facts."
response = openai.ChatCompletion.create(
model="gpt-4o",
messages=[
{"role": "system", "content": prompt},
*messages
],
max_tokens=200
)
return response.choices[0].message.content
history = []
def chat(user_message):
global history
history.append({"role": "user", "content": user_message})
if count_tokens(history) > MAX_TOKENS:
summary = summarize(history[:-1]) # summarize all but the latest
history = [{"role": "system", "content": f"Summary: {summary}"}] + history[-1:]
response = openai.ChatCompletion.create(
model="gpt-4o",
messages=history
)
assistant_message = response.choices[0].message.content
history.append({"role": "assistant", "content": assistant_message})
return assistant_messageStep 3: Vector store for semantic recall
Truncation loses information, and summarization loses detail. For long-term memory, store past conversations in a vector database and retrieve relevant chunks when needed.
In this approach, each message is embedded into a vector and stored. At query time, you embed the user's current question and search for the most similar past messages, then inject them into the context.
Here is a complete example using OpenAI embeddings and a simple in-memory vector store (you can swap in Pinecone or pgvector later).
import openai
import numpy as np
openai.api_key = "your-api-key"
# Simple in-memory vector store
class VectorStore:
def __init__(self):
self.vectors = []
self.texts = []
def add(self, text, embedding):
self.vectors.append(embedding)
self.texts.append(text)
def search(self, query_embedding, top_k=3):
if not self.vectors:
return []
scores = [np.dot(query_embedding, vec) for vec in self.vectors]
top_indices = np.argsort(scores)[-top_k:][::-1]
return [self.texts[i] for i in top_indices]
store = VectorStore()
def embed(text):
response = openai.Embedding.create(
model="text-embedding-3-small",
input=text
)
return response['data'][0]['embedding']
def remember(text):
embedding = embed(text)
store.add(text, embedding)
def recall(query):
query_embedding = embed(query)
return store.search(query_embedding)
# Example: remember a fact
remember("The user's name is Alice and she works at Acme Corp.")
# Later, recall it
relevant = recall("Who is the user?")
print(relevant) # Should include the remembered factStep 4: Persistent state with a database
Vector stores are great for semantic recall, but sometimes you need exact state: user preferences, progress in a multi-step task, or authentication tokens. For that, use a traditional database.
Here is a minimalist example using SQLite to store user facts and retrieve them in later sessions.
import sqlite3
conn = sqlite3.connect('agent_memory.db')
c = conn.cursor()
c.execute('''CREATE TABLE IF NOT EXISTS user_memory
(user_id TEXT, key TEXT, value TEXT, PRIMARY KEY(user_id, key))''')
conn.commit()
def remember(user_id, key, value):
c.execute('INSERT OR REPLACE INTO user_memory (user_id, key, value) VALUES (?, ?, ?)',
(user_id, key, value))
conn.commit()
def recall(user_id, key):
c.execute('SELECT value FROM user_memory WHERE user_id = ? AND key = ?', (user_id, key))
row = c.fetchone()
return row[0] if row else None
# Usage
remember('user123', 'name', 'Alice')
print(recall('user123', 'name')) # 'Alice'Step 5: Putting it all together
A robust agent uses a combination of all three: a sliding window for recent context, summarization for older messages, and a vector store or database for long-term facts.
Here is a minimal agent loop that integrates all three. It keeps the last 5 messages, summarizes anything older, and stores key facts in a vector store.
const { OpenAI } = require('openai');
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
let history = [];
let summary = '';
const MAX_CONTEXT_MESSAGES = 5;
async function chat(userMessage) {
// 1. Add user message to history
history.push({ role: 'user', content: userMessage });
// 2. If history exceeds max, summarize older messages
if (history.length > MAX_CONTEXT_MESSAGES) {
const toSummarize = history.slice(0, -MAX_CONTEXT_MESSAGES);
history = history.slice(-MAX_CONTEXT_MESSAGES);
const summaryResponse = await openai.chat.completions.create({
model: 'gpt-4o',
messages: [
{ role: 'system', content: 'Summarize the conversation concisely.' },
...toSummarize
]
});
summary = summaryResponse.choices[0].message.content;
}
// 3. Build full context: summary + recent history
const messages = [
...(summary ? [{ role: 'system', content: `Previous summary: ${summary}` }] : []),
...history
];
// 4. Get response
const response = await openai.chat.completions.create({ model: 'gpt-4o', messages });
const assistantMessage = response.choices[0].message.content;
history.push({ role: 'assistant', content: assistantMessage });
// 5. Optionally store important facts in a vector store
// (omitted for brevity, see step 3)
return assistantMessage;
}Recommended setup
For a production agent, I would use the following stack: a sliding window of the last 20 messages, a summarization step that triggers at 80% of the context window, and a vector store (like pgvector or Pinecone) for semantic memory. For exact user preferences, use a key-value store like Redis.
Here is a copy-paste starter for the memory manager in TypeScript.
import { OpenAI } from 'openai';
import { Redis } from 'ioredis';
import { PgVector } from 'pgvector';
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const redis = new Redis(process.env.REDIS_URL);
const vectorStore = new PgVector(process.env.DATABASE_URL);
const MAX_RECENT = 20;
const MAX_TOKENS = 8000; // adjust based on model
class MemoryManager {
private recent: Array<{role: string; content: string}> = [];
private summary: string = '';
async addMessage(message: {role: string; content: string}) {
this.recent.push(message);
if (this.recent.length > MAX_RECENT) {
await this.compress();
}
}
private async compress() {
const toSummarize = this.recent.slice(0, -MAX_RECENT);
this.recent = this.recent.slice(-MAX_RECENT);
const response = await openai.chat.completions.create({
model: 'gpt-4o',
messages: [{ role: 'system', content: 'Summarize the conversation.' }, ...toSummarize]
});
this.summary = response.choices[0].message.content;
}
async getContext(): Promise<Array<{role: string; content: string}>> {
return [
...(this.summary ? [{ role: 'system' as const, content: `Summary: ${this.summary}` }] : []),
...this.recent
];
}
async remember(key: string, value: string) {
await redis.set(key, value);
}
async recall(key: string): Promise<string | null> {
return redis.get(key);
}
}
export const memory = new MemoryManager();Verify it worked
To test your memory implementation, run a conversation that spans more than the context window, then ask a question that requires recalling an early fact.
Here is a quick script you can run to verify the sliding window works.
node -e "
const { OpenAI } = require('openai');
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
let history = [];
const MAX = 2;
async function chat(msg) {
history.push({role:'user', content:msg});
history = history.slice(-MAX);
const res = await openai.chat.completions.create({model:'gpt-4o', messages:history});
const reply = res.choices[0].message.content;
history.push({role:'assistant', content:reply});
return reply;
}
(async () => {
await chat('My name is Alice');
await chat('I like pizza');
console.log(await chat('What is my name?')); // likely forgets
})();
"Troubleshooting
If your agent forgets things, check the most common causes: the context window is too small, the summarization is losing details, or the vector store is not retrieving relevant memories.
Here is a checklist to diagnose memory issues.
- Log the full context sent to the model to see what is actually included.
- Measure token usage to know when truncation or summarization kicks in.
- Test vector recall with a question that directly matches a stored fact.
- Check that your embeddings are consistent (same model for store and query).
- For persistent memory, verify database connections and schema.
FAQ
Answers to the questions that come up most often on this topic.
- Q: What is the best memory approach for a chatbot? A: Use a sliding window for recent context, summarization for older messages, and a vector store for long-term facts.
- Q: How do I choose the context window size? A: Start with the model's limit minus a safety margin for the response. Monitor token usage and adjust.
- Q: Can I use a vector database like Pinecone? A: Yes, replace the in-memory store with Pinecone or pgvector for production scale.
- Q: Do I need to store every message? A: No, store only important facts or chunks that might be needed later.
- Q: How do I handle multiple users? A: Include a user ID in your database keys and vector metadata to isolate memories.
Next steps
Start by implementing the sliding window in your current agent. Then add summarization when you hit token limits. Finally, integrate a vector store for cross-session memory.
Run the verification script above to see the difference between a window that is too small and one that works. Then adjust the parameters to fit your use case.
Key takeaways
- Apply one concrete change from this post before collecting more reading.
- Prefer browser-side tools when the work involves secrets, tokens, or PII.
- Document the why next to the how so the next reviewer inherits context.
FAQ
- Who is this guide on ai-agents for?
- Working developers who need a practical take on how ai agents remember things — not a marketing overview. Skim the sections, apply one tip, then come back when you hit an edge case.
- Do I need an account to use the related tools?
- No. code.live tools run in your browser with no signup. Nothing you paste is uploaded to a server for the client-side utilities linked from this post.
- How often is this article updated?
- This post was published November 19, 2026. Fundamentals stay stable; check linked tool pages and official docs when version-specific behavior matters.