Tools, Memory and Planning: How AI Agents Actually Work
Learn how AI agents combine tools, memory, and planning by building a minimal agent loop with Python and the OpenAI API.
Before you start
You have probably seen demos where an AI agent books a flight or writes code. Underneath the magic is a simple loop: the model decides what to do, calls a tool, gets a result, and repeats until the task is done. In this article you will build that loop from scratch using Python and the OpenAI API. By the end you will have a working agent that can query a database and send a Slack message, along with a clear mental model of how production agents are structured.
This is a hands-on tutorial. You need Python 3.10 or later, an OpenAI API key, and about 30 minutes. We will keep the code minimal but production-shaped. If you do not have an OpenAI key, you can adapt the examples to any chat-completions API that supports tool calling, such as Anthropic or a local model via Ollama.
python -m venv venv
source venv/bin/activate
pip install openai python-dotenv
- Python 3.10+ installed
- An OpenAI API key with a small credit balance (set as OPENAI_API_KEY)
- A virtual environment for dependencies
- A local SQLite database (we will create one)
- A Slack incoming webhook URL if you want to test the Slack tool
Step 1: Define the agent's tools
An agent is only as useful as the tools it can call. In the OpenAI API, you declare tools as JSON schemas. The model sees these schemas and decides when to call them. Each tool has a name, a description, and a set of parameters. The description is critical: it tells the model when the tool is relevant.
We will define two tools. One queries a SQLite database for customer orders. The other sends a Slack message. The database tool will read from a table we create in the next step. The Slack tool will post a message to a webhook URL.
from openai import OpenAI
from dotenv import load_dotenv
import json, sqlite3, requests, os
load_dotenv()
client = OpenAI()
TOOLS = [
{
"type": "function",
"function": {
"name": "query_orders",
"description": "Get orders from the database. Returns a list of orders with id, customer, amount, and status.",
"parameters": {
"type": "object",
"properties": {
"customer": {"type": "string", "description": "Customer name filter, optional"}
},
"required": []
}
}
},
{
"type": "function",
"function": {
"name": "send_slack_message",
"description": "Send a message to a Slack channel via webhook.",
"parameters": {
"type": "object",
"properties": {
"message": {"type": "string", "description": "The message text"}
},
"required": ["message"]
}
}
}
]
Step 2: Implement the tool functions
Now we write the actual Python functions that the tool names map to. Each function takes a JSON string of arguments and returns a string result. The agent loop will call these functions and feed the result back to the model.
For the database, we will create a simple SQLite database with a few sample orders. The Slack function uses requests to POST to the webhook. In a real system you would add error handling and logging.
def query_orders(customer=None):
conn = sqlite3.connect('orders.db')
cur = conn.cursor()
if customer:
cur.execute("SELECT id, customer, amount, status FROM orders WHERE customer = ?", (customer,))
else:
cur.execute("SELECT id, customer, amount, status FROM orders")
rows = cur.fetchall()
conn.close()
return json.dumps([{'id': r[0], 'customer': r[1], 'amount': r[2], 'status': r[3]} for r in rows])
def send_slack_message(message):
webhook_url = os.getenv('SLACK_WEBHOOK_URL')
if not webhook_url:
return "Error: SLACK_WEBHOOK_URL not set"
resp = requests.post(webhook_url, json={'text': message})
return f"Slack response: {resp.status_code}"
Step 3: Create the database and test data
Before we run the agent, we need the database. Run the following script to create orders.db with a few rows. This is the data the agent will query.
sqlite3 orders.db <<EOF
CREATE TABLE orders (
id INTEGER PRIMARY KEY,
customer TEXT,
amount REAL,
status TEXT
);
INSERT INTO orders (customer, amount, status) VALUES ('Alice', 120.50, 'shipped');
INSERT INTO orders (customer, amount, status) VALUES ('Bob', 75.20, 'pending');
INSERT INTO orders (customer, amount, status) VALUES ('Alice', 200.00, 'delivered');
EOF
Step 4: Write the agent loop
The core of any agent is the loop. We send the user's message and the tool schemas to the model. If the model returns tool calls, we execute them, append the results as a new message, and send it back. We repeat until the model returns a final answer with no tool calls.
Here is the full loop. Note that we maintain a conversation history list, which is the model's short-term memory. In a production agent you would also have long-term memory, but we will cover that later.
def run_agent(user_input):
messages = [{"role": "user", "content": user_input}]
while True:
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
tools=TOOLS,
tool_choice="auto",
)
msg = response.choices[0].message
messages.append(msg)
if msg.tool_calls:
for tc in msg.tool_calls:
func_name = tc.function.name
args = json.loads(tc.function.arguments)
if func_name == "query_orders":
result = query_orders(**args)
elif func_name == "send_slack_message":
result = send_slack_message(**args)
else:
result = f"Unknown tool: {func_name}"
messages.append({
"role": "tool",
"tool_call_id": tc.id,
"content": result
})
else:
return msg.content
if __name__ == "__main__":
print(run_agent("Show me all orders from Alice and then send a slack message summarizing the total."))
Step 5: Run the agent and observe the flow
Run the script. You should see something like: The agent first calls query_orders with customer Alice, gets the rows, then calls send_slack_message with a summary. The final output is a confirmation message.
The key insight is that the model decides the order of tool calls based on the conversation. It can also call multiple tools in parallel if needed. This is the planning part: the model reasons about what to do next, not a fixed script.
If you do not set SLACK_WEBHOOK_URL, the Slack tool will return an error message, and the agent will handle it gracefully by reporting the error to the user. Try it.
python agent.py
- The conversation history grows with each tool call; this is the agent's short-term memory.
- The model uses tool descriptions to decide which tool to call; write them carefully.
- Errors from tools are passed back as text; the model can react to them.
- The loop terminates when the model produces a final message without tool calls.
Step 6: Add planning with a simple planner
The loop above is reactive: the model decides the next step based on the current state. For more complex tasks, you might want an explicit planning phase. One common pattern is to have the model first produce a plan, then execute it step by step.
We can modify the loop to ask for a plan before executing. We add a system message that instructs the model to first output a plan as a numbered list, then execute it. This is a crude but effective way to force planning.
Here is a modified system prompt and a small change to the loop: we add a system message and check if the final answer starts with 'PLAN' to enforce a two-phase approach.
PLANNER_SYSTEM = "You are a helpful agent. First, output a plan as a numbered list. Then execute the plan by calling tools. Finally, give the answer."
def run_agent_with_plan(user_input):
messages = [
{"role": "system", "content": PLANNER_SYSTEM},
{"role": "user", "content": user_input}
]
while True:
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
tools=TOOLS,
tool_choice="auto",
)
msg = response.choices[0].message
messages.append(msg)
if msg.tool_calls:
# same tool execution as before
for tc in msg.tool_calls:
func_name = tc.function.name
args = json.loads(tc.function.arguments)
if func_name == "query_orders":
result = query_orders(**args)
elif func_name == "send_slack_message":
result = send_slack_message(**args)
else:
result = f"Unknown tool: {func_name}"
messages.append({"role": "tool", "tool_call_id": tc.id, "content": result})
else:
return msg.content
Step 7: Add memory with a vector store
Short-term memory is the conversation history. Long-term memory lets the agent remember facts across sessions. A common approach is to store embeddings of past conversations or documents in a vector database and retrieve relevant snippets when needed.
We will use a simple in-memory vector store with numpy to keep it dependency-light. In production you would use something like Pinecone or pgvector. The idea is to embed a user query, find similar past interactions, and inject the results into the system prompt.
Here is a minimal implementation using OpenAI's embedding model. We store a few example memories and retrieve the top match.
import numpy as np
def embed(text):
resp = client.embeddings.create(model="text-embedding-3-small", input=text)
return resp.data[0].embedding
memory_store = [
("Alice prefers email over Slack", "communication_preference"),
("Bob is on vacation until March", "availability")
]
def retrieve_memory(query):
if not memory_store:
return ""
q_emb = np.array(embed(query))
best = None
best_score = -1
for text, _ in memory_store:
t_emb = np.array(embed(text))
score = np.dot(q_emb, t_emb) / (np.linalg.norm(q_emb) * np.linalg.norm(t_emb))
if score > best_score:
best_score = score
best = text
return best if best_score > 0.7 else ""
Step 8: Combine memory with the agent loop
Now we integrate memory retrieval into the agent. Before sending the user message, we retrieve relevant memories and prepend them as a system message. This gives the agent context it would otherwise lack.
This is a simplified version of how agents like ChatGPT's memory feature work. The key is that the retrieved context is injected into the prompt, not stored in the model itself.
def run_agent_with_memory(user_input):
mem = retrieve_memory(user_input)
messages = []
if mem:
messages.append({"role": "system", "content": f"Relevant memory: {mem}"})
messages.append({"role": "user", "content": user_input})
# continue with the same loop as before, using TOOLS
while True:
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
tools=TOOLS,
tool_choice="auto",
)
msg = response.choices[0].message
messages.append(msg)
if msg.tool_calls:
for tc in msg.tool_calls:
func_name = tc.function.name
args = json.loads(tc.function.arguments)
if func_name == "query_orders":
result = query_orders(**args)
elif func_name == "send_slack_message":
result = send_slack_message(**args)
else:
result = f"Unknown tool: {func_name}"
messages.append({"role": "tool", "tool_call_id": tc.id, "content": result})
else:
return msg.content
What I would do in production
The examples above are minimal but real. For a production agent, I would add several layers: structured logging, rate limiting, retries, and a more robust planning mechanism. Also, I would separate the tool definitions into their own module and use a registry pattern.
Here is a recommended setup for a production agent. It uses environment variables for configuration, a tool registry, and a simple retry decorator. This is a starting point you can copy and extend.
# config.py
import os
from dotenv import load_dotenv
load_dotenv()
OPENAI_MODEL = os.getenv("OPENAI_MODEL", "gpt-4o-mini")
SLACK_WEBHOOK_URL = os.getenv("SLACK_WEBHOOK_URL", "")
DATABASE_PATH = os.getenv("DATABASE_PATH", "orders.db")
# registry.py
TOOL_REGISTRY = {}
def register_tool(func):
TOOL_REGISTRY[func.__name__] = func
return func
@register_tool
def query_orders(customer=None):
# implementation
pass
@register_tool
def send_slack_message(message):
# implementation
pass
# agent.py
import openai
from config import OPENAI_MODEL
from registry import TOOL_REGISTRY
def run_agent(user_input):
messages = [{"role": "user", "content": user_input}]
tools = [{"type": "function", "function": {"name": name, "description": func.__doc__, "parameters": ...}} for name, func in TOOL_REGISTRY.items()]
# loop...
- Use a tool registry to avoid if-else chains.
- Log every tool call with request/response for debugging.
- Set a max iteration count to prevent infinite loops.
- Use retries with exponential backoff for API calls.
- Store conversation history in a database for resumability.
Troubleshooting
If the agent does not call tools as expected, check the tool descriptions. The model relies on them heavily. If the agent loops forever, add a max iterations parameter. If you get a 401 error, check your API key. If the Slack message is not sent, verify the webhook URL.
Common issues and fixes:
- No tool calls: Make the tool description more explicit or add an example in the system prompt.
- Tool calls with invalid arguments: Validate arguments in the function and return a clear error message.
- Infinite loop: Add a counter and break after 10 iterations.
- Memory not retrieved: Increase the similarity threshold or use a better embedding model.
FAQ
- Q: Do all agents use the same loop? A: Most use a similar loop, but some frameworks like LangGraph or AutoGPT add more structure like state machines.
- Q: Can I use local models? A: Yes, if they support function calling. Ollama and vLLM have compatible APIs.
- Q: How do I handle multiple tools with the same name? A: Use a registry and unique names.
- Q: What is the difference between tools and plugins? A: Tools are functions the model can call; plugins are bundles of tools and configuration.
- Q: How much does this cost? A: With gpt-4o-mini, a few cents per session. Use caching to reduce costs.
Next step
You now have a working agent loop with tools, memory, and planning. The next step is to extend it with more tools, add a real vector database, and deploy it as a service. Start by adding a tool that reads a file, then connect it to a simple web API.
Run the agent with your own data and see how it behaves. The code is yours to modify.
Key takeaways
- Apply one concrete change from this post before collecting more reading.
- Prefer browser-side tools when the work involves secrets, tokens, or PII.
- Document the why next to the how so the next reviewer inherits context.
FAQ
- Who is this guide on ai-agents for?
- Working developers who need a practical take on tools, memory and planning: how ai agents actually work — not a marketing overview. Skim the sections, apply one tip, then come back when you hit an edge case.
- Do I need an account to use the related tools?
- No. code.live tools run in your browser with no signup. Nothing you paste is uploaded to a server for the client-side utilities linked from this post.
- How often is this article updated?
- This post was published November 15, 2026. Fundamentals stay stable; check linked tool pages and official docs when version-specific behavior matters.