How AI Agents Decide Which Tool to Use
Learn how AI agents choose tools via function calling, with a hands-on Python example using OpenAI and a step-by-step guide.
Before you start
You have an LLM that can chat, but you need it to actually do things: query a database, send an email, call an API. The agent must decide which tool to use, when, and with what arguments. This article shows you exactly how that decision works and how to implement it in Python.
We will build a minimal agent loop using OpenAI's function calling. You will see the request/response flow, how the model outputs a tool call, and how your code executes it. By the end, you will have a working pattern you can extend to any tool.
You need Python 3.9 or later, an OpenAI API key, and the openai package installed. Run the setup command below to get started.
python -m venv venv
source venv/bin/activate
pip install openai==1.30.0
pip install python-dotenv- Create a .env file with OPENAI_API_KEY=your-key-here
- Use a recent model like gpt-4o-mini for cost efficiency
- Keep your tools simple: start with two functions
The core problem: from text to action
When you ask an agent 'What is the weather in Berlin?', it cannot fetch weather data itself. The model must decide that a get_weather tool exists and that it should call it with city=Berlin. This decision happens through a structured protocol called function calling.
The model does not execute code. It returns a JSON object describing which function to call and with what arguments. Your application code then runs that function and sends the result back to the model. This loop continues until the model produces a final answer.
Here is the minimal agent loop in Python. It sends the user message, checks if the response contains a tool call, executes the function, and appends the result.
import json
from openai import OpenAI
client = OpenAI() # reads OPENAI_API_KEY from env
def get_weather(city: str) -> str:
# In reality, call a weather API.
return f"The weather in {city} is sunny, 22C."
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a city",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string", "description": "City name"}
},
"required": ["city"]
}
}
}
]
messages = [{"role": "user", "content": "What is the weather in Berlin?"}]
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
tools=tools,
tool_choice="auto"
)
msg = response.choices[0].message
print(msg)
if msg.tool_calls:
# Execute the tool call
tool_call = msg.tool_calls[0]
args = json.loads(tool_call.function.arguments)
result = get_weather(args["city"])
print("Tool result:", result)- The model may return multiple tool calls; loop through them.
- Always append the assistant message and the tool response to messages.
- tool_choice auto lets the model decide; you can force a tool with tool_choice={"type":"function","function":{"name":"get_weather"}}.
Step 1: Define your tools with schemas
The first step is to define what tools exist. Each tool needs a JSON schema that describes its name, description, and parameters. The model reads these schemas to decide which tool fits the user's request.
Write clear descriptions. The model uses them to map user intent to the right tool. For example, if you have both get_weather and get_time, the description must mention city for weather and timezone for time.
Here is a schema for two tools: one to get the time in a timezone, and one to calculate a discount.
tools = [
{
"type": "function",
"function": {
"name": "get_time",
"description": "Get current time in a given IANA timezone",
"parameters": {
"type": "object",
"properties": {
"timezone": {"type": "string", "description": "IANA timezone, e.g., Europe/Berlin"}
},
"required": ["timezone"]
}
}
},
{
"type": "function",
"function": {
"name": "calculate_discount",
"description": "Calculate discounted price given original price and discount percent",
"parameters": {
"type": "object",
"properties": {
"price": {"type": "number"},
"discount_percent": {"type": "number"}
},
"required": ["price", "discount_percent"]
}
}
}
]- Use descriptive names that are self-explanatory.
- List all required parameters; the model will fill them from the conversation.
- Keep parameter types strict (string, number, boolean) to avoid parsing errors.
Step 2: Run the decision loop
Now you need a loop that sends the conversation, receives a response, and if the model asks for a tool call, executes it and sends the result back. This loop continues until the model responds with a normal message.
Here is a complete loop that handles any number of tool calls in one response. It uses a dictionary to map function names to actual Python functions.
The loop appends each tool result as a message with role tool. The model then uses that result to craft a final answer.
import json
from openai import OpenAI
client = OpenAI()
# Tool implementations
def get_time(timezone: str) -> str:
from datetime import datetime
import zoneinfo
tz = zoneinfo.ZoneInfo(timezone)
return datetime.now(tz).isoformat()
def calculate_discount(price: float, discount_percent: float) -> str:
final = price * (1 - discount_percent / 100)
return f"Discounted price: {final:.2f}"
function_map = {
"get_time": get_time,
"calculate_discount": calculate_discount,
}
messages = [{"role": "user", "content": "What is the time in New York? Also, give me the price of a $100 item with 15% off."}]
for _ in range(5): # safety limit
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
tools=tools,
tool_choice="auto"
)
msg = response.choices[0].message
messages.append(msg)
if not msg.tool_calls:
break
for tc in msg.tool_calls:
fn = function_map[tc.function.name]
args = json.loads(tc.function.arguments)
result = fn(**args)
messages.append({
"role": "tool",
"tool_call_id": tc.id,
"content": result
})
print(messages[-1].content)- Always include a safety limit to prevent infinite loops.
- The tool response must include the tool_call_id from the assistant message.
- If the model returns multiple tool calls, execute them all and append all results before the next request.
Step 3: Control tool choice with tool_choice
Sometimes you want to force the model to use a specific tool, or prevent it from using any tool at all. The tool_choice parameter gives you that control.
Set tool_choice to none to disable tool calling, auto to let the model decide, or a specific function object to force that tool.
This is useful when you have a classification step: first force a classify tool, then based on its output, choose the next tool.
# Force no tool call
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
tools=tools,
tool_choice="none"
)
# Force a specific tool
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
tools=tools,
tool_choice={"type": "function", "function": {"name": "get_time"}}
)- Use none when you want a plain chat response, e.g., for small talk.
- Use forced tool for routing: first classify the request, then branch.
- Auto is the default and works for most cases.
Step 4: Handle errors and edge cases
Real agents fail: the model might call a tool with missing arguments, the function might throw an exception, or the tool result might be too long. You need to handle these gracefully.
Wrap tool execution in a try-except and return a meaningful error message to the model. The model can then ask the user for clarification or try another tool.
Also validate arguments before calling the function. The model may output invalid JSON or wrong types.
for tc in msg.tool_calls:
fn_name = tc.function.name
try:
args = json.loads(tc.function.arguments)
except json.JSONDecodeError:
result = "Error: invalid arguments JSON"
else:
try:
result = function_map[fn_name](**args)
except Exception as e:
result = f"Error: {str(e)}"
messages.append({
"role": "tool",
"tool_call_id": tc.id,
"content": result
})- Always catch exceptions and return a string message.
- If the tool is not in function_map, return an 'unknown tool' error.
- Consider truncating long tool results to avoid token limits.
Step 5: Add a routing layer for complex agents
For production agents, you often need a routing layer: first classify the user intent, then decide which set of tools to expose. This reduces the chance of the model picking the wrong tool from a large list.
You can implement routing by making an initial call with a small set of 'router' tools, each representing a category. Based on the category, you then call the main model with the relevant tools.
Here is a minimal router that decides between 'weather' and 'calculator' domains.
router_tools = [
{"type": "function", "function": {"name": "route", "description": "Route to domain", "parameters": {"type": "object", "properties": {"domain": {"enum": ["weather", "calculator"]}}, "required": ["domain"]}}}
]
# First call: get routing decision
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": user_input}],
tools=router_tools,
tool_choice={"type": "function", "function": {"name": "route"}}
)
domain = json.loads(resp.choices[0].message.tool_calls[0].function.arguments)["domain"]
# Then use domain-specific tools
if domain == "weather":
tools = weather_tools
else:
tools = calc_tools- Routing reduces the number of tools the model sees, improving accuracy.
- Keep the router schema simple: only a few options.
- You can also use a plain text classification instead of a tool call.
What I would do: recommended setup
For a production agent, I would start with a small set of well-described tools, use tool_choice auto, and implement robust error handling. I would also log every tool call for debugging and observability.
Here is a copy-paste starter that includes logging and a simple retry mechanism. It is ready to extend with your own tools.
import json, logging
from openai import OpenAI
logging.basicConfig(level=logging.INFO)
client = OpenAI()
# Define your tools and functions here (as above)
def run_agent(user_input: str, max_turns: int = 3) -> str:
messages = [{"role": "user", "content": user_input}]
for turn in range(max_turns):
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
tools=tools,
tool_choice="auto"
)
msg = response.choices[0].message
messages.append(msg)
logging.info(f"Turn {turn}: {msg.tool_calls}")
if not msg.tool_calls:
return msg.content
for tc in msg.tool_calls:
try:
args = json.loads(tc.function.arguments)
result = function_map[tc.function.name](**args)
except Exception as e:
result = f"Error: {e}"
messages.append({"role": "tool", "tool_call_id": tc.id, "content": result})
return "Max turns reached"- Set max_turns to prevent runaway loops.
- Log tool calls to trace decisions.
- Store conversation history in a database for multi-turn interactions.
Verify it worked
Run the starter with a sample input and check the output. You should see the agent call the weather tool and then produce a final answer.
Run the command below to test your setup.
python agent.py
# Expected output:
# The weather in Berlin is sunny, 22C.- If you see a tool_call_id error, make sure you append the assistant message before the tool response.
- If the model does not call any tool, check your tool descriptions and parameters.
- If the model hallucinates arguments, tighten the schema with enums or patterns.
Troubleshooting
Common issues include the model calling the wrong tool, invalid JSON arguments, and infinite loops. Here are specific fixes.
First, improve your tool descriptions. The model relies on them for selection. Second, validate arguments in your function. Third, always include a loop limit.
- Wrong tool: add more specific descriptions and use routing.
- Invalid JSON: wrap json.loads in try-except and return an error message.
- Infinite loop: set a max_turns counter and break after that.
- Tool result too long: truncate or summarize the result before sending back.
- Rate limits: add exponential backoff to your API calls.
FAQ
Answers to the questions that come up most often on this topic.
- Q: Can the model call multiple tools in one response? A: Yes, it can return multiple tool_calls. Loop through them and execute all.
- Q: How do I add a new tool? A: Add a schema to the tools list and a corresponding function in function_map.
- Q: What if the model returns a tool call with missing arguments? A: Your function should handle missing args with defaults or return an error message.
- Q: Is tool_choice auto always best? A: For general agents yes, but forced routing can improve accuracy for specific tasks.
- Q: How do I test without spending money? A: Use a local model like Ollama with function calling support, or mock the API response.
Next step
Now that you understand the core loop, extend it with a real API call. Replace the mock get_weather function with a call to OpenWeatherMap or a similar service.
Create a new function that fetches data from an external API and returns a string. Then test your agent with a live request.
import requests
def get_weather(city: str) -> str:
api_key = "YOUR_API_KEY"
url = f"http://api.openweathermap.org/data/2.5/weather?q={city}&appid={api_key}&units=metric"
resp = requests.get(url)
data = resp.json()
return f"{data['main']['temp']}C, {data['weather'][0]['description']}"- Sign up for a free OpenWeatherMap key.
- Handle HTTP errors in your function.
- Test with a few cities.
Key takeaways
- Apply one concrete change from this post before collecting more reading.
- Prefer browser-side tools when the work involves secrets, tokens, or PII.
- Document the why next to the how so the next reviewer inherits context.
FAQ
- Who is this guide on ai-agents for?
- Working developers who need a practical take on how ai agents decide which tool to use — not a marketing overview. Skim the sections, apply one tip, then come back when you hit an edge case.
- Do I need an account to use the related tools?
- No. code.live tools run in your browser with no signup. Nothing you paste is uploaded to a server for the client-side utilities linked from this post.
- How often is this article updated?
- This post was published November 20, 2026. Fundamentals stay stable; check linked tool pages and official docs when version-specific behavior matters.