How AI Agents Actually Use Tools

It's Still Just Text Prediction

Underneath the abstraction, tool use is the model predicting a structured piece of text — usually JSON — that names a function and supplies arguments, based on a description of available tools it was given in its context. There is no separate 'tool-using' mode; it's the same next-token prediction the model always does, just constrained toward a schema.

This is worth internalizing because it explains most of the failure modes. If a tool's description is ambiguous, the model will sometimes pick the wrong one, not because it 'misunderstood' in some deep sense, but because the description didn't sufficiently narrow down which token sequence was most likely to be correct.

The Round Trip

The actual mechanics: the orchestrating code sends the model a prompt plus a list of tool schemas, the model returns a tool call instead of a plain-text answer, the code executes that call against the real tool, and the result gets appended back into the conversation as if it were another message. The model then continues from there, often producing a final answer or another tool call.

Everything the agent 'knows' about the outside world comes through this channel. It has no other way to sense what actually happened — if the tool result is wrong, misleading, or absent, the agent's next decision is built on that misinformation with no way to independently verify it.

What Makes Tool Use Reliable

Reliable tool use comes down to boring engineering discipline: clear, non-overlapping tool descriptions, strict schemas the model can't easily violate, and error messages that are informative enough for the model to self-correct rather than just retrying the same broken call. A tool that returns a bare 500 error teaches the model nothing useful about what to do differently.

It also helps to keep the number of available tools small and well-differentiated. Models get measurably worse at picking the right tool as the tool list grows and the descriptions start to overlap, which is a strong argument for narrow, purpose-built agents over one agent with fifty tools bolted on.

• Tool calls are structured text output, not a special model capability
• The orchestrating code executes the call; the model never touches the tool directly
• Results are appended to context as messages, which is the model's only window into the world
• Fewer, clearer tools consistently beat a large overlapping tool list