In Part 2 you saw the agent loop: observe, think, act, observe again. The thinking part is the language model reasoning about what to do next. The acting part is where things get interesting, because a language model does not actually do anything. It predicts text. It cannot query a database, send an email, check the weather or run a calculation on its own. Everything an agent 'does' in the world happens through tools, and the mechanism that connects a model's output to a real action is called function calling. This article is about that connection: what a tool actually is, how function calling works under the hood, how to design tools a model can use well, and the mistakes that make tool use unreliable.
Where tools fit in the agent loop
Recall the loop from Part 2: the model observes the current state (the user's request, any prior results), thinks about what to do next, and then acts. The 'act' step has exactly two possible shapes. Either the model produces a final answer in plain text, or it produces a request to call a tool. If it requests a tool call, your code runs that tool, captures the result, and feeds the result back into the conversation as a new observation. The model then thinks again, with that new information in hand. This repeats until the model decides it has enough to answer, or until some stopping condition kicks in (Part 8 covers stopping conditions in depth).
So a tool is not a feature bolted onto a chatbot. It is the only way an agent's reasoning turns into an effect outside the conversation. Get tool design wrong and the whole loop becomes unreliable, no matter how good the underlying model is.
What function calling actually is
It helps to be precise about what is and is not happening. The model is not executing code. It has no access to your database, your filesystem or the internet. What it can do is produce structured output that follows a format your application recognizes as 'please call this function with these arguments'. Your application reads that structured output, runs the corresponding real function in your own code, and sends the result back to the model as part of the conversation. The model then continues, now with that result visible to it as text.
This means function calling has three separate pieces, each owned by a different party:
- The tool definition: a description of what the function does and what arguments it takes, written by you and shown to the model before it starts reasoning.
- The tool call: a structured request, produced by the model, naming a tool and giving arguments, following the shape you defined.
- The tool execution: ordinary code, written and run by you, that actually performs the action and returns a result.
The model only ever touches the first two. It reads definitions and writes calls. It never runs anything. This separation is what makes tool use safe to reason about: you are always in control of what code actually executes, because that code is yours.
Anatomy of a tool
Every tool definition you hand to a model needs three things: a name, a description, and a schema for its parameters. The exact format differs a little between providers and frameworks, but the shape is consistent enough that it is worth learning the pattern rather than any one vendor's syntax. Here is a typical tool definition, written in the JSON Schema style that most providers use as of this writing. Check your specific provider's current documentation for the exact field names, since these details do change between versions.
{
"name": "get_weather",
"description": "Get the current weather for a named city. Use this whenever the user asks about current conditions, temperature, or whether it is raining somewhere. Do not use it for forecasts more than a day ahead.",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "City name, e.g. 'Lisbon' or 'Austin, Texas'"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit to return. Default to celsius unless the user specifies otherwise."
}
},
"required": ["city"]
}
}
Three things are doing real work here. The name is what the model will refer to when it wants to call this tool; keep it short and literal. The description is the single most important field, because it is the only information the model has about when and why to use the tool, and models read it as plain language, not as documentation for a human engineer. The parameters block, written in JSON Schema, tells the model exactly what shape of arguments is valid: which fields exist, their types, which ones are required, and in this case a closed set of allowed values for unit. This schema is also what lets your application validate the model's output before running anything, which matters because models occasionally produce arguments that do not quite match the schema, especially with smaller or older models.
Behind this definition sits an ordinary function in your own code, something like a function that takes a city name and a unit, calls a weather API, and returns a plain result. The model never sees this code. It only ever sees the definition above and, later, the result that comes back.
A full round trip, step by step
It is worth walking through one complete exchange in detail, because the abstraction 'the model calls a tool' hides four separate messages passing back and forth. Suppose the user asks: 'Is it warm in Lisbon right now?'
Step one: your application sends the model the conversation so far, plus the list of available tool definitions, including get_weather from above. Nothing has happened yet except handing the model information.
Step two: the model's response is not a final answer. It is a tool call, something the API returns in a structured field rather than as a chat message. It looks roughly like this.
{
"tool_call": {
"name": "get_weather",
"arguments": {
"city": "Lisbon",
"unit": "celsius"
}
}
}
Step three: your code, not the model, reads this structured call, matches the name to the real Python (or other language) function you wrote, parses the arguments, and runs it. Say the function queries a weather service and returns a temperature of 19 degrees Celsius and the condition 'partly cloudy'. Your code packages that as a tool result and appends it to the conversation.
{
"tool_result": {
"name": "get_weather",
"content": {
"city": "Lisbon",
"temperature_c": 19,
"condition": "partly cloudy"
}
}
}
Step four: your application sends the model the whole conversation again, now including that tool result. The model reads it as plain information, exactly as if a person had told it 'the weather in Lisbon is 19 degrees and partly cloudy', and produces an ordinary text answer: 'It is mild in Lisbon right now, around 19 degrees Celsius and partly cloudy, not especially warm.'
Notice what did and did not happen. The model never touched a weather API. It produced a request, your code satisfied that request, and the model reasoned over the result. This is the entire mechanism, repeated as many times as needed within a single turn of the agent loop. A request that needs three tool calls in sequence is just this four-step pattern happening three times before a final text answer appears.
Why not just ask the model to compute the answer itself
A natural question is why you would bother with all this structure instead of just asking the model to answer directly. Weather is an easy case, since the model genuinely has no way to know current conditions. But the same logic applies even when the model could, in principle, work something out itself, such as arithmetic.
Consider asking a language model to calculate 12.5 percent of 8,450. Language models are not calculators; they predict the next token based on patterns in training text, and arithmetic on specific numbers is exactly the kind of task where that process can go wrong, especially with larger numbers or multiple steps. The correct answer is 1,056.25. A model might get this right, or it might produce something close but wrong, like 1,050 or 1,062.50, with no visible sign that it is unsure. Hand the same calculation to a calculator tool, a short function that does unit = 8450 * 0.125 and returns the exact value, and the agent gets a guaranteed correct number every time, with no dependence on the model's arithmetic ability at all. The model's job shifts from 'compute this' to 'recognize that this needs computing, extract the right numbers, and call the tool', which is a task language models are considerably better at.
This is the general pattern behind good tool design: push anything that needs to be precise, current, or verifiable out of the model's head and into a tool, and let the model do what it is actually good at, which is understanding intent, extracting structured information from messy language, and composing results into a coherent answer.
How the model chooses among tools, or chooses none
A realistic agent usually has several tools available at once, not one. Suppose an agent has get_weather, search_flights, and convert_currency. When a request comes in, the model is given all three definitions together with the conversation, and it decides, based purely on the descriptions and the user's message, which one applies, or whether none do and a plain text answer is enough.
This decision is made the same way the model makes every other decision: by predicting what is most likely to follow, given everything in its context. There is no separate routing logic unless you build one yourself. This has a practical consequence worth sitting with: tool selection is only as reliable as the descriptions are clear and as distinguishable as the tools are from each other. If get_weather and get_forecast have descriptions that overlap heavily, expect the model to occasionally pick the wrong one, not because it is confused in some mysterious way, but because you gave it genuinely ambiguous instructions in natural language, and it did what ambiguous instructions tend to produce.
Most tool-calling setups also let the model answer in plain text when no tool is needed at all. If a user asks 'what's the capital of Peru', a well-designed agent should recognize no tool applies and just answer, rather than contorting the question to fit one of the available functions. If you notice an agent reaching for a tool on questions it could answer directly, that is usually a sign the tool descriptions are too eager, implying they should be used more broadly than they should.
Parallel calls, sequences, and multi-step tool use
Some requests need more than one tool call before a final answer is possible. These fall into two patterns, and it is worth telling them apart because they behave differently.
Independent calls happen when the two pieces of information do not depend on each other. 'What's the weather in Lisbon and in Porto?' needs two calls to get_weather, but neither call needs the result of the other. Some providers let a model request both calls in a single turn, which you then execute (possibly concurrently) and return together. This is faster than doing them one at a time, but it is an optimization, not something every provider or model supports in the same way, so check current documentation before relying on it.
Dependent calls happen when one call's arguments come from another call's result. 'What's the weather where my last flight landed?' needs search_flights first, to learn the destination city, and only then can get_weather be called with that city as an argument. Here the calls must happen in sequence: call one, feed the result back, let the model think again and produce call two, execute that, feed its result back, and only then does the model have enough to answer. This is exactly the agent loop from Part 2, just with the 'act' step resolving to a tool call more than once before resolving to a final answer.
It is this second pattern, chained and conditional tool use, that separates an agent from a single-shot tool-augmented prompt. A script that always calls the same tool in the same order is a workflow (Part 4 draws this line precisely). An agent is what you have when the number and order of tool calls is not fixed in advance but decided by the model, turn by turn, based on what it has learned so far.
Designing tools a model can actually use well
Writing a tool is, in an important sense, writing documentation for a reader who cannot ask clarifying questions and who will act immediately on its best guess. A few practical habits make a measurable difference.
Keep each tool narrow and single-purpose
A tool called manage_calendar that can create, delete, move, and list events, all through a single 'action' parameter, is harder for a model to use correctly than four separate tools: create_event, delete_event, move_event, list_events. The narrower tools each have an obvious job, a short description, and a small, unambiguous parameter list. The model is choosing among clearly distinct options rather than filling in a generic action field correctly, which is a strictly easier task.
Write descriptions for the model, not for a future colleague
Compare two descriptions for the same tool. 'Retrieves metereological data via REST from the internal WX service' tells a human engineer where the data comes from but tells the model almost nothing about when to use it. 'Get the current weather for a named city. Use this whenever the user asks about current conditions, temperature, or whether it is raining somewhere' tells the model exactly the situations in which this tool is the right choice. The description is read at decision time, in the same way the rest of the prompt is read, so it should be written the way you would write an instruction, not a code comment.
Make required fields and types explicit, and keep them minimal
If a parameter has a small fixed set of valid values, declare it as an enum rather than a free-text string, as with the unit field earlier. This removes a whole category of mistakes, since the model is choosing from a visible list rather than guessing at acceptable spellings. Keep optional parameters genuinely optional with sensible defaults described in the schema, and resist adding parameters 'just in case'; every extra field is one more thing the model has to decide correctly.
Return results the model can read, and say what went wrong when it can't
A tool result does not need to look like a raw API response. If the underlying weather API returns forty fields of metadata, pass back the three the model needs in plain terms, not a wall of JSON it has to parse by eye. Equally important: when a tool fails, do not let the failure crash silently or return an empty result that the model might mistake for 'no weather data exists'. Return a clear, short error message as the tool result, such as 'city not found: check the spelling or provide a country', and let the model see that and react, perhaps by asking the user to clarify, or trying a different spelling. A model that receives a clear error can often recover gracefully within the same turn. A model that receives nothing, or a result that looks superficially valid but is wrong, usually cannot.
Common mistakes
Most tool-use failures trace back to a small number of recurring problems, and it is worth naming them directly since they are easy to miss when you are the one who wrote the tool and already know what it does.
- Vague or overlapping descriptions. Two tools whose descriptions could both plausibly apply to the same request will cause inconsistent choices between runs, not because the model is unreliable in general but because you have given it a genuinely ambiguous decision.
- Too many tools at once. Every tool definition sits in the model's context and competes for attention; an agent with thirty tools available for every request is harder to steer than one with five relevant ones, and selection accuracy tends to degrade as the list grows. Group tools and only expose what is relevant to the current task where you can.
- Trusting tool output without validation. A tool call's arguments come from the model's prediction, not from a trusted source; validate them against the schema and sane bounds before running anything that writes data, spends money, or sends a message, the same way you would validate any external input.
- No limit on tool-calling turns. If a model gets stuck in a loop, calling the same tool repeatedly without making progress, and there is no cap on how many turns a single request may take, you can burn time and money without ever reaching an answer. Part 8 covers stopping conditions, but the short version here is: always have a maximum number of tool calls per request.
- Letting the model invent a tool that does not exist. If the available tools genuinely cannot satisfy a request, a well-prompted model should say so in plain text rather than attempting an invalid call. If you see tool calls to names you never defined, that is usually a sign the system prompt or tool descriptions did not make the boundaries of what is available clear enough.
- Silent partial failures. A tool that returns an empty list when it actually means 'no results found' versus 'the search itself failed' looks the same to the model unless you distinguish them in the result. The model will then confidently report 'there are no flights' when the truth is 'the flight search was down'.
- Treating the first draft of a tool description as final. Good descriptions tend to be found by trial, not designed perfectly upfront; if an agent keeps misusing or ignoring a tool, the fastest fix is usually rewriting its description and testing again, before suspecting the model or the framework.
Where this shows up beyond chatbots
Function calling is the mechanism underneath far more than conversational agents. Retrieval-augmented generation, where a model searches a document store before answering, is usually implemented as a search tool: the model calls search_documents with a query, gets back relevant passages, and writes an answer grounded in them. Code-executing assistants expose a run_code or run_python tool, letting the model write a snippet, have your sandboxed environment actually execute it, and read back the real output rather than guessing what the code would print. Agents that orchestrate business systems, such as creating support tickets or updating records, do so through tools that wrap existing internal APIs, often the very same APIs a human engineer would call directly, just described in language a model can read and decide to use.
In every one of these cases the pattern from this article holds unchanged: a definition the model reads, a structured call the model produces, and real code that you own and control actually doing the work. The variety is in what the tool does, not in how the mechanism operates.
Summary and what's next
A tool is a function your code can run, described to the model as a name, a plain-language description, and a schema for its arguments. Function calling is the structured format that lets a model request a specific tool with specific arguments, without ever executing anything itself; your application reads that request, runs the real function, and feeds the result back as a new observation in the conversation. A single exchange can involve one tool call or several, called in sequence when later calls depend on earlier results, or in parallel when they do not. Good tools are narrow, clearly described, validated before execution, and informative when they fail; most of the reliability problems people attribute to 'the model' turn out to be ambiguous tool descriptions or missing error handling instead.
With the agent loop and tool mechanics both in place, the next question is when to let a model decide its own sequence of tool calls at all, versus simply writing the sequence yourself. Part 4, Workflows Versus Agents, draws that line and gives you a concrete way to decide which approach a given task actually needs.
Comments (0)
No comments yet. Be the first to share your thoughts.