Why an agent needs a safety net
In Part 2 you saw the agent loop: observe, think, act, observe again. That loop is what makes an agent useful, but it is also what makes it dangerous if left unsupervised. A normal program fails in predictable ways: a function throws an exception, you catch it, you log it, you move on. An agent loop can fail in a much quieter way. It can keep running, keep calling tools, keep producing plausible-looking text, and never actually finish or never notice that something has gone wrong. A script that crashes is annoying. An agent that calls a payment API ten times because it never got confirmation of the first call is expensive. An agent that reads a file, then reads it again, then reads it again because it forgot it already has the answer, is slow and costly even though nothing has technically broken.
This article is about the two halves of that safety net. The first half is error handling and retries: when a step fails, what do you do about it? The second half is stopping conditions: how does the agent, or the code around it, know when to give up, succeed, or hand off to a person? Neither of these is exotic. Most of the ideas come straight from ordinary distributed-systems practice. What is new is applying them to a loop where one of the components is a language model that can be wrong in ways a database connection never is.
A map of what can go wrong
It helps to separate failures by where they come from, because the right response is different in each case. Roughly, an agent step can fail for three reasons: the tool failed, the model failed, or the loop itself is stuck even though every individual step looked fine.
Errors from tools
These are the failures you would recognise from any application that calls external services, because that is exactly what is happening. A web search tool times out. A database query returns an error because a table was renamed. An API responds with a rate-limit status because the agent, or other users of the same key, made too many calls in the last minute. A file-reading tool is pointed at a path that does not exist. These are the same failures you would see in a normal backend service, and most of them already have a well understood answer: catch the specific exception, decide if it is worth retrying, and otherwise surface a clear error.
What is different in an agent is who sees the error. In a normal program, the error goes to a log and maybe to a developer. In an agent loop, the error can also go back into the context as an observation, and the model gets to decide what to do next. That is powerful, because a model can sometimes work around a failed tool the way a person would, for example by trying a different search query after the first one returns nothing. It is also a risk, because a model can misread an error message and take a worse action than doing nothing, for example interpreting a timeout as 'the file is empty' and confidently reporting that to the user.
Errors from the model
The second class of failure is specific to agents built on language models, and it did not exist in traditional software at all. The model is asked to produce a tool call with structured arguments, as described in Part 3, and sometimes it does so badly. Three common patterns show up again and again:
- Malformed output: the model returns text that is supposed to be a JSON tool call but is missing a closing brace, uses single quotes where double quotes were required, or wraps the JSON in an explanation the parser was not expecting.
- Wrong or hallucinated arguments: the model calls a real tool with an argument that does not exist, such as passing a user_id when the tool expects an account_id, or it invents a plausible-looking value for a field it does not actually know, such as guessing a file path.
- Hallucinated tools: the model tries to call a tool that was never given to it, often because it remembers a similar tool from its training data or from a previous turn in a long conversation.
None of these are the tool's fault, and none of them are a network problem. They come from the fact that the model is predicting text that looks like a correct tool call, and most of the time that prediction is right, but it is still a prediction. The fix is not a network retry; it is telling the model clearly what went wrong and asking it to try again, which is a different kind of retry, covered below.
Errors from the loop itself
The third class is the hardest to see coming, because no single step fails. The model calls a tool, gets a reasonable result, reasons about it, calls another tool, gets another reasonable result, and so on, forever. This can happen when the task is genuinely ambiguous and the model keeps gathering more information without ever deciding it has enough, when the model forgets earlier progress because the context grew too long (the problem described in Part 7), or when two tools disagree and the model keeps checking one against the other without a way to break the tie. From the outside this looks like an agent that is busy but never finishes. It will not raise an exception, so ordinary error handling will not catch it. This is exactly why stopping conditions have to be a separate concern from error handling, and we will treat them separately in this article.
Deciding what to retry
Before writing any retry code, answer one question for each kind of failure: is trying again likely to produce a different, better result? If the answer is no, retrying just delays an inevitable failure and burns time and money. This splits errors into two broad groups.
Transient versus permanent
A transient error is caused by something temporary: a server was briefly overloaded, a network packet was dropped, a rate limit will reset in a few seconds. Trying the same request again, perhaps after a short wait, has a real chance of succeeding. Typical examples are HTTP 429 (too many requests), HTTP 503 (service unavailable), connection timeouts, and DNS resolution failures.
A permanent error is caused by something that will not change no matter how many times you ask. The request was invalid, the resource does not exist, or the caller does not have permission. Typical examples are HTTP 400 (bad request), HTTP 401 or 403 (authentication or authorisation), and HTTP 404 (not found). Retrying a 404 ten times still gives you a 404 ten times; all you have done is waste ten round trips and, if the model is in the loop, given it ten more chances to misread the error as something else.
The practical rule is: retry on status codes and exceptions that indicate a temporary condition, and fail fast, with a clear message, on anything that indicates the request itself was wrong. If you are not sure which category an error falls into, treat it as permanent on the first attempt and only move it to the retryable list once you have seen it happen and recover after a wait. Guessing in the optimistic direction, 'maybe it will work next time', is how retry loops quietly turn into infinite loops.
Idempotency: can you safely try again?
There is a second question that matters just as much as transient versus permanent, and it is easy to miss: if the first attempt actually succeeded but you never found out, what happens if you do it again? This is the idempotency question, and it matters more for agents than for most software because an agent decides on its own when to retry, without a person checking first.
Consider a tool that searches a knowledge base. Calling it twice with the same query returns the same answer twice. No harm done; this operation is naturally idempotent. Now consider a tool that sends a confirmation email, or charges a card, or appends a row to a spreadsheet. If the first call actually went through but the response was lost to a timeout, retrying will send a second email, charge the card twice, or add a duplicate row. These operations are not idempotent by default.
There are two ways to deal with this, and good systems use both. The first is to make the operation idempotent at the source, usually by attaching an idempotency key, a unique identifier generated once per logical action, so that the receiving system recognises a repeated request with the same key and simply returns the earlier result instead of doing the action twice. Many payment and messaging APIs support this directly. The second, when you cannot change the tool itself, is to be conservative in the agent: for any tool that has a side effect in the real world, do not retry automatically on an ambiguous failure such as a timeout, because a timeout tells you the response was lost, not that the action failed. Instead, surface the uncertainty, either to the model as 'the result of this action is unknown, check before repeating it' or to a person, which is the subject of Part 9.
A simple mental checklist: read-only tools (search, lookup, read a file) are usually safe to retry freely. Write tools (send, charge, delete, post) need an idempotency key, a check-before-retry step, or a human confirmation before a retry is allowed.
Giving the model the error, not just retrying blindly
For failures that come from the model rather than the tool, the fix is different from a network retry. If the model produced malformed JSON, trying the exact same prompt again usually produces the same mistake, because nothing about the input changed. The useful move is to put the parser's error message back into the context and ask the model to correct its own output. For example, if the parser reports 'Expecting property name enclosed in double quotes: line 1 column 14', that specific message, not a generic 'try again', is what lets the model see exactly what it did wrong and fix it.
The same logic applies to a tool call with the wrong arguments. If a tool returns a validation error such as 'missing required field: account_id', feed that error back as the observation for this step, the same way you would feed back a successful tool result. The model then has a chance to notice the mistake and correct the next call. This is sometimes called a repair loop, and it is usually far more effective than looping on the exact same request, because it gives the model new information rather than asking it to guess better at random.
It is worth setting a small limit on repair attempts too, typically two or three, because a model that cannot produce valid arguments after being shown the exact error twice is unlikely to succeed on the fourth try, and at that point the right move is to stop and escalate rather than keep spending tokens on the same mistake.
How to retry: backoff and limits
Once you know an error is worth retrying, the next question is how. Retrying immediately, in a tight loop, is usually a bad idea: if the tool is overloaded, hammering it with requests one after another makes the overload worse, and if many users or many agent instances are doing this at once, it can turn a brief blip into an outage. The standard fix is exponential backoff with jitter.
The idea is simple: wait a little after the first failure, wait longer after the second, longer still after the third, up to some maximum, and add a small random amount to each wait so that many retrying clients do not all wake up and retry at exactly the same instant. A common formula is to double the wait time on each attempt, starting from a base delay, and cap it at some maximum.
Worked example with small numbers. Suppose the base delay is 1 second, you double it each time, and you cap the wait at 30 seconds, with up to 5 attempts total:
- Attempt 1 fails. Wait roughly 1 second (plus a small random jitter, say between 0 and 0.5 seconds), then try attempt 2.
- Attempt 2 fails. Wait roughly 2 seconds, then try attempt 3.
- Attempt 3 fails. Wait roughly 4 seconds, then try attempt 4.
- Attempt 4 fails. Wait roughly 8 seconds, then try attempt 5.
- Attempt 5 fails. Stop retrying and treat the step as a hard failure.
Total elapsed waiting time in the worst case here is about 1 plus 2 plus 4 plus 8, which is 15 seconds, before giving up after the fifth attempt. That is a deliberate design choice: it gives a brief outage time to clear, without leaving a user staring at a spinner for several minutes. The exact numbers, base delay, multiplier, cap, and number of attempts, depend on how expensive a single call is and how quickly the underlying service tends to recover; 1 to 5 attempts with a base delay between 0.5 and 2 seconds is a reasonable starting point for most APIs, and you can adjust after watching how your own retries actually behave in practice.
import random
import time
class PermanentError(Exception):
"""Raised for errors that will not be fixed by retrying."""
pass
class TransientError(Exception):
"""Raised for errors that might succeed on a later attempt."""
pass
def call_with_backoff(func, *args, max_attempts=5, base_delay=1.0, max_delay=30.0, **kwargs):
"""Call func(*args, **kwargs), retrying on TransientError with
exponential backoff and jitter. Raises on PermanentError immediately,
and re-raises the last TransientError if all attempts are used up."""
attempt = 0
while True:
attempt += 1
try:
return func(*args, **kwargs)
except PermanentError:
raise # no point waiting, the request itself is invalid
except TransientError as exc:
if attempt >= max_attempts:
raise
delay = min(base_delay * (2 ** (attempt - 1)), max_delay)
delay += random.uniform(0, delay * 0.25) # jitter
time.sleep(delay)
# Example tool wrapper that classifies a raw HTTP-style status code
def search_tool(query):
status, body = fake_http_call(query) # stand-in for a real HTTP call
if status in (429, 503, 500):
raise TransientError(f"status {status}: try again later")
if status in (400, 401, 403, 404):
raise PermanentError(f"status {status}: request will not succeed")
return body
# result = call_with_backoff(search_tool, "agent reliability patterns")
This example is deliberately plain Python with no third-party library, so you can see every moving part: a clear split between permanent and transient errors as two distinct exception types, a capped exponential delay, and a small amount of jitter. In a real project you might use a retry library that provides this pattern ready-made, but the underlying idea is exactly what is shown here, and it is worth understanding it at this level before reaching for a library, so you can tell whether the library's defaults actually fit your situation.
One more detail matters: retries cost something even when they succeed. Every retried tool call adds latency, and if the tool call is itself preceded or followed by a model call, it adds tokens and money too. A retry budget is not free, so the max_attempts and max_delay numbers above are also, indirectly, a cost control, which brings us to the second half of this article.
Stopping conditions: when enough is enough
Retries answer 'what do I do when one step fails'. Stopping conditions answer a different question: 'when does the whole loop end, whether or not every step succeeded'. Every agent loop, no matter how it is built, needs at least one way to stop that does not depend on the model deciding, on its own, that it is done, because the model cannot always be trusted to notice that. A production agent typically combines several stopping conditions at once, each catching a different kind of runaway behaviour.
Step and iteration limits
The simplest and most important guard is a hard cap on the number of think-act cycles the loop is allowed to run, regardless of what is happening. If a task that normally takes 3 to 6 tool calls has not finished after, say, 20 iterations, something is wrong, and letting it run to 200 iterations will not fix it; it will just consume more time and money before you find out. A concrete number depends on the task. A simple lookup-and-answer agent might cap at 5 to 8 steps. A multi-tool research task, of the kind covered in Part 5 on planning, might reasonably need 15 to 25 steps. The right number comes from watching how your own agent behaves on real tasks, not from a rule of thumb borrowed from a different project; set it generously enough that normal tasks always finish comfortably under the cap, and tightly enough that a stuck agent is caught quickly.
Budget limits: tokens, cost and wall-clock time
A step limit catches loops that take too many turns, but a single turn can also be expensive on its own, especially once the context has grown large (see Part 7 on context engineering). A long conversation history, a large tool result pasted into context, or a verbose system prompt can make each individual call to the model cost far more tokens than the one before it. It is worth tracking, separately from the step count, a running total of tokens used or an estimated cost in whatever unit you bill in, and stopping the loop if that total crosses a threshold you set in advance, for example 'stop if this task has used more than 50,000 tokens' or 'stop if the estimated cost has exceeded a set ceiling for this task'. This protects you from the specific case where the agent is technically making progress, one tool call at a time, but each call's context has ballooned so much that the task is no longer economical even though it has not technically looped.
Wall-clock time is a third, independent budget, and it matters for a different reason: a user or a downstream system waiting on the agent's answer usually has its own patience limit, separate from cost. An agent that is well within its step and token budgets can still be unacceptable if it takes four minutes to answer a question that needed an answer in ten seconds. A wall-clock timeout, measured from when the task started, is the right guard for this, and it should be enforced regardless of what the step and token counters say, because a single slow tool call, like a search that hangs instead of timing out cleanly, can blow a time budget without using up many steps or tokens at all.
Progress checks: detecting loops
Step and budget limits are a safety net of last resort; they stop the agent eventually, but only after it has already wasted a lot of effort. A tighter check tries to catch the specific pattern of a stuck loop earlier: the agent repeating the same or a very similar action without the result changing. A simple version of this check keeps a short history of recent actions, for example the last 3 tool calls together with their arguments, and compares each new action against that history. If the agent calls the same tool with the same arguments twice in a row, that is a strong signal that it is stuck, either because it forgot it already has the result, or because the result it got was not what it expected and it is trying the exact same thing hoping for a different outcome.
What to do when you detect this depends on how confident you are in the signal. A cautious response is to not stop immediately but to inject a pointed observation into the context, something like 'note: you already called this tool with these exact arguments two steps ago and got the following result,' which gives the model a chance to notice and change course on its own. If the same action repeats a third time after that nudge, that is a reasonable point to stop the loop entirely and treat it as a failure to escalate, because a model that repeats itself after being told it is repeating itself is not going to self-correct on the fourth attempt.
Explicit completion signals
Not every stopping condition exists to catch a failure. The normal, happy-path way for an agent loop to end is for the model itself to signal that it is done, usually by producing a final answer instead of another tool call, as described in Part 2. It is worth being precise about what counts as 'done' in your own system, because a vague definition is a common source of bugs. Some designs use a dedicated tool, often named something like finish or submit_answer, that the model must call explicitly with its final result, which makes completion a structured, parseable event rather than something inferred from the shape of the model's text. Others treat any response that does not contain a tool call as the final answer. The structured version is usually more reliable, because it removes the ambiguity of trying to guess, from free text, whether the model meant to finish or was just thinking out loud.
Whichever design you choose, it is worth validating the completion itself before accepting it, especially for tasks with a checkable answer. If the task was 'find the current population of a named city and cite the source', a reasonable check before stopping is: does the final answer actually contain a number and a source. If not, that is a case where you may want to reject the completion and send it back for one more turn, with a note explaining what was missing, rather than accepting an incomplete answer simply because the model stopped calling tools.
Escalation as a stopping condition
Not every stop is a success and not every stop is a hard failure either; there is a useful middle category, which is handing control to a person. Part 9 covers human-in-the-loop design in depth, but it is worth naming here as one of the stopping conditions you should design for from the start, rather than bolting on later. Any of the limits above, running out of steps, running out of budget, detecting a stuck loop, failing a repair attempt repeatedly, can and often should end not with an error message but with a clear handoff: here is what the agent tried, here is where it got stuck, here is what it needs from a person to continue. Treating escalation as a first-class outcome, with its own clear trigger conditions, tends to produce much calmer systems than treating every non-success as an exception to be caught somewhere upstream.
Putting it together: a resilient loop
The snippet below sketches how these pieces fit around the basic agent loop from Part 2. It is simplified, with the actual model call and tool dispatch left as stand-in functions, but the control flow, the part this article is about, is complete and worth reading closely.
import time
import hashlib
MAX_STEPS = 20
MAX_WALL_SECONDS = 120
MAX_TOKENS = 50_000
REPEAT_WINDOW = 3 # how many recent actions to compare against
def action_signature(tool_name, arguments):
"""A stable fingerprint of an action, used to detect repeats."""
raw = tool_name + str(sorted(arguments.items()))
return hashlib.sha256(raw.encode()).hexdigest()
def run_agent(task, call_model, call_tool):
start_time = time.monotonic()
tokens_used = 0
history = [] # full conversation / observation history
recent_actions = [] # last few (tool_name, arguments) signatures
for step in range(1, MAX_STEPS + 1):
if time.monotonic() - start_time > MAX_WALL_SECONDS:
return escalate(task, history, reason="wall-clock timeout")
if tokens_used > MAX_TOKENS:
return escalate(task, history, reason="token budget exceeded")
decision, tokens = call_model(task, history) # the "think" step
tokens_used += tokens
if decision.kind == "final_answer":
if not looks_complete(decision.text, task):
history.append({"role": "system",
"content": "Answer seems incomplete, please check and finish properly."})
continue
return decision.text
# otherwise it is a tool call: decision.tool, decision.arguments
sig = action_signature(decision.tool, decision.arguments)
if recent_actions.count(sig) >= 2:
history.append({"role": "system",
"content": f"You already tried {decision.tool} with these "
f"exact arguments {REPEAT_WINDOW} steps ago with no new result. "
f"Try a different approach or report what is blocking you."})
recent_actions.append(sig)
recent_actions = recent_actions[-REPEAT_WINDOW:]
continue
try:
result = call_with_backoff(call_tool, decision.tool, decision.arguments)
history.append({"role": "tool", "name": decision.tool, "content": result})
except PermanentError as exc:
history.append({"role": "tool", "name": decision.tool,
"content": f"error (will not succeed by retrying): {exc}"})
except TransientError as exc:
return escalate(task, history, reason=f"tool still failing after retries: {exc}")
recent_actions.append(sig)
recent_actions = recent_actions[-REPEAT_WINDOW:]
return escalate(task, history, reason="max steps reached")
def looks_complete(text, task):
# a placeholder for a real check: does the answer contain
# what the task actually asked for (a number, a citation, etc.)
return len(text.strip()) > 0
def escalate(task, history, reason):
return {"status": "needs_human", "reason": reason, "history": history}
Notice what each piece is doing and why it is separate from the others. The wall-clock and token checks run at the top of every iteration, independent of whether the previous step succeeded, because they are budgets, not error responses. The repeat-action check runs before the tool is even called, because its job is to catch a pattern across steps, not a single bad result. The retry-with-backoff call wraps only the tool call itself, because that is the one operation where a transient failure is expected and worth trying again. And completion is checked before being accepted, rather than trusted the moment the model stops calling tools. None of these checks replace each other; a real agent needs all of them, because each one catches a different failure mode, and in practice you will see all of these failure modes eventually, usually in combinations you did not plan for.
Common mistakes
- Retrying everything the same way. Treating a 404 the same as a 429 means wasting attempts on errors that will never succeed, and treating a payment call the same as a search call risks duplicate side effects. Classify errors as transient or permanent, and as safe-to-retry or not, before writing the retry logic.
- Retrying blindly instead of feeding the error back. If a tool call failed because an argument was wrong, trying the identical call again produces the identical failure. Put the actual error message into the context so the model, or your code, has something new to act on.
- No cap on repair attempts. If the model cannot produce valid arguments after seeing the exact validation error twice, a third or fourth attempt is unlikely to succeed either. Stop and escalate rather than looping on the same mistake.
- Only one stopping condition. A step limit alone will not catch a single tool call that hangs for ten minutes; a timeout alone will not catch a short loop of twenty rapid, cheap, pointless tool calls. Use step, token, and wall-clock limits together, because they catch different failure shapes.
- Treating 'the model stopped calling tools' as proof the task is done. A model can stop for the wrong reasons, including running out of patience or misunderstanding that the task is finished. Check that a final answer actually satisfies the task before accepting it.
- Letting side-effecting tools retry automatically on ambiguous failures. A timeout on a 'send' or 'charge' action tells you the response was lost, not that the action failed. Retrying automatically can duplicate the action. Use idempotency keys where the tool supports them, and require confirmation otherwise.
- No limit on how large the conversation history is allowed to grow before it affects the budget checks. A loop that is well within its step count can still be expensive if each step's context has grown huge; track tokens or cost directly rather than assuming step count is a good proxy for cost.
- Silent failure instead of escalation. Catching every error and swallowing it so the loop 'never crashes' hides problems instead of solving them. A clear, structured escalation with a reason and the relevant history is far more useful than a loop that quietly gives up or quietly keeps trying forever.
Summary and what comes next
An agent loop fails in more ways than a normal program, because it has tools that can be temporarily or permanently unavailable, a model that can produce malformed or wrong output even when every tool is working fine, and a control flow that can get stuck without any single step ever raising an error. Handling this well means separating two concerns. Error handling and retries decide what to do when one step fails: classify the error as transient or permanent, check whether the action is safe to repeat, back off with increasing delays rather than hammering a struggling service, and when the problem is the model's output rather than the tool, feed the actual error back into context instead of retrying blindly. Stopping conditions decide when the whole loop ends, independent of whether individual steps succeeded: a cap on steps, a cap on tokens or cost, a wall-clock timeout, a check for repeated actions that signals a stuck loop, and a clear, validated definition of what counts as a finished answer. Together these turn an agent from something that either works perfectly or runs away unpredictably into something that fails in small, visible, recoverable ways, which is the entire point of building this kind of safety net.
The natural next question is what to do with the cases that these mechanisms deliberately do not resolve on their own, the ones marked for escalation rather than quietly retried or silently stopped. Part 9 picks up exactly there, looking at how to bring a person into the loop well: when to ask before acting rather than after, how to present a stuck agent's situation so a human can actually help quickly, and how to avoid designing a human-in-the-loop step that is really just a human doing all the work.
Comments (0)
No comments yet. Be the first to share your thoughts.