Planning and Task Decomposition

From One Big Ask to a Series of Smaller Ones

In part 2 we described the agent loop as observe, think, act, repeat. The think step is where the model decides what to do next, and so far we have treated that as one decision at a time: look at what has happened so far, pick the next action. Many tasks go much better if, before diving into actions, the agent first works out a rough plan, a sequence of steps toward the goal, and keeps that plan in view as it works. This part is about how that planning happens inside an agent, when it genuinely helps, and the ways it commonly goes wrong.

Here is a motivating example. Suppose a user asks an agent: find out which of our three regional offices had the biggest increase in customer complaints last quarter, and suggest two possible causes. Give this to an agent with a single query_database tool and let it think-and-act one step at a time, with no plan to speak of, and you will often see something like this: it queries one office, writes a plausible-sounding paragraph about causes, and stops, having quietly forgotten the other two offices it was supposed to check.

This is not because the model is careless in some mysterious way. It is a structural problem. Without an explicit plan, each step only has to look locally sensible: given what I have seen so far, what is a reasonable next move. Local sensibility can drift a long way from global correctness over several steps, because nothing is forcing the model to check its progress against the original, full scope of the task. A plan gives the loop something concrete to check itself against: have all three offices actually been queried, or not.

What "Planning" Means Here

Classical AI has a long history of planning as a formal discipline: representing the world as states, actions as operators that transform one state into another, and searching for a sequence of actions that reaches a goal state, with languages like PDDL used to describe all this precisely. That field still exists and is used in robotics and logistics. The planning we are talking about in agent systems built around large language models is much less formal. The agent does not search a mathematically defined state space; it writes, in natural language or in a loosely structured format like a numbered list or JSON, a proposed sequence of steps, then tries to follow it. This informal version is usually called task decomposition: breaking one larger, vaguer goal into smaller, more concrete subtasks.

A few terms are worth pinning down before going further, because they will come up repeatedly. A plan is the ordered or partially ordered collection of steps the agent intends to carry out. A step, or subtask, is one unit of that plan, usually small enough to map onto a single tool call or a single focused piece of reasoning. A dependency is a relationship between steps where one cannot sensibly start until another has finished, because it needs that earlier step's output. Re-planning is revising the plan, in part or in full, once new information makes the old plan wrong or incomplete. These four ideas are really all there is to planning in this context; the rest of this article is about how to use them well.

A Worked Example: Weather for Photography

Take a concrete, fairly small task: compare the weather in Lisbon, Porto and Faro this weekend and recommend which city is best for outdoor photography. Suppose the agent has one tool available, get_weather(city, date_range), which returns a short forecast. Without a plan, a single think-act step might call get_weather for Lisbon, see a reasonable-looking forecast, and jump straight to an answer, never checking Porto or Faro, simply because nothing told the loop that three calls, not one, were required before an answer could be correct.

Now suppose instead that, before taking any action, the agent first writes a short plan: get the forecast for Lisbon for Saturday and Sunday, get the forecast for Porto for Saturday and Sunday, get the forecast for Faro for Saturday and Sunday, compare cloud cover, rain chance and light quality across the three, then recommend the best city with a one line reason. Writing this list costs very little, typically a short burst of text, compared with the benefit of making sure all three cities actually get checked before any recommendation is made.

A plan like this can be written as plain text, but it becomes much easier for a program to track if it is structured. Here is one simple way to represent it, as a list of steps with an id, a description, the tool involved, and a status field the surrounding code can update as work proceeds.

[
  {"id": 1, "description": "Get weekend forecast for Lisbon", "tool": "get_weather", "args": {"city": "Lisbon", "date_range": "Sat-Sun"}, "status": "pending"},
  {"id": 2, "description": "Get weekend forecast for Porto", "tool": "get_weather", "args": {"city": "Porto", "date_range": "Sat-Sun"}, "status": "pending"},
  {"id": 3, "description": "Get weekend forecast for Faro", "tool": "get_weather", "args": {"city": "Faro", "date_range": "Sat-Sun"}, "status": "pending"},
  {"id": 4, "description": "Compare the three forecasts and pick the best for photography", "tool": null, "args": {}, "status": "pending", "depends_on": [1, 2, 3]}
]

With a plan like this sitting alongside the agent loop, each turn becomes simpler to reason about: look at the plan, find the next step whose dependencies are satisfied and whose status is still pending, carry it out using the tool-calling mechanics covered in part 3, mark it done or failed, and move on. Once steps 1 through 3 are done, step 4 becomes runnable, and only then does the agent attempt the comparison. The plan is not magic, it is simply an explicit checklist that keeps the loop honest about what the full task actually required.

Why Bother Decomposing at All

There are three concrete reasons to decompose a task rather than handing it to the model whole. The first is coverage: an explicit list of steps makes it much harder to quietly skip part of the task, as happened in the single-shot weather example above. The second is focus: a model asked to do one well-defined thing, such as fetch and summarise Lisbon's forecast, tends to produce a more careful, more accurate result than a model asked to juggle three cities and a comparison and a recommendation all inside one response. The third is visibility: a decomposed task produces intermediate results you, or a monitoring system, can inspect along the way, which matters enormously for debugging and for trusting what the agent is doing, a theme that will come back when we discuss human oversight in part 9.

It helps to make the coverage argument concrete with a small, deliberately illustrative calculation, not a measured statistic, just a way to build intuition. Suppose, purely hypothetically, that each individual reasoning step an agent takes has a 90 percent chance of being locally correct. If a task genuinely requires five such steps chained together with no checking in between, the chance that all five land correctly is roughly 0.9 to the power of 5, which works out to about 0.59, or just under 60 percent. That is a rough illustration of how small per-step error rates compound across an unchecked chain, and it is exactly the kind of compounding that an explicit plan, with status tracking and the option to verify or retry a step, is meant to interrupt before it reaches the final answer.

Two Styles of Planning

Plan first, then execute

In this style, the agent produces the entire plan in one go, before taking any action, then a loop works through the steps in order. The plan might be executed by calling the model again at each step just to fill in details such as tool arguments, or in simpler cases the steps might be deterministic enough that no further model calls are needed until the final synthesis step. The advantage is predictability: you have a complete plan you can inspect, log, or even show to a user for approval before anything runs, which matters for any task with real consequences, a topic we return to in part 9 on human in the loop. The disadvantage is that a plan written before any information has been gathered can be wrong about things that were not yet knowable, such as whether a particular city's forecast data is even available.

Plan as you go

This style is closer to the think-act-observe loop from part 2: at each turn, the agent reconsiders what remains to be done in light of everything observed so far, effectively replanning continuously rather than following a document written in advance. The advantage is adaptability: if the Porto forecast tool returns an error, the agent does not need to discard and rewrite an entire upfront plan, it simply decides what to do next, perhaps retrying with different arguments or trying an alternative source. The disadvantage is that without any explicit checklist, it is easier for the loop to lose track of the overall goal on a longer task, which is precisely the problem that an upfront plan is good at solving.

In practice, most working agents use a hybrid: a short plan is sketched out at the start, and execution and small-scale replanning happen step by step within that outline, with the whole plan only being rewritten when something significant changes. That hybrid is the simplest to reason about and is what the rest of this article assumes.

Decomposition Granularity: How Fine Should Steps Be

A plan that is too coarse does not really decompose anything. A single step that reads research the market and write a report is not meaningfully different from having no plan at all, because it gives the loop nothing concrete to check itself against. A plan that is too fine goes the other way, turning every tiny action into its own step and its own model call: open the search tool, type the query, read the first result, read the second result, and so on. This multiplies latency and cost without adding reliability, and it actually creates more places for the chain of reasoning to drift, since each extra step is another opportunity for a small misunderstanding to creep in.

A rough numeric illustration helps make the cost side concrete. Suppose, as a round illustrative figure rather than a measured benchmark, that each model call in a given setup takes somewhere around one to two seconds and a modest number of tokens. A plan with four genuinely distinct steps, each needing roughly one call to decide the tool arguments and one to interpret the result, comes to about eight calls, perhaps fifteen to twenty seconds end to end. Stretch that same task into twelve finely sliced steps and the call count might roughly triple to around twenty four, tripling the wait time for a task that has not become any more accurate, just slower and more expensive to run. Exact numbers depend heavily on the model, the provider and the task, so treat this as a way of thinking about the trade-off rather than a quoted figure.

A practical rule of thumb: decompose along natural boundaries, meaning distinct tool calls, genuinely independent sub-questions, or points where new information becomes available that the rest of the plan depends on. If two candidate steps would use the same tool with the same information and produce no new, independently checkable result, they are probably one step wearing two names.

Sequential Versus Parallel Decomposition

Not every pair of steps in a plan depends on the one before it. In the weather example, checking Lisbon, Porto and Faro are three independent subtasks; none of them needs the result of another, and only the final comparison step needs all three to be finished. This is a small dependency graph: three independent steps feeding into one final step. Recording this explicitly, rather than just writing a numbered list that implies strict order, matters because it tells you, or the program running the loop, which steps could in principle be run concurrently to save time, and which absolutely cannot be started early because they are missing information they need.

Contrast this with a task where order genuinely matters: look up this week's top technology headline, then search for background on that specific story, then write a short summary. Step two cannot begin before step one has produced an actual headline to search for, and step three needs the results of step two. Treating this as three parallel steps would be a mistake, since two of them would have nothing to work from. Getting this distinction right, independent branches versus a genuine sequence, is most of what good decomposition is about; the actual list of steps is usually the easy part.

Hierarchical Decomposition: Plans Within Plans

Sometimes a single subtask inside a plan is itself complicated enough to deserve its own mini-plan. Take the earlier complaints example: a top level plan might read gather complaint data for three offices, analyse the trend, write the recommendation. The first of those three steps, gather complaint data for three offices, is exactly the same shape of problem as the weather example, and can be expanded into its own three sub-steps, one query per office, before the top-level plan moves on to analysis. This is hierarchical decomposition: a plan whose steps are themselves plans, at least for the more complex branches.

Some agent designs push this further and assign different branches of a hierarchical plan to separate agent loops, sometimes described as a manager coordinating several workers, each with its own tools and its own smaller task. That is an architectural choice we will look at more closely once frameworks come up in part 10; the underlying idea, nested decomposition, is identical whether it happens inside one loop or is split across several. What changes is coordination overhead: someone has to collect the results from each branch and merge them back into the parent plan, and that hand-off is a place where misunderstandings can slip in, for instance a worker reporting a number in different units than the manager expected. Hierarchical decomposition is worth the extra complexity when a subtask genuinely needs a different toolset or a very different amount of context to work with, not as a default structure applied to every task regardless of size.

When Plans Need to Change: Re-planning

A plan written before any action has been taken is, by definition, written with incomplete information. Things happen during execution that the plan could not have anticipated: a tool call fails, a search API times out, a result comes back in an unexpected format, or the user adds new information partway through that changes what is actually being asked for. An agent that cannot revise its plan in response to these events will either plough ahead with a plan that no longer makes sense, or get stuck repeating a step that keeps failing in the same way. Handling failures robustly is the subject of part 8, but it is worth flagging here that plans are exactly where that handling has to hook in.

Here is a small concrete case. The plan says step 2 is get weather for Porto, but the tool call comes back with an error such as city not found, perhaps because the plan wrote the name slightly wrong or the tool expects a country code. A well-designed agent notices the failure during its next think step, considers what went wrong, and decides on a sensible next move, such as retrying with a corrected spelling, trying an alternative tool, or marking the step as unresolved and saying so rather than silently producing a recommendation based on only two of the three cities. That check, noticing that the final answer would otherwise cover fewer cities than the plan called for, is only possible because there was an explicit plan to compare the actual outcome against in the first place.

A few situations reliably call for re-planning rather than pressing on with the original list: a step has failed even after a reasonable number of retries, a step's result contradicts an assumption that a later step was relying on, new information from the user has changed the goal itself, or it becomes clear partway through that the plan was simply missing a step nobody thought of at the start. Good agent design reacts to these triggers by revising the plan, ideally just the affected part of it, rather than either ignoring the problem or throwing the whole plan away and starting again from nothing every time something minor goes wrong.

Representing a Plan So the Loop Can Use It

The JSON structure shown earlier is a reasonable starting point, but a plan the agent loop can actually act on needs a little more: a way to express dependencies between steps, and a simple rule for picking the next runnable step. Here is a short, deliberately minimal sketch in Python showing the idea. It is not a complete framework, and it skips over real tool calls and error handling, both of which are built properly in the hands-on project in part 11; the point here is just to make the data structure and the selection logic concrete.

plan = [
    {"id": 1, "description": "Get weekend forecast for Lisbon", "depends_on": [], "status": "pending"},
    {"id": 2, "description": "Get weekend forecast for Porto", "depends_on": [], "status": "pending"},
    {"id": 3, "description": "Get weekend forecast for Faro", "depends_on": [], "status": "pending"},
    {"id": 4, "description": "Compare forecasts and recommend a city", "depends_on": [1, 2, 3], "status": "pending"},
]

def next_step(plan):
    done_ids = {s["id"] for s in plan if s["status"] == "done"}
    for step in plan:
        if step["status"] != "pending":
            continue
        if all(dep in done_ids for dep in step["depends_on"]):
            return step
    return None

# Minimal driver loop
while True:
    step = next_step(plan)
    if step is None:
        if all(s["status"] == "done" for s in plan):
            print("Plan complete.")
        else:
            print("Stuck: no runnable step but plan is not finished.")
        break
    print(f"Running step {step['id']}: {step['description']}")
    # In a real agent, this is where a tool call or model call happens.
    step["status"] = "done"

The next_step function picks the first step that is still pending and whose listed dependencies have all reached done status. Steps 1 through 3 have no dependencies, so the first call returns step 1; once it is marked done, the next call returns step 2, then step 3, and only once all three are done does step 4 become eligible, because that is the step whose depends_on list names all of them. The driver loop keeps calling next_step until either every step is done, in which case the plan is complete, or no step is runnable even though some remain pending, which signals that something is wrong, perhaps a circular dependency, or a step stuck in a failed state that nothing can proceed past. That stuck case is exactly where the stopping conditions discussed in part 8 need to take over, rather than letting the loop spin forever.

Letting the Model Write Its Own Plan

In an agent, the plan itself is usually produced by the model, not hand-written by the developer for every possible task. This means prompting matters: it helps to ask explicitly for a short, structured plan rather than free-flowing prose, because structured output, such as a numbered list or JSON matching the shape used above, is far easier for the surrounding program to parse, track and update. A reasonable instruction might look like this.

Before taking any action, write a short plan of 3 to 6 steps
to achieve the goal below. Each step that needs a tool should
name the tool exactly as given in the tool list. Respond with
only the plan, one step per line, numbered.

Goal: Compare the weather in Lisbon, Porto and Faro this
weekend and recommend the best city for outdoor photography.

Available tools: get_weather(city, date_range)

Asking the plan to name tools exactly as they are defined ties this directly back to the tool and function-calling mechanics from part 3: a plan is only useful if every step that needs a tool can actually be carried out with a tool the agent really has. Models sometimes produce a plausible-looking plan that references a tool, a data source, or a file that does not exist, simply because the name sounds right for the task, or they quietly skip a step that seems obvious to them but was not actually obvious at all, such as checking units or confirming a date range. A plan is still just text the model guessed at; it is worth treating it the same way you would treat any other model output, something to check rather than something to trust blindly, particularly before letting the agent carry out a multi-step plan unattended on anything with real consequences, which is exactly why reviewing a plan before execution is a common pattern we will return to in part 9.

Planning and the Workflow-Versus-Agent Line

Part 4 drew a line between workflows, fixed sequences of steps decided in advance by a developer, and agents, where the model itself decides what to do next based on what it observes. Planning sits right on that line, and it is worth being precise about it. A plan that a developer writes once, hard-codes, and that never changes at run time is exactly a workflow, a predetermined sequence of steps, even if it happens to be dressed up as a numbered list inside a prompt. What makes planning agentic is not the existence of a list of steps, but who writes it and whether it can be revised: if the model proposes the plan itself, and can revise it in light of what it observes while carrying it out, that is agent behaviour, even on a run where the resulting plan happens to look identical to one a human would have written by hand in advance.

This has a practical consequence worth acting on. If you notice that your agent produces the same plan every single time for a given type of request, for instance always doing the same three steps for any weather comparison question regardless of the cities involved, you may not need to spend a model call generating that plan afresh each time. You could hard-code it as a workflow and reserve the planning capability, and the latency and cost that comes with it, for the genuinely variable parts of the task. This is a specific instance of the broader question from part 1: do you actually need an agent here, or would a simpler, fixed sequence do the job just as well, more cheaply and more predictably.

Common Mistakes

• Planning for tasks that do not need it. A single, unambiguous lookup does not benefit from a five-step checklist, it just adds latency and tokens for no real gain, which connects back to the when-you-do-not-need-an-agent discussion in part 1.
• Writing a plan once and never revisiting it, even after a step clearly fails or returns something unexpected, which leads the agent to either stall or produce an answer based on incomplete information.
• Making the plan too granular, turning every small action into its own step and its own model call, which multiplies cost and latency without making the result more reliable.
• Letting the plan reference tools, data sources or files that do not actually exist because the model guessed at a plausible-sounding name instead of being constrained to its real, defined toolset from part 3.
• Treating steps assigned to separate sub-agents as independent and running them in parallel when they actually depend on each other's output, which can produce inconsistent or contradictory results when they are merged back together.
• Allowing unlimited re-planning when a step keeps failing, so a single stubborn problem causes an endless cycle of new plans instead of the agent stopping and surfacing the issue, a case properly covered by the stopping conditions discussed in part 8.
• Confusing the existence of a plan with proof that it was followed. A plan states intent, not a verified outcome; the agent's results still need to be checked against what each step actually claimed to deliver, not just ticked off as done.

Summary and What's Next

Planning, in the sense used throughout this path, is the practice of turning one broader goal into a sequence of smaller, checkable steps, either written out in full before execution begins, worked out continuously as the agent goes, or more commonly some hybrid of the two. Good decomposition finds natural boundaries that match real tool calls or genuinely independent sub-questions, rather than slicing a task into pieces that are either too coarse to catch missed work or so fine that they multiply cost and latency for no benefit. A plan is most useful when it is structured enough for a program to track progress against it, detect when something has gone wrong or gone missing, and revise course when reality disagrees with what was expected, rather than being followed blindly to the end regardless of what the agent actually observes along the way.

A plan is only as useful as the agent's ability to remember what it has already done and learned while carrying it out, especially once a task runs across many steps or several turns of conversation. That is the subject of part 6, memory: short-term, long-term and notes, which looks at how an agent keeps track of a plan's progress, the results it has already gathered, and earlier context, without constantly re-asking for information it has already seen.