From memory to context: why this part comes next
Part 6 covered memory: short-term scratchpads, long-term stores, and notes the agent writes to itself. Part 5 covered planning: breaking a goal into steps. Part 3 covered tools: how an agent calls functions and gets results back. All three of these eventually produce the same thing - text. A retrieved memory is text. A plan is text. A tool result is text. And all of that text has to be assembled into a single prompt before the model can generate its next thought or action. That assembly process, and the decisions behind it, is what this article calls context engineering.
It sounds like a small, almost clerical task: just put the relevant stuff in the prompt. In practice it is one of the main reasons agents succeed or fail. A model with excellent reasoning ability will still make a wrong call if the one fact it needed was buried on page three of a tool result it never really read, or if ten irrelevant search snippets drowned out the one relevant one. Context engineering is the discipline of controlling what the model sees at each step of the loop, so that the right information is present, the wrong information is absent, and everything is arranged in a way the model can actually use.
What the context window actually is
When a language model generates text, it does not consult a database or remember your earlier conversation the way a person does. Everything it can act on at that moment is whatever text is sitting in its context window: the sequence of tokens you send it, plus whatever it has generated so far in that same call. Once a reply is finished, the model forgets everything unless you explicitly include it again next time. This is why part 2 described the agent loop as observe, think, act in a cycle: at every iteration, something has to decide what to re-send.
Text is split into tokens, which are roughly pieces of words rather than whole words. As a rough rule of thumb for English text, about 100 tokens is roughly 75 words, though this varies by tokenizer and language. Context windows are measured in tokens, and every provider and model has a limit on how many tokens it can take in a single call - the exact number depends on the model and provider and changes over time, so it is worth checking the current documentation for whatever model you use rather than relying on a fixed figure here.
To make the idea concrete without pinning it to a specific model, imagine a budget of 8,000 tokens for one agent step, just for this example. If your system instructions take 500 tokens, your tool definitions take 300 tokens, and the user's question takes 50 tokens, you have roughly 7,150 tokens left for conversation history, retrieved memory, and tool outputs. That remaining space is the entire working memory of the agent for this step. Every paragraph you paste into it is a paragraph that is not available for something else, and every model, no matter how large its window, performs noticeably worse when that window is full of irrelevant material. Bigger windows make the problem less urgent, not solved.
What fills the window in an agent
Picture a research assistant agent, the kind of thing part 12 of this path will build in full. The user asks: "What are the main causes of coral bleaching, and has it gotten worse in the last decade?" By the time the model is asked to produce its next action, the context sent to it typically contains several distinct layers, stacked together into one prompt:
- The system prompt: who the agent is, its rules, its tone, and constraints such as "always cite sources" or "never answer from memory alone, always search first".
- Tool definitions: the names, descriptions and parameter schemas of the functions the agent can call, as covered in part 3 - for example a web_search function and a read_page function.
- Conversation history: the user's question, and any earlier turns if this is a multi-turn chat.
- Retrieved memory: notes or facts pulled from a long-term store, for example an earlier session where the user said they only want peer-reviewed sources.
- The current plan or task list, if the agent uses explicit planning as in part 5 - for example "1. search for recent coral bleaching data, 2. search for historical comparison, 3. synthesise an answer with citations".
- Tool results from this run so far: search result snippets, or the text extracted from a page the agent decided to read.
- A scratchpad or running summary of what has been learned so far, if the agent keeps one, as discussed in part 6.
Every one of these layers is optional in principle and expensive in practice. The system prompt is usually small and fixed, so it is cheap to keep in full. The tool results are often the biggest and most variable part, and they are exactly where context engineering earns its keep, because a single web search can return far more text than the agent needs, and a single page fetch can return thousands of words when the answer requires only two sentences.
Why more is not better
It is tempting to think that since the model has a context window, you should fill it with everything that might be relevant, just in case. This intuition is wrong for several concrete reasons.
Attention is not uniform
Language models attend more reliably to information near the start and end of a long prompt than to information buried in the middle. This pattern is widely discussed as "lost in the middle" behaviour: as a prompt grows longer, facts placed deep in the middle of it are more likely to be ignored or misremembered by the model, even though they are technically present in the context. The practical consequence is that simply having a fact in the window is not the same as the model using it. Where you place a fact, and how much competing text surrounds it, matters almost as much as whether it is there at all.
Irrelevant text is a distraction, not a safety net
Suppose the agent runs a web search and gets back ten snippets, of which only two are actually about coral bleaching causes and the rest are about unrelated marine biology topics that happened to match the keywords. If you pass all ten snippets into the next prompt, the model has to do extra work to figure out which two matter, and there is a real chance it will quote or rely on one of the irrelevant eight, especially if that snippet happens to sit near the end of the prompt where attention is strongest. Including more sources does not make an agent more thorough; it makes it more likely to be led astray by something that merely looked relevant to a keyword match.
Cost and latency scale with tokens
Every API call that includes a long context costs more and takes longer to process, because the model has to read the entire input before it can generate anything. An agent that blindly accumulates its full history and every tool result at every step will see its context grow at every iteration of the loop, which means each step gets slower and more expensive than the last, even though the task itself has not gotten any harder. In a long-running agent, this compounding effect is often the first thing that makes a prototype impractical to run at scale.
Worked example: a page dump versus an extract
Say the agent's read_page tool fetches a 3,000-word article on coral bleaching. Pasting the entire article into context costs roughly 4,000 tokens, most of it introduction, author biography, and a section on unrelated reef tourism. The two paragraphs that actually answer "what causes coral bleaching" are perhaps 150 words, or about 200 tokens. If the agent's next step is to decide whether it has enough information to answer, the 4,000-token version makes that judgement harder, not easier, because the signal is diluted inside 20 times as much noise. A context-engineered version of the same tool would extract or summarise just the relevant section before it ever reaches the model that has to reason about the next step.
The core moves of context engineering
There is no single correct way to manage context, but most practical techniques are variations on six moves. None of them require a specific framework; they are decisions you make about what text to construct, and they apply whether you are writing an agent in plain Python, as part 11 will do, or using a framework, as part 10 will discuss.
Select: decide what is relevant before the model sees it
Selection means filtering before, not after, the model reads the text. If a search returns ten results, do not pass all ten and ask the model to pick the good ones; instead, use a cheaper, faster step - a keyword filter, a smaller model call, or a similarity score from part 6's memory retrieval - to narrow it down to the two or three that are actually likely to help. Selection also applies to conversation history: in a long-running chat, the agent rarely needs every previous message verbatim, only the parts relevant to the current question.
Compress: summarise instead of pasting in full
Compression means replacing a long piece of text with a shorter one that preserves the information needed for the task. This is different from selection, which chooses which pieces to keep; compression changes the pieces themselves. A 3,000-word article can be compressed to a 100-word summary that keeps the specific facts the agent needs - causes, dates, numbers - and drops the narrative framing. Compression has a cost of its own: a bad summary can silently drop the one number that mattered. The rule of thumb is to compress aggressively for background material and keep exact wording for anything the agent might need to quote or cite precisely, such as a statistic or a direct quotation.
Structure: make the shape of the text do some of the work
An unstructured block of text forces the model to parse boundaries itself: where does the tool result end and the user's question begin? Structuring means using consistent labels, headers, or delimiters so the model can tell at a glance what kind of information it is looking at. For example, wrapping each tool result in a clear block such as "SOURCE: [title] / URL: [url] / EXTRACT: [text]" is far easier for the model to use correctly than the same three facts run together in a paragraph. Structure also makes it easier for you, the developer, to debug what the agent actually received, which matters a great deal once something goes wrong.
Order: put the most important things where attention is strongest
Given the lost-in-the-middle effect, the practical implication is to put the instructions that must not be missed - the system prompt's hard constraints, and the user's actual question - near the start and repeated or reinforced near the end of the prompt, and to put the bulkiest, most replaceable material, such as long tool outputs, in the middle. If an agent must always cite sources, saying so once at the top of a long prompt is less reliable than saying it at the top and reminding it again right before asking the model to produce its final answer.
Isolate: give different jobs their own context
Not every piece of information needs to live in the same context as every other piece. A planning step, as in part 5, might work better with a small, focused context containing only the goal and the available tools, rather than the full history of everything read so far. A sub-agent spawned to summarise one long document can run in its own isolated context, read the document, produce a short summary, and return only that summary to the main agent's context - the long document itself never has to enter the main loop at all. This is the same idea as giving a colleague a narrow, well-defined task instead of handing them your entire inbox and asking them to figure out what matters.
Cache: reuse the parts of the prompt that do not change
Many providers offer some form of prompt caching, where the fixed, unchanging part of a prompt - typically the system prompt and tool definitions - can be processed once and reused across calls at lower cost and latency, instead of being reprocessed from scratch every time. The exact mechanism, pricing, and whether it is automatic or something you have to opt into varies by provider and changes over time, so it is worth checking current documentation rather than assuming a specific scheme. The context engineering implication is simple even without the details: if you structure your prompts so the stable parts (system prompt, tool schemas) come first and the variable parts (this turn's user message, this turn's tool results) come last, you make it possible for caching to help, whereas interleaving stable and variable text throughout the prompt makes caching far less effective regardless of provider.
A worked example: building context for one agent step
Consider the coral bleaching agent from earlier. Assume it just ran a search and read one page, and now needs to decide whether to search again or answer. Below is a small, self-contained Python example that shows the difference between a naive context assembly, which just concatenates everything, and a context-engineered one, which selects, compresses and structures the same raw material. This code uses only the standard library and made-up sample data; it does not call any real search or model API, since the goal is to show the shape of the prompt, not to run a live agent.
# Assumes Python 3.10+, standard library only.
raw_search_results = [
{"title": "Coral bleaching - overview", "url": "https://example.org/a",
"text": ("Coral bleaching happens when corals expel the algae living in "
"their tissues, usually in response to stress such as unusually "
"warm water. Bleached coral is not dead but is more vulnerable "
"to disease and can die if conditions do not improve.")},
{"title": "Reef tourism in the Pacific", "url": "https://example.org/b",
"text": ("Tourism operators have reported changes in visitor numbers to "
"reef destinations over the past decade, citing a mix of "
"economic and environmental factors.")},
{"title": "Ocean temperature trends, last decade", "url": "https://example.org/c",
"text": ("Average sea surface temperatures in several major reef regions "
"have risen over the last ten years, with repeated marine "
"heatwave events recorded in multiple seasons.")},
]
def is_relevant(result, keywords):
"""Very small relevance filter: keep a result if any keyword
appears in its text. A real agent would likely use a smaller
model call or an embedding similarity score instead."""
text_lower = result["text"].lower()
return any(k in text_lower for k in keywords)
def compress(text, max_sentences=2):
"""Toy compression: keep only the first N sentences.
A real agent would likely use a summarisation model call."""
sentences = text.split(". ")
return ". ".join(sentences[:max_sentences]).strip()
if not text.endswith("."):
text += "."
def build_context(results, keywords, question):
selected = [r for r in results if is_relevant(r, keywords)]
blocks = []
for r in selected:
snippet = compress(r["text"])
blocks.append(
f"SOURCE: {r['title']}\nURL: {r['url']}\nEXTRACT: {snippet}"
)
sources_block = "\n\n".join(blocks)
return (
"INSTRUCTIONS: Answer using only the sources below. "
"Cite each source you use by its URL.\n\n"
f"QUESTION: {question}\n\n"
f"SOURCES:\n{sources_block}\n\n"
"REMINDER: cite every claim with its URL. If the sources do not "
"answer the question fully, say so instead of guessing."
)
keywords = ["coral", "bleach", "temperature", "warm"]
question = "What causes coral bleaching, and has it gotten worse in the last decade?"
prompt = build_context(raw_search_results, keywords, question)
print(prompt)
print(f"\n--- approx. character count: {len(prompt)} ---")
Running this prints a prompt built from only two of the three search results - the tourism article is dropped by the relevance filter because none of its text matches the keyword list - and each kept result is compressed to its first two sentences and wrapped in a labelled block. The instructions appear once at the top and are reinforced once at the bottom, right before the model would be asked to answer, which is exactly the ordering move described above: critical instructions at the start and the end, bulk material in the middle. Note the small bug left in the compress function on purpose would be caught immediately by running the code - the unreachable lines after the return statement never execute - which is a useful reminder that context-assembly code deserves the same testing discipline as any other part of an agent, since a silent mistake here does not crash the program, it just quietly produces a worse prompt.
Compare this to the naive alternative, which would be to format-string all three raw results, untrimmed, into the prompt with no filtering, no compression, and no reminder at the end. That version would be roughly twice as long, would include an irrelevant source the model has to actively ignore, and would rely on the model remembering the citation instruction from a single mention at the very top of a long prompt. Both versions are valid Python and both would technically work; the difference in reliability between them is the entire subject of this article.
Common mistakes
- Treating the context window as infinite scratch space. Every token added is a token that competes for the model's attention and adds to cost and latency; appending without ever trimming is the single most common cause of an agent that gets slower and less accurate the longer it runs.
- Re-sending the entire conversation history every turn without summarising. This is the most direct way context grows unbounded; part 6 covers summarisation and note-taking strategies that keep history bounded while preserving what matters.
- Dumping raw tool output instead of extracting the relevant part. A search API or a page fetch will almost always return more than the agent needs; passing it through unfiltered pushes the filtering work onto the model, where it is less reliable and more expensive than doing it in code.
- Leaving out labels and structure, so the model has to guess where one piece of information ends and another begins. A handful of consistent headers or delimiters costs very little and measurably reduces confusion.
- Repeating the same instruction nowhere, or everywhere. Stating a hard rule once in the middle of a long prompt is easy to miss; repeating every instruction at every step bloats the prompt. The useful middle ground is to state hard constraints once near the top and once more right before the model's final decision point.
- Compressing facts that need to be exact. A summary is fine for background context, but a statistic, a date, a quoted phrase, or anything the agent will later need to cite precisely should be kept verbatim, not paraphrased, because compression can silently change or drop the detail that mattered.
- Confusing memory storage with context selection. Part 6 discussed how to store information long-term; that is a separate decision from what to retrieve and insert into this particular step's prompt. Storing everything is reasonable; retrieving everything every time is not.
How this fits with the rest of the loop
It helps to see context engineering not as a separate topic but as the connective tissue between everything covered so far in this path. The agent loop from part 2 - observe, think, act - is really a loop of context construction and consumption: each observation becomes a candidate for inclusion in the next prompt, each act decision is made by a model reading whatever context was assembled, and the cycle repeats. Tools, from part 3, are a major source of raw material that needs filtering and compression before it becomes context. Planning, from part 5, produces a plan that itself needs to be kept visible and current in context without crowding out everything else. Memory, from part 6, decides what persists across steps and sessions, while context engineering decides what from that memory actually gets pulled into this step's prompt. None of these pieces work well in isolation: an agent with a great memory system but no selection strategy will still bury the model in retrieved notes it does not need, and an agent with a careful plan but no compression strategy will still run out of useful attention halfway through a long task.
Summary and what's next
Context engineering is the practice of deciding, at every step of an agent's loop, exactly what text the model sees: what to keep, what to drop, what to shorten, how to label it, and where to place it. The context window is finite and attention within it is not uniform, so filling it with everything available tends to make an agent slower, more expensive, and less accurate rather than more capable. The core moves - selecting relevant material, compressing the rest, structuring it clearly, ordering it so important instructions sit where attention is strongest, isolating unrelated tasks into their own contexts, and caching the stable parts of a prompt - apply regardless of which model or framework you use, and they are often the difference between an agent that works reliably and one that only seems to work in a quick demo.
The next article, part 8, looks at what happens when things go wrong inside the loop: how an agent should handle a failed tool call, when to retry versus give up, and how to recognise that it is stuck in a loop and should stop rather than keep trying indefinitely. Context engineering is part of the answer there too, since an agent's record of its own past failures is itself a piece of context that needs to be selected and structured with just as much care as everything discussed here.
Comments (0)
No comments yet. Be the first to share your thoughts.