ego (lite) is just a browser, ego is your personal agent across devices.
Join waitlist
Context engineeringAI agentsToken optimizationBrowser automationAgent tooling

Context engineering for browser agents: what an observation costs

Sep 20, 202612 min read
Context engineering for browser agents: what an observation costs

Context engineering is the practice of deciding what goes into the model's context window on each turn. Anthropic frames it as curating the smallest set of high-signal tokens for the desired outcome.

The published definitions converge on that core. What they disagree about is containment, that is, which concept contains which. Harrison Chase of LangChain and Philipp Schmid both published widely cited definitions in 2025, and IBM published its own. Böckeler, writing on martinfowler.com in April 2026, places a user harness for a coding agent inside context engineering. Databricks and Wikipedia put the containment the other way round.

This article adds the part almost every article on the term leaves out. Most general-purpose articles on this topic use calendar examples or Pokémon, as Sourcegraph wrote, and the cost of a single observation from a graphical environment is missing from all of them. We build ego (lite), a browser agent, so we have a stake in that measurement.

What context engineering actually is

Anthropic frames context engineering as curating the smallest set of high-signal tokens for the desired outcome. Harrison Chase and Philipp Schmid both published widely cited definitions in 2025. IBM published its own. The definitions converge on the core practice: deciding what goes into the model's context window on each turn.

The disagreement is about containment, that is, which concept contains which. Böckeler, writing on martinfowler.com on 2026-04-02, writes that engineering a user harness for a coding agent is a specific form of context engineering. Databricks and Wikipedia put the containment the other way round, treating context engineering as a component of the broader harness or agent design.

Separating the definition from the containment claim matters because the two get merged in summaries. The definition is stable across sources. The containment claim is not, and no source has settled it.

Prompt engineering vs context engineering

Prompt engineering is about the wording of a single instruction or a small set of instructions. Context engineering is about what else is in the window when that instruction runs: retrieved documents, tool schemas, prior turns, observations, and state.

The boundary is not a replacement story. Prompt engineering remains the work of getting the instruction itself right. Context engineering adds the surrounding decision of what to include, what to leave out, and what to compress or isolate.

LangChain's four things you can do with context are write it, select it, compress it, and isolate it. Those four actions describe the surrounding decision. They do not describe how to phrase the instruction, which is still prompt engineering.

The five layers question, answered honestly

There is no canonical five-layer framework for context engineering. Anthropic publishes no enumerated list of layers. LangChain uses four categories. IBM describes six steps plus four concepts and does not use the word layers.

Six mutually inconsistent five-item lists circulate, all on vendor blogs, SEO content, or Substack. Three of the six are the same model relabeled: instructions, tools, memory, retrieval, state. The relabeling changes the names, not the structure.

The convergent cluster across the published definitions is smaller than five. It covers instructions, tools, memory, retrieval, and state, which is the relabeled model, and it maps onto LangChain's four actions of writing, selecting, compressing, and isolating context. The number five is a convention, not a finding.

What every list leaves out

Every list leaves out observation volume. Context rot and lost-in-the-middle describe the same practical problem: quality degrades as the window fills, and material in the middle of a long context is attended to less reliably than material at the edges. The lists describe what to put in the window. They do not describe how much a single observation from a graphical environment costs.

The published numbers are large. Playwright MCP can return a page snapshot over 50,000 tokens on a complex page. Provar reported one Salesforce accessibility tree at 114K tokens. Microsoft's Playwright CLI release reported 114K tokens per test through MCP against 27K through the CLI. An independent eight-step test reported about 89K through MCP against about 24K through the CLI, and noted that by steps 12 to 15 the session was carrying 60K to 90K of stale snapshots and beginning to reference elements that no longer existed. Speakeasy reported that input schemas were 60 to 80 percent of a static toolset's token usage, and Playwright MCP's own tool schemas were measured at about 4,200 tokens across 26 or more tools, while a CLI's --help was about 68 tokens. Anthropic's Code execution with MCP reports 150,000 tokens down to 2,000, a 98.7% saving. Our own measurement of the top 24 results for this term found 19 fetchable pages and 64,225 words, with browser appearing 9 times and 8 of those being navigation or footer noise, and DOM, accessibility tree, screenshot, GUI, and browser automation each appearing zero times. Anthropic's article contains exactly one token figure.

The four things you can do with context

LangChain published a four-part description of what you can do with context: write it, select it, compress it, isolate it. The four are actions, not layers. They describe what an engineer does to the contents of the window on a given turn, and they apply whether the window holds a chat history, a document set, or a browser session.

The categories are useful because they are exhaustive at the level of verbs. Anything you do to context is one of those four, or a combination. The sections below take each in turn and give one browser-side example. The browser examples are our own extension of LangChain's categories, not part of LangChain's published description.

MoveWhat it meansOn the browser side
WritePut material into the context window that was not there before. This covers system instructions, tool descriptions, scratchpads, and any state the agent records for later turns.Writing a compact page summary into a scratchpad after a navigation, so the next turn does not need to re-read the page.
SelectChoose which of the available material enters the window on this turn. Selection is the decision to include or exclude, and it is where most of the token budget is won or lost.Pulling only the form fields and their labels out of a page snapshot instead of passing the whole accessibility tree.
CompressReduce the size of material that has already been selected, while keeping the signal the task needs. Summaries, truncation, and structured extraction all sit here.Replacing a 50,000 token snapshot with a list of interactive elements and their @N references.
IsolateKeep material out of the main window by putting it somewhere else: a subagent, a separate call, a sandbox, or a file the model reads on demand.Running a multi-step form fill inside a single JavaScript call so the intermediate DOM states never enter the conversation.

What one page observation actually costs

We measured the top 24 results for this term. 19 pages were fetchable, totaling 64,225 words. The word "browser" appears 9 times, and 8 of those are navigation or footer noise. "DOM", "accessibility tree", "screenshot", "GUI" and "browser automation" each appear zero times. Anthropic's article contains exactly one token figure. Sourcegraph wrote: "Most general-purpose articles on this topic use calendar examples or Pokémon. Coding agents are a better laboratory."

The browser is the unoccupied laboratory. It is where context engineering has the largest measurable cost and the fewest published measurements. A single page observation can exceed the entire context budget of a small task. Tool schemas are paid before the first action. Stale observations accumulate across a session and are never reclaimed. None of this is covered by the calendar and Pokémon examples that dominate the general literature.

The three sections that follow break the cost into three sources: the observation itself, the tool schemas, and what the session is still carrying. Each source has published numbers. Together they explain why a browser agent's token bill is not a function of how many steps it takes, but of how much it holds at each step.

Source one: the observation itself

Playwright MCP can return a page snapshot over 50,000 tokens on a complex page. Provar reported one Salesforce accessibility tree at 114K tokens. Microsoft's Playwright CLI release reported 114K tokens per test through MCP against 27K through the CLI. The same task and the same page, with the gap driven by what the observation returns.

The observation is the largest single line item in a browser agent's context. It is also the one most often passed through unmodified, because the snapshot is what the tool returns and the tool is what the agent calls. A 50,000 token snapshot on a page with a few hundred interactive elements is not an edge case. It is the default behavior of a general-purpose browser tool.

The Microsoft comparison is the cleanest published evidence that the observation format, not the task, drives the number. 114K through MCP against 27K through the CLI is a measurement of two observation formats on the same workload. The CLI returns less because it was designed to return less. The MCP server returns more because the snapshot is the interface.

Source two: the tool schemas, paid before you start

Speakeasy reported that input schemas were 60 to 80 percent of a static toolset's token usage. Playwright MCP's own tool schemas were measured at about 4,200 tokens across 26 or more tools. A CLI's --help was about 68 tokens. The schema is paid on every turn, whether or not the tool is called.

This is the cost that is easiest to miss because it does not appear in any single observation. It is a fixed overhead attached to the toolset, and it scales with the number of tools exposed. A browser MCP server with 26 tools pays 4,200 tokens before the agent has looked at a page. A CLI with a --help flag pays 68.

The ratio matters more than the absolute number. 4,200 tokens of schema against a 50,000 token snapshot looks small. 4,200 tokens of schema against a 2,000 token compressed observation is more than double the observation. The fixed cost does not shrink when the per-step cost does, so it becomes the dominant term in any agent that has been optimized on the observation side.

Source three: what you are still carrying

An independent eight-step test reported about 89K tokens through MCP against about 24K through the CLI. By steps 12 to 15, the session was carrying 60K to 90K of stale snapshots and beginning to reference elements that no longer existed. The observations were still in the window. The page had moved on.

This is the same practical problem that context rot and lost-in-the-middle describe. Quality degrades as the window fills, and material in the middle of a long context is attended to less reliably than material at the edges. A stale snapshot is worse than a missing one, because it is present, it is plausible, and it describes a page state that no longer exists.

The eight-step test is the clearest published account of the failure mode. The agent did not run out of context. It ran out of accurate context, while still holding plenty of inaccurate context. The references to elements that no longer existed are the symptom. The 60K to 90K of stale snapshots is the cause.

One design that changes two of the three

We build ego (lite), so treat this as an interested account of one design rather than a recommendation. It is worth including because it changes two of the three sources above rather than tuning them, and because the tradeoff it makes is easy to state.

Against source one, a page arrives as a Snapshot, which is an accessibility tree with stable @N references instead of raw HTML. Against source three, the agent can run several actions on the page with a few lines of JavaScript in a single round trip, rather than one tool call per action, which is what keeps a long task from accumulating a stack of superseded observations. Source two, the fixed schema cost, is mostly unchanged by this design.

The saving is on the whole task rather than on any individual step, and it comes from those two mechanisms together. A compact observation alone does not help much if the agent still needs one round trip per click, and batching actions does not help much if every observation is a full page of HTML. Stating it as a per-step number would be wrong.

The app is free. Model and network costs still apply, and there are real gaps: no headless CI mode, no cross-browser testing, and no debugging panel, so if you need a network waterfall or a breakpoint this is not the tool for that part of the work.

How to measure it yourself

Log tokens per observation. Not tokens per session, not tokens per task. Per observation. Without that number you cannot tell whether a change reduced the observation or just reduced the number of observations. The two have different fixes and different costs.

Separate fixed from per-step cost. Tool schemas are fixed. They are paid on every turn regardless of what the agent does. Observations are per-step. A toolset with 4,200 tokens of schema and a 2,000 token observation has a fixed cost larger than its variable cost, and optimizing the observation will not change that. Report the two separately or the optimization will target the wrong term.

Report cost per completed task, not cost per call. Failed runs are paid for and produce nothing. A tool that costs less per call but fails more often can cost more per completed task. Our own bench, 31 tasks, 5 tools, one coding agent, one model at max effort, one judge, 2026-08-19, reported ego lite at 93.5% perfect rate at $1.64 per task, which is $1.75 per completed task, against playwright-cli at 71.0% and $4.82. The bench does not isolate tokens as the cause. Turns, latency and recovery all move together, there is no per-observation token field, and no context-strategy ablation was run. Any per-observation number must come from a separate measurement, never from this bench.

Context engineering vs harness engineering

Böckeler (Thoughtworks, martinfowler.com, 2026-04-02) writes: "Engineering a user harness for a coding agent is a specific form of context engineering." Databricks and Wikipedia put the containment the other way round, treating context engineering as one part of harness design. The definitions disagree on direction, not on substance.

The practical split is one line: context engineering decides what is in the window, harness engineering decides what the agent can do and how it recovers. The two overlap at the tool boundary, which is why the containment argument has no clean answer. A separate article covers harnesses.

Frequently asked questions

What is context engineering?

The practice of deciding what goes into the model's context window on each turn. Anthropic frames it as curating the smallest set of high-signal tokens for the desired outcome. Harrison Chase (LangChain) and Philipp Schmid both published widely cited definitions in 2025, and IBM published its own. The definitions converge. What they disagree about is containment, that is, which concept contains which.

What are the five layers of context engineering?

There is no canonical five layers. Anthropic publishes no enumerated list of layers. LangChain uses four categories. IBM describes six steps plus four concepts and does not use the word "layers". Six mutually inconsistent five-item lists exist, all on vendor blogs, SEO content or Substack. Three of the six are the same model relabeled: instructions, tools, memory, retrieval, state.

Is context engineering replacing prompt engineering?

The verified sources do not make that claim. Prompt engineering is one input to the window. Context engineering covers the whole window, including tool schemas, retrieved material, observations, and prior turns. LangChain's four actions (write, select, compress, isolate) apply to all of those, not only to the prompt.

What is context rot?

Context rot and lost-in-the-middle describe the same practical problem: quality degrades as the window fills, and material in the middle of a long context is attended to less reliably than material at the edges. In a browser agent this shows up as stale snapshots that remain in the window after the page has changed.

How do I measure context cost in a browser agent?

Log tokens per observation, not per session. Separate fixed cost (tool schemas, paid every turn) from per-step cost (observations). Report cost per completed task rather than cost per call, because failed runs are paid for and produce nothing. Our own bench does not isolate tokens as the cause: turns, latency and recovery all move together, there is no per-observation token field, and no context-strategy ablation was run.