How GPT-6.1 Sol performed with three browser tools

We'd run GPT-6.1 Sol in ego(lite) via ego-browser as the default setting. It finished 27 of its 28 saved tasks, against 23 for agent-browser and 22 for playwright-cli, and it was the fastest tool on the 18 tasks all three completed. It was not the cheapest on every measure: playwright-cli spent less per execution, and it passed one flight-search task that ego-browser failed.
The biggest gap between GPT-6.1 Sol browser tools shows up in multi-site data collection, where a task pulls the same fields from several official sources. Ego-browser passed all ten of those tasks, against six for agent-browser and seven for playwright-cli. Ego-browser, agent-browser and playwright-cli matched on the three research and synthesis tasks, and on the four web app automation tasks inside our test apps, so speed and cost decide those two groups.
On our settings, ego-browser leads on completion and speed, and playwright-cli leads on average cost per execution at $0.4182. A higher pass rate gave ego-browser the lowest cost per passed task at $0.4646, and on the 18 shared tasks its average cost sat 5.4% from playwright-cli's.
The other variants in this test stayed fixed across all attempts, so the differences below come from the tool that carried out the browsing. An attempt is one saved run of one task from that tool's first benchmark pass.
| Browser tool | Success rate | Average time per task | Average recorded cost per task |
|---|---|---|---|
| ego-browser | 96.4% (27/28) | 246.7 s | $0.3352 |
| agent-browser | 82.1% (23/28) | 452.5 s | $0.4702 |
| playwright-cli | 78.6% (22/28) | 399.9 s | $0.3542 |
Success rate uses 28 saved attempts per tool. Time and cost use the same 18 tasks all three tools passed. Costs are USD usage-priced execution records. The ego-browser row uses its November 7, 2026 test build, and the other rows use November 5, 2026 runs.
Success rate

The highest success rate belongs to ego-browser, and completion is what we'd pay for first, because one missing field sends you back to the research. It passed 27 of 28 tasks, four more than agent-browser, and it lost no task that agent-browser completed. Ego-browser passed six tasks that playwright-cli failed, and playwright-cli passed one that ego-browser failed. Playwright-cli also passed all eight search and compare tasks, while ego-browser passed seven.
Agent-browser failed in two ways: three answers carried wrong or missing values, and two attempts ended without a final answer. Playwright-cli left a required value missing in five of its six failures. Three of those failures came on X posts, where it reported the timestamps as unavailable; agent-browser passed all three of those X posts.
The other two missing values were the Qobuz track counts in an album task that ego-browser completed, and Phi-4's training tokens in an open-model comparison.
In a flight-search comparison, GPT-6.1 Sol with ego-browser reported a fare above a lower price already seen under the same settings, so the answer failed the task's acceptance conditions while the playwright-cli attempt passed. A complete table can still hold the one value that changes your decision.
Speed
In terms of speed, ego-browser cut the wait nearly in half on the 18 tasks all three tools passed. It averaged 246.7 seconds per task, 45.5% less than agent-browser and 38.3% less than playwright-cli. It was also faster on 15 of the 18 tasks against agent-browser and on 17 against playwright-cli.

GPT-6.1 Sol made fewer tool calls on the same work, where a tool call is one request to a tool: 28.1 per task with ego-browser, 44.1 with agent-browser and 36.8 with playwright-cli. Tool execution took 84.4 seconds with ego-browser, against 210.0 seconds with agent-browser and 215.6 with playwright-cli.
The camera task shows the gap on ordinary work, where all three tools finished the comparison across eight camera brands. GPT-6.1 Sol took 330.2 seconds with ego-browser, 597.2 with agent-browser and 598.5 with playwright-cli, and used 29, 57 and 54 tool calls for the same accepted result.

Cost
Without harming completion rate, ego-browser costs less than agent-browser. On the 18 shared tasks it came in 28.7% cheaper and was cheaper on 15 of them, with the model held fixed and fewer tool calls per task.
Playwright-cli is close enough on shared-task cost that we would not choose between ego-browser and playwright-cli on that number alone. Their averages differ by 5.4%, and ego-browser was cheaper on ten tasks while playwright-cli was cheaper on eight. Once failures count, ego-browser's recorded cost per pass falls below playwright-cli's, at $0.4646 against $0.5322.
Average cost across all attempts and cost per passed task answer different questions, so we keep both. The first shows what one execution uses; the second divides the recorded spend, failures included, by the number of passes.
| Browser tool | Average recorded cost per task across all 28 attempts | Recorded cost per passed attempt |
|---|---|---|
| ego-browser | $0.4480 | $0.4646 |
| agent-browser | $0.5938 | $0.7229 |
| playwright-cli | $0.4182 | $0.5322 |
These figures use the same first-pass runs: ego-browser on November 7, 2026, and the other tools on November 5, 2026. They are usage-priced records, excluding evaluation costs and earlier retries absent from the saved rows, and they are not verified invoices.
Playwright-cli is the cheapest per execution at $0.4182. Ego-browser has the lowest recorded cost per pass at $0.4646, because more of its executions met every requirement. Neither figure makes ego-browser the cheapest on every task.
Multi-site data collection
When GPT-6.1 Sol must collect the same fields from several official sources, ego-browser is the clearest choice. It passed all ten tasks, while agent-browser passed six and playwright-cli passed seven, and the gap is wider than in the overall comparison.
| Browser tool | Success rate for Multi-site data collection |
|---|---|
| ego-browser | 100.0% (10/10) |
| agent-browser | 60.0% (6/10) |
| playwright-cli | 70.0% (7/10) |
This group uses the same first-pass runs as the overview: ego-browser on November 7, 2026, and the other two tools on November 5, 2026.

When every row has to be filled, a three- or four-task gap decides whether you can use the table at all. These ten tasks covered central-bank decisions, product specifications and SaaS compliance comparisons.
The SaaS task shows why a filled cell can still be wrong. The task asked for certification status as of September 30, 2026, and both agent-browser and playwright-cli recorded Smartsheet as certified when the required status was certification in progress. The ego-browser attempt matched every field in the comparison.
Which tool we'd use
For complete research deliverables, especially multi-site data collection, we'd pair GPT-6.1 Sol at high reasoning effort with ego-browser through pi. That combination had the highest completion rate, the shortest wait on the shared tasks, and a cost below agent-browser's that stayed close to playwright-cli's.
We'd keep playwright-cli when execution cost matters most or when the work looks like search and compare. It passed all eight of those tasks, while ego-browser and agent-browser each passed seven, so these eight give no reason to replace a working playwright-cli setup.
Agent-browser passed one more task than playwright-cli, but at a higher recorded cost, so the overall count alone gives no strong reason to prefer it. It passed three X posts where playwright-cli passed none, and that explains an important part of the one-task gap.
FAQ
Which GPT-6.1 Sol browser tool completed the most attempts?
ego-browser completed the most, passing 27 of 28 saved attempts, while agent-browser passed 23 and playwright-cli passed 22. The model, the pi agent harness and the tasks stayed fixed, and each tool contributed one saved execution per task from its first pass. Multi-site data collection accounts for the four-task gap over agent-browser.
Which browser tool was fastest with GPT-6.1 Sol?
ego-browser was fastest, averaging 246.7 seconds per task on the 18 tasks all three tools passed. Agent-browser averaged 452.5 seconds and playwright-cli averaged 399.9 seconds with the same model. Ego-browser was also quicker on 15 tasks against agent-browser and 17 against playwright-cli, so the average reflects a broad lead.
What does GPT-6.1 Sol cost per browser task?
On the 18 shared tasks, GPT-6.1 Sol's average recorded cost was $0.3352 with ego-browser, $0.4702 with agent-browser and $0.3542 with playwright-cli. Across all 28 attempts those averages were $0.4480, $0.5938 and $0.4182. These are usage-priced execution records rather than verified bills, and they exclude evaluation costs.
Did the cheapest average execution also give the lowest cost per pass?
Playwright-cli had the cheapest average execution at $0.4182, but ego-browser had the lowest recorded cost per pass at $0.4646. Agent-browser's cost per pass was $0.7229 and playwright-cli's was $0.5322. We divided each tool's recorded spend across 28 attempts, failures included, by its number of passes.
Did ego-browser win every task against playwright-cli?
ego-browser passed 6 tasks that playwright-cli failed, but playwright-cli passed 1 that ego-browser failed. The counterexample was a flight-search comparison where GPT-6.1 Sol with ego-browser reported a fare above a lower qualifying price. We count that answer as a failure even though the model returned a table.
Which tool would we choose for Multi-site data collection?
We'd choose ego-browser for GPT-6.1 Sol's multi-site data collection, where it passed all 10 attempts. Agent-browser passed six and playwright-cli passed seven on the same tasks. The SaaS comparison shows why completion matters: both alternatives used the wrong certification status for the requested cutoff.
Would we replace playwright-cli for search and compare work?
We wouldn't replace playwright-cli for completion reasons in these 8 search and compare tasks, where it passed all 8. Ego-browser and agent-browser each passed 7 of those tasks. The speed comparison favors ego-browser, so waiting time is the stronger reason to consider a switch in this group.
Does one extra pass make agent-browser a better choice than playwright-cli?
Agent-browser's 23 passes against playwright-cli's 22 do not give a strong overall reason to prefer it. Agent-browser also cost more on the shared tasks, averaging $0.4702 per task against $0.3542 for playwright-cli. Their outcomes split by task, including three X posts passed by agent-browser and none by playwright-cli, so we would match the choice to the work.
Did all three browser tools return the same result on the research and Web app tasks?
All three tools passed the 3 research and synthesis tasks and the 4 web app automation tasks in our test apps. Those results give no reason to rank them within either group. Our preference for ego-browser rests on waiting time and recorded cost, not on a completion advantage in these 7 tasks.
Did using ego-browser's second pass change the overall choice?
ego-browser passed 26 of the same 28 tasks on its second pass, while agent-browser and playwright-cli stayed at 23 and 22 on their first passes. Its average time on the 18 shared tasks was 252.4 seconds, against 452.5 and 399.9 seconds for the other two tools. We keep the first pass as the main result rather than mixing both ego-browser passes into the ranking.
How we tested
We analyzed archived GPT-6.1 Sol runs at high reasoning effort through pi 1.5.0 on macOS, using the recorded openai-codex route. The run configuration specified agent-browser 0.39.1 and playwright-cli 0.1.22 for the November 5, 2026 executions, and ego-browser's local-202611071027-ba90f1c test build ran on November 7, 2026. Each setup used its supplied skill and CLI to observe pages and carry out browser actions.
The 28 tasks are the ones both alternative tools had already run, drawn from the 72-task efficiency-20261019 dataset. Each task belongs to one group: multi-site data collection, search and compare, social media verification, research and synthesis, or web app automation.
Each tool's first benchmark pass contributes one saved execution per task, so each tool has 28 attempts. Ego-browser also ran a second pass, which we report separately. The step budget was 100 model response turns, with a 3,600-second backstop. A tool call is one request to a tool, and a pass requires every mandatory result and the final deliverable to meet the task's acceptance conditions.
Time and cost use the 18 tasks all three tools passed. Time includes thinking, model requests, tool execution and remaining overhead. Costs are recorded USD usage amounts. The tools ran with different supplied browser state, so we also checked the results without the three X posts: 24/25, 20/25 and 22/25 in the same tool order.
Full results
Each cell shows passed attempts, elapsed seconds and recorded USD cost for the saved execution from the first benchmark pass. Elapsed time includes waiting for the model API, and each task has one included attempt per tool, so a task average equals that attempt's value. The rows use ego-browser's November 7, 2026 run and the alternatives' November 5, 2026 runs.
| Task | Type of work | ego-browser | agent-browser | playwright-cli |
|---|---|---|---|---|
| Compare album metadata across nine services | Multi-site data collection | 1/1 · 307.7 s · $0.7724 | 1/1 · 740.8 s · $0.9950 | 0/1 · 524.0 s · $0.6029 |
| Collect specifications from eight camera brands | Multi-site data collection | 1/1 · 330.2 s · $0.4751 | 1/1 · 597.2 s · $0.6865 | 1/1 · 598.5 s · $0.5201 |
| Collect decisions from twelve central banks | Multi-site data collection | 1/1 · 382.8 s · $0.6578 | 0/1 · 760.3 s · $1.0079 | 1/1 · 816.4 s · $0.9166 |
| Reconcile conference keynotes and recordings | Multi-site data collection | 1/1 · 385.6 s · $0.5659 | 1/1 · 672.0 s · $0.9489 | 1/1 · 884.2 s · $0.9596 |
| Compare electric-SUV specifications | Multi-site data collection | 1/1 · 633.2 s · $1.2590 | 0/1 · 748.7 s · $0.8006 | 1/1 · 981.4 s · $0.7713 |
| Compare open-model cards and announcements | Multi-site data collection | 1/1 · 424.5 s · $0.6486 | 1/1 · 828.6 s · $1.0199 | 0/1 · 680.6 s · $0.7187 |
| Compare minimum wages across nine jurisdictions | Multi-site data collection | 1/1 · 451.9 s · $0.6108 | 1/1 · 762.6 s · $1.0761 | 1/1 · 763.2 s · $0.7392 |
| Compare flagship-phone specifications | Multi-site data collection | 1/1 · 448.7 s · $0.6444 | 1/1 · 931.9 s · $0.9423 | 1/1 · 676.3 s · $0.6352 |
| Compare SaaS compliance at a cutoff | Multi-site data collection | 1/1 · 472.3 s · $0.6257 | 0/1 · 895.0 s · $0.7363 | 0/1 · 848.0 s · $0.5770 |
| Compare serverless free-tier allowances | Multi-site data collection | 1/1 · 205.5 s · $0.3225 | 0/1 · 682.0 s · $0.5293 | 1/1 · 411.8 s · $0.3951 |
| Read an interactive ARC leaderboard | Research and synthesis | 1/1 · 224.4 s · $0.2708 | 1/1 · 416.6 s · $0.4658 | 1/1 · 220.2 s · $0.1853 |
| Analyze a model-training dataset | Research and synthesis | 1/1 · 87.1 s · $0.0809 | 1/1 · 137.7 s · $0.1474 | 1/1 · 155.5 s · $0.1321 |
| Trace sources behind a model table | Research and synthesis | 1/1 · 285.6 s · $0.2288 | 1/1 · 245.5 s · $0.3004 | 1/1 · 377.5 s · $0.1890 |
| Compare and configure Mac laptops | Search and compare | 1/1 · 301.3 s · $0.6119 | 1/1 · 383.0 s · $0.4399 | 1/1 · 337.2 s · $0.3669 |
| Reconcile six films’ box-office figures | Search and compare | 1/1 · 184.9 s · $0.2034 | 1/1 · 360.5 s · $0.3394 | 1/1 · 201.6 s · $0.1506 |
| Screen four companies in a public registry | Search and compare | 1/1 · 182.5 s · $0.2883 | 1/1 · 626.9 s · $0.3927 | 1/1 · 230.2 s · $0.1784 |
| Cross-check trials and drug labels | Search and compare | 1/1 · 372.8 s · $0.4787 | 1/1 · 368.7 s · $0.4176 | 1/1 · 383.1 s · $0.3515 |
| Reconcile inflation series and statistics | Search and compare | 1/1 · 132.8 s · $0.1957 | 1/1 · 507.1 s · $0.3723 | 1/1 · 548.9 s · $0.2209 |
| Compare qualifying flights and fares | Search and compare | 0/1 · 341.5 s · $0.4464 | 0/1 · 986.7 s · $1.1087 | 1/1 · 504.3 s · $0.4713 |
| Reconcile browser-compatibility tables | Search and compare | 1/1 · 175.1 s · $0.1916 | 1/1 · 147.8 s · $0.2092 | 1/1 · 294.6 s · $0.1973 |
| Cross-check lecture, code, and paper | Search and compare | 1/1 · 202.3 s · $0.4243 | 1/1 · 274.3 s · $0.3491 | 1/1 · 666.0 s · $0.7001 |
| Verify X security posts against releases | Social media verification | 1/1 · 535.7 s · $0.8979 | 1/1 · 780.7 s · $0.8923 | 0/1 · 405.8 s · $0.3921 |
| Verify X product posts against announcements | Social media verification | 1/1 · 710.3 s · $0.7155 | 1/1 · 851.2 s · $0.9236 | 0/1 · 466.5 s · $0.4007 |
| Reconcile X launch posts and reference lists | Social media verification | 1/1 · 168.9 s · $0.1650 | 1/1 · 263.9 s · $0.1505 | 0/1 · 172.0 s · $0.0880 |
| Book a trip within a test app’s policy | Web app automation | 1/1 · 194.1 s · $0.2570 | 1/1 · 303.2 s · $0.2853 | 1/1 · 208.4 s · $0.2240 |
| Update service monitors in a test app | Web app automation | 1/1 · 137.5 s · $0.1338 | 1/1 · 419.9 s · $0.4141 | 1/1 · 152.9 s · $0.1528 |
| Reconcile receivables in a test app | Web app automation | 1/1 · 227.7 s · $0.2312 | 1/1 · 628.3 s · $0.4329 | 1/1 · 291.9 s · $0.3085 |
| Create replenishment orders in a test app | Web app automation | 1/1 · 116.8 s · $0.1415 | 1/1 · 361.8 s · $0.2437 | 1/1 · 208.4 s · $0.1640 |


