DeepSeek V4.1 Flash Benchmarked with three Browser Tools
We found that DeepSeek V4.1 Flash does its best browser work with ego-browser, and that no browser tool fixes its biggest weakness, long jobs across many sites. With ego-browser it passed 51.8% of attempts (29 of 56), against 46.4% with agent-browser (13 of 28). On the 8 tasks both setups passed every time, ego-browser finished a task in 393.1 seconds on average, against 506.0 with agent-browser. Those tasks also cost less with ego-browser, $0.047 against $0.058. Counted over every attempt, failures included, agent-browser comes out cheaper, $0.066 per task against $0.074.
The model is at its best inside a single web app. We asked it to book trips, file purchase orders and open invoice disputes, and it got them right on 11 of 12 attempts across both tools. It also did well checking posts on X against other sources with ego-browser, passing all 6 attempts there, against 1 of 3 with agent-browser. And when it hands in an answer, the answer is usually right: 29 of 39 with ego-browser and 13 of 18 with agent-browser.
Breadth is where it struggles. When we asked it to collect the same fields from 8 to 12 sites, it passed only 3 of 30 attempts across both tools. In 22 of those attempts it used the whole 100-step budget without writing an answer. When it did answer wrong, the cause was usually a rule in the task rather than a misread page, such as reporting 0 people for a company the register marks as exempt.
We traced ego-browser's speed lead to fewer steps. On the shared tasks, DeepSeek V4.1 Flash made 44.1 tool calls per task with ego-browser and 66.5 with agent-browser, and each call took about the same time.
Success rate
DeepSeek V4.1 Flash's success rate varied more by task type than by browser tool. Overall, it passed 51.8% of attempts with ego-browser and 46.4% with agent-browser. Web app automation produced the strongest results, with 8 of 8 attempts passing with ego-browser and 3 of 4 with agent-browser. The gap was wider on social media verification tasks: 6 of 6 attempts passed with ego-browser, compared with 1 of 3 with agent-browser. Search and comparison results were closer, with 11 of 16 attempts passing with ego-browser and 5 of 8 with agent-browser, while research and synthesis yielded 3 of 6 and 2 of 3, respectively. Both setups struggled most with multi-site data collection, passing only 1 of 20 attempts with ego-browser and 2 of 10 with agent-browser.
Eight tasks passed on every attempt in both tools: three web app automation tasks, the Mac laptop comparison, the box office and inflation reconciliations, the SpaceX launch log and the ARC Prize chart reading. Four tasks split by tool in ego-browser's favor. It passed the trip booking, the MDN check and both X verification tasks on every attempt, while the agent-browser attempt at each failed. One task split the other way: agent-browser passed the camera specs task, which ego-browser failed twice.
We saw DeepSeek V4.1 Flash fail in two ways. Most often it uses the whole 100-step budget and stops with no answer: 17 attempts with ego-browser and 10 with agent-browser. Less often it hands in a confident answer with a wrong value: 10 and 5 attempts. The first kind decides the multi-site tasks, where 22 of 30 attempts ran out of steps.
The central bank task shows the step problem clearly. We asked for each of 12 central banks' first 2026 rate decision, with the rate, dissents and next meeting date. With ego-browser, DeepSeek V4.1 Flash worked through the banks one at a time, in the order the task listed them. It reached the twelfth, the Reserve Bank of India, in its last few steps. The budget ran out there, and it had not written a single row of the table. All 3 attempts at this task, in both tools, ended without an answer.
The Companies House task shows the wrong-value problem. We asked for a KYC snapshot of four UK companies, including each one's PSC count, the people the register lists as controlling the company. For Unilever, the register says the information isn't available because the company is exempt, and the task counts "exempt" as the right reading and 0 as wrong. DeepSeek V4.1 Flash read that notice and still wrote 0 in 2 of 3 attempts, once in each tool. One of those answers even quoted the exemption right next to the 0.
The 15 wrong answers follow a pattern. Five broke a status or date rule, such as calling a vendor certified while its page said certification was in progress. Three cited the wrong source: every attempt at the Wikipedia tracing task picked a later Bloomberg footnote over the required India Today one. Four took a figure from the wrong place, such as a model's training length instead of its context window. Two miscounted or missed the lowest fare in a list, and one skipped part of the requested table format.
The two tools hand the page to the model differently. agent-browser gives DeepSeek V4.1 Flash an accessibility-tree snapshot with element refs to act on. With ego-browser, the model writes a short script that drives ego(lite) and prints what it extracts. That didn't change answer quality: 74.4% of answered attempts passed with ego-browser and 72.2% with agent-browser. ego-browser's lead comes from finishing more often, with 30.4% of attempts out of steps against 35.7%.
Speed
ego-browser makes DeepSeek V4.1 Flash noticeably faster. On the 8 tasks both setups passed every time, a task took 393.1 seconds with ego-browser and 506.0 with agent-browser. ego-browser was faster on 6 of the 8. The box office reconciliation and the Mac laptop comparison were the two where agent-browser finished first.
We looked at where the time goes, and the gap sits in the browser tool's share. Time inside the browser tool was 237.0 seconds per task with ego-browser and 335.7 with agent-browser, while model time was 75.5 and 83.5. Each call took a little longer with ego-browser, 5.37 seconds against 5.05. But the model needed a third fewer of them, 44.1 calls per task against 66.5.
The invoice task shows why. With ego-browser, DeepSeek V4.1 Flash wrote scripts that set a filter and read a whole payments table in one call. It finished in 43 calls and 378.1 seconds on average. With agent-browser it took 93 calls and 821.1 seconds.
The order flips when failures count. Averaged over every attempt, a task took 814.4 seconds with ego-browser and 597.8 with agent-browser. The attempts that ran out of steps cause it. Each of those used the same 100 steps, and with ego-browser those steps produced more output and ran longer: 1075.4 seconds and 34,137 output tokens per attempt, against 767.6 seconds and 24,421 tokens.
Cost
On tasks it passes, DeepSeek V4.1 Flash costs less with ego-browser. On the 8 shared tasks a task cost $0.047 with ego-browser and $0.058 with agent-browser, and ego-browser was cheaper on 6 of the 8. Averaged over every attempt, failures included, agent-browser was cheaper: $0.066 per task, against $0.074.
The model and its price are the same in both tools, so the whole gap is token usage. On the shared tasks, DeepSeek V4.1 Flash produced 19,161 output tokens per task with ego-browser and 22,882 with agent-browser, over 44.1 tool calls against 66.5. On attempts that ran out of steps the usage reverses: $0.098 per attempt with ego-browser against $0.079 with agent-browser.
Counting every attempt, each passing result cost almost exactly the same. Total spend divided by passes comes to $0.143 with ego-browser and $0.142 with agent-browser. Every attempt is priced at one fixed rate: $0.30 per million input tokens, $0.006 per million cache reads and $1.20 per million output tokens. DeepSeek lists its own prices in RMB, with off-peak hours at half price, so your bill will differ. The comparison between tools holds because every attempt is priced the same.
Web app Automation Tasks
If your agent books travel, reconciles invoices or files purchase orders in a web app, we recommend running DeepSeek V4.1 Flash with ego-browser. It passed all 8 attempts on these four tasks with ego-browser, and 3 of 4 with agent-browser. On the three tasks both setups passed, a task took 279.9 seconds with ego-browser and 543.2 with agent-browser. Those tasks cost $0.039 and $0.052.
The trip booking shows the difference. We asked the agent to book round-trip flights to Denver within travel policy, take aisle seats and pick a hotel within a mile of the venue. With ego-browser, DeepSeek V4.1 Flash searched, picked in-policy fares, took seats 13C and 9D, chose a hotel 1.0 mile away and booked, in 49 tool calls. With agent-browser it spent many of its 100 steps on the airport combobox, the date pickers and the seat map, and ran out before confirming a booking.
In our runs this was the type of work where DeepSeek V4.1 Flash came closest to faultless, so it's where we'd deploy it first.
Which browser tool to use with DeepSeek V4.1 Flash
If your agent works through a web app, such as booking, approvals or back-office updates, use ego-browser. DeepSeek V4.1 Flash passed every attempt there and finished in about half the time it took with agent-browser.
If your agent checks posts on X against other sources, ego-browser is the clear pick too. It passed all 6 attempts, against 1 of 3 with agent-browser.
If your agent looks values up on a few sites and compares them, either tool works. ego-browser passed 11 of 16 attempts and agent-browser 5 of 8. Check answers that depend on a rule in the task, such as an exemption or a cutoff date.
If your agent collects the same fields from 8 or more sites, no tool made DeepSeek V4.1 Flash reliable: 3 of 30 attempts passed. We'd split that job into runs of a few sites each, so every run ends with an answer.
To get these results, run DeepSeek V4.1 Flash (deepseek-flash) at high reasoning effort in the pi agent harness, with its read, write, edit and bash tools and a skill file for the browser tool. ego-browser comes with ego(lite) and drives it as the agent's browser.
FAQ
How good is DeepSeek V4.1 Flash at browser use?
It is good inside single web apps and on short lookups, and weak on long multi-site jobs. Across 28 real web tasks run in the same pi agent harness, it passed 51.8% of attempts with ego-browser and 46.4% with agent-browser. It passed 11 of 12 web app automation attempts but 3 of 30 attempts that collect data from 8 to 12 sites. Most of those failures used the whole 100-step budget without writing an answer. Full setup and per-task results are in How we tested.
What is the best browser tool for DeepSeek V4.1 Flash?
ego-browser, in our tests. With DeepSeek V4.1 Flash at high reasoning effort on 28 tasks, it passed 51.8% of attempts against 46.4% for agent-browser. On the 8 tasks both tools passed every time, it took 393.1 seconds per task against 506.0, and cost $0.047 against $0.058. agent-browser was cheaper only when failed attempts are averaged in. Full setup and per-task results are in How we tested.
Is ego-browser or agent-browser faster with DeepSeek V4.1 Flash?
ego-browser is faster on tasks the model completes. On the 8 tasks both setups passed every time, a task took 393.1 seconds with ego-browser and 506.0 with agent-browser, and ego-browser was faster on 6 of 8. The model made 44.1 tool calls per task with ego-browser against 66.5, at about the same time per call. On attempts that ran out of steps, ego-browser attempts ran longer. Full setup and per-task results are in How we tested.
How much does DeepSeek V4.1 Flash cost per browser task?
A few cents per task. On 8 tasks both setups passed, a task cost $0.047 with ego-browser and $0.058 with agent-browser. Over all 28 tasks, failures included, it cost $0.074 and $0.066. Counting failed attempts, each passing result cost about 14 cents in both setups. Prices are token usage at one fixed rate per million tokens. Full setup and per-task results are in How we tested.
Why does DeepSeek V4.1 Flash fail browser tasks?
Mostly because it runs out of steps on long jobs. Of its failed attempts on 28 tasks, 17 of 27 with ego-browser and 10 of 15 with agent-browser used the whole 100-step budget without an answer. On a 12-site central bank task it reached the last bank in its final steps with no table written. The rest were wrong values, usually a task rule applied wrongly. Full setup and per-task results are in How we tested.
Can DeepSeek V4.1 Flash fill forms and complete web app workflows?
Yes, reliably in our tests. Four of our tasks were web app automation, such as booking a trip with seats and a hotel. It passed 8 of 8 attempts with ego-browser and 3 of 4 with agent-browser. The agent-browser attempt that failed ran out of steps on the booking form's pickers. Full setup and per-task results are in How we tested.
Can DeepSeek V4.1 Flash verify social media posts against other sources?
Yes, with ego-browser. Three of our tasks check posts on X against other sources, such as a week of SpaceX launch posts against Wikipedia's launch lists. It passed 6 of 6 attempts with ego-browser and 1 of 3 with agent-browser. On the SpaceX task, which both tools passed, ego-browser took 679.9 seconds against 980.4. Full setup and per-task results are in How we tested.
Can DeepSeek V4.1 Flash scrape data from many websites?
Not reliably when a job spans 8 to 12 sites. On 10 such tasks, such as rate decisions from 12 central banks, it passed 1 of 20 attempts with ego-browser and 2 of 10 with agent-browser. 22 of those 30 attempts used all 100 steps without writing an answer. On search-and-compare tasks across a few sites it did much better, passing 11 of 16 attempts with ego-browser. Full setup and per-task results are in How we tested.
What mistakes does DeepSeek V4.1 Flash make when reading web pages?
Its most common mistake is applying a rule in the task wrongly. Of 15 wrong answers across both tools, 5 broke a status or date rule, 3 cited the wrong source and 4 took a figure from the wrong place. On Companies House it read that Unilever is exempt from reporting its controllers and still wrote 0 in 2 of 3 attempts. Full setup and per-task results are in How we tested.
When DeepSeek V4.1 Flash finishes a browser task, how accurate is it?
About three in four finished answers pass. Across 28 tasks, 29 of 39 answered attempts passed with ego-browser and 13 of 18 with agent-browser. Within those answers, 88.9% and 88.7% of the checked items were right. The tool made little difference here, since both ran the same model in the same agent harness. Full setup and per-task results are in How we tested.
How we tested
Setup
We held the model fixed and changed the browser tool. DeepSeek V4.1 Flash (deepseek-flash) ran at high reasoning effort in the pi agent harness, with its read, write, edit and bash tools and a skill file for each browser tool. The two tools were ego-browser 2.2.0 and agent-browser 0.39.1. agent-browser hands the model an accessibility-tree snapshot with element refs. ego-browser runs scripts the model writes against ego(lite) and returns what they print. Each attempt had 100 steps and 60 minutes at most. ego-browser ran each task twice and agent-browser once.
Tasks
28 tasks on live websites and on test web apps seeded with fresh data for each attempt, in five types of work. Web apps automation (4 tasks), for example booking a trip within travel policy. Search and compare on one to three sites (8), for example matching ClinicalTrials.gov trials to DailyMed labels. Social media verification (3), for example checking a week of SpaceX launch posts on X against Wikipedia. Multi-site data collection on 8 to 12 sites (10), for example the first 2026 rate decision at 12 central banks. Research and synthesis (3), for example reading cost and score off the ARC Prize leaderboard chart. Each task's type of work comes from the task group it was written in and the sites it uses, set before any result was read.
What counts as a pass
Each task lists between 3 and 10 required values, plus a check that the answer comes in the requested format. Most values are fixed in advance; on the test web apps they come from that attempt's seeded data. An attempt passes only when every required value is right. An attempt that ends without a final answer fails every value.
How we measured
An attempt is one run of a task from a fresh start. Success rate counts passing attempts. Time is wall time from start to final answer, and time and cost per task are averages over a task's attempts. Speed and cost are compared on the 8 tasks both setups passed on every attempt. Cost is token usage priced at one fixed rate for every attempt. The model and its price are the same in both tools, so cost gaps are usage. No attempt in the results failed from a provider error or a timeout.
Full results
Every task, its type of work, and each setup's passes, average time and average cost per task.
| Task | Type of work | ego-browser | agent-browser |
|---|---|---|---|
| Book a policy-compliant trip | Web apps automation | 2/2 · 285.2 s · $0.045 | 0/1 · 625.1 s · $0.072 |
| Bring service monitors in line with a standard | Web apps automation | 2/2 · 219.2 s · $0.035 | 1/1 · 439.8 s · $0.060 |
| Match invoices to payments, open disputes | Web apps automation | 2/2 · 378.1 s · $0.048 | 1/1 · 821.1 s · $0.061 |
| Create purchase orders from reorder points | Web apps automation | 2/2 · 242.2 s · $0.033 | 1/1 · 368.6 s · $0.036 |
| Compare three Mac laptops | Search and compare | 2/2 · 606.1 s · $0.081 | 1/1 · 364.2 s · $0.069 |
| Reconcile six films' box office | Search and compare | 2/2 · 368.2 s · $0.040 | 1/1 · 258.4 s · $0.029 |
| KYC snapshot of four UK companies | Search and compare | 1/2 · 355.2 s · $0.049 | 0/1 · 252.1 s · $0.044 |
| Match trials to drug labels | Search and compare | 1/2 · 490.9 s · $0.056 | 1/1 · 394.3 s · $0.049 |
| Inflation from FRED, checked against BLS | Search and compare | 2/2 · 203.9 s · $0.026 | 1/1 · 248.0 s · $0.027 |
| Cheapest nonstop fares on Google Flights | Search and compare | 0/2 · 1707.8 s · $0.126 | 0/1 · 1008.2 s · $0.080 |
| Web features on MDN and Can I use | Search and compare | 2/2 · 612.4 s · $0.064 | 0/1 · 214.6 s · $0.036 |
| Study notes from a lecture, code and paper | Search and compare | 1/2 · 606.4 s · $0.069 | 1/1 · 482.1 s · $0.036 |
| SpaceX launch posts on X against Wikipedia | Social media verification | 2/2 · 679.9 s · $0.034 | 1/1 · 980.4 s · $0.057 |
| Security releases of 8 open-source projects, X against advisories | Social media verification | 2/2 · 1088.2 s · $0.105 | 0/1 · 1612.0 s · $0.096 |
| 8 product launches, X posts against press releases | Social media verification | 2/2 · 2816.5 s · $0.108 | 0/1 · 910.9 s · $0.092 |
| One album across 9 music services | Multi-site data collection | 1/2 · 851.8 s · $0.067 | 1/1 · 410.4 s · $0.039 |
| 8 cameras from 8 brand sites | Multi-site data collection | 0/2 · 850.1 s · $0.080 | 1/1 · 719.2 s · $0.069 |
| Rate decisions at 12 central banks | Multi-site data collection | 0/2 · 1027.9 s · $0.132 | 0/1 · 496.5 s · $0.082 |
| Keynotes of 9 developer conferences | Multi-site data collection | 0/2 · 1484.3 s · $0.116 | 0/1 · 643.7 s · $0.071 |
| 9 electric SUVs | Multi-site data collection | 0/2 · 1042.4 s · $0.077 | 0/1 · 814.0 s · $0.074 |
| 9 open-weight models | Multi-site data collection | 0/2 · 1007.3 s · $0.124 | 0/1 · 598.3 s · $0.109 |
| Minimum wage in 9 jurisdictions | Multi-site data collection | 0/2 · 947.4 s · $0.082 | 0/1 · 532.4 s · $0.064 |
| Flagship phones from 9 brands | Multi-site data collection | 0/2 · 1117.6 s · $0.081 | 0/1 · 584.8 s · $0.071 |
| Compliance of 9 SaaS vendors | Multi-site data collection | 0/2 · 1130.2 s · $0.085 | 0/1 · 761.2 s · $0.071 |
| Free tiers of 9 serverless platforms | Multi-site data collection | 0/2 · 871.8 s · $0.099 | 0/1 · 657.9 s · $0.081 |
| ARC Prize leaderboard chart | Research and synthesis | 2/2 · 447.5 s · $0.077 | 1/1 · 567.6 s · $0.122 |
| Models over EU compute thresholds | Research and synthesis | 1/2 · 637.6 s · $0.081 | 1/1 · 347.8 s · $0.063 |
| Trace a Wikipedia table to its sources | Research and synthesis | 0/2 · 728.2 s · $0.057 | 0/1 · 624.9 s · $0.092 |
Update log: first published with two attempts per task for ego-browser and one for agent-browser, on 28 tasks.
