Puppeteer vs Playwright vs Selenium for scraping

14 min read
Puppeteer vs Playwright vs Selenium for scraping: 2026 comparison

Choose Playwright for a new workflow that needs its three browser engines and locator checks, Puppeteer for a compatible Node.js browser integration, or Selenium for its supported bindings and established Grid. Puppeteer also supports Firefox through BiDi. Profile reuse and authorized login state are separate configuration choices in all three. Match the required API, browser and deployment first, then measure your own permitted scraping task.

Every scraping team eventually holds this three-way debate, usually with one member defending the Selenium suite they've run for a decade, one who came up on Puppeteer, and one who read that Playwright won.

All three positions have merit, because the tools were built in different eras for different jobs: Selenium (2004) for cross-browser testing breadth, Puppeteer (2017) for direct Chrome control, Playwright (2020) for modern-web reliability.

How do the three architectures differ?

The SeleniumHQ/selenium GitHub repository, described as a browser automation framework and ecosystem
SeleniumHQ/selenium, the oldest project in this comparison and the only one that describes itself as an ecosystem rather than a library. Its WebDriver architecture is the structural difference the next paragraphs walk through.

Selenium speaks the WebDriver protocol: classic commands pass through a browser-specific driver. Current Selenium also exposes bidirectional capabilities. Its driver and Grid arrangement affects deployment, but a driver hop alone is not a measured end-to-end throughput result.

Puppeteer uses CDP by default with Chrome and BiDi with Firefox. Playwright uses its own protocol for full-featured connections; connectOverCDP is a separate Chromium-only, lower-fidelity route. Selenium also exposes BiDi capabilities. Check the API you need on the selected browser and connection.

Network requests, page scripts, waiting conditions and extraction work can dominate a scraper. Compare the same targets, browser builds, concurrency and acceptance rules before attributing a throughput difference to the controller.

One fairness note on trajectory: the gap is narrowing. WebDriver BiDi, the W3C's bidirectional successor protocol, is landing across the Selenium ecosystem and already powers things like console and network capture in Selenium-based tooling such as mcp-selenium's diagnostics tool. The architecture argument above describes 2026's shipping reality, not a law of nature.

Protocol support matters when a required operation is unavailable; performance still needs a comparable workload.

How do they compare on the scraping matrix?

Five rows cover what scraping teams actually ask. Each cell includes the limitation, because that's the half that decides.

DimensionSeleniumPuppeteerPlaywright
LanguagesOfficial bindings: Java, Python, C#, Ruby and JavaScriptJS/TS only; Python port unofficial and laggingFour language families: Node.js (JS/TS), Python, Java and .NET
BrowsersDriver-dependent browser support; check the current target and driverChrome and Firefox via BiDi; documented feature gaps, no WebKitChromium, Firefox, WebKit; no legacy browsers
Waiting modelExplicit waits for conditions; optional implicit element wait is off by defaultLocators auto-wait; lower-level flows need explicit readiness checksLocator actions auto-wait for relevant checks; assertions can retry
Stealth ecosystemundetected-chromedriver, SeleniumBasepuppeteer-extra-plugin-stealth, the classicplaywright-extra, plus current forks (Patchright, Camoufox)
Parallel sessionsGrid: mature but heavyweight infrastructureProcess-per-browser typical; contexts less ergonomicBrowser contexts: cheap isolated sessions in one process

The waiting row is worth seeing in code. Each example waits for the button to become actionable, but none verifies that the click loaded the next results page:

# Selenium: you compose the wait
WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.CSS_SELECTOR, "#load-more"))).click()

// Puppeteer: Locator waits for actionability
await page.locator('#load-more').click()

// Playwright: Locator waits for actionability
await page.locator('#load-more').click()

Puppeteer's older page.click path and ElementHandle flows differ from its recommended Locator API. Both Locator examples reduce manual element waits; the Selenium example composes an explicit wait. There is no measured flake-rate comparison here. After any of these clicks, assert that the expected new results actually appeared.

Source: Selenium waiting strategies, Puppeteer Locator guidance, Playwright actionability checks, and Puppeteer's supported browser versions (checked September 28, 2026).

What are the pros and cons of each?

Selenium

Pros: supported language bindings, WebDriver standards and established Grid deployments. Cons: driver and Grid operations add configuration, and application readiness needs explicit conditions. BiDi capabilities should be checked for the selected browser and binding rather than dismissed as HTTP-only.

Puppeteer

The puppeteer/puppeteer GitHub repository, 95.5k stars, JavaScript API for Chrome and Firefox
puppeteer/puppeteer at 95.5k stars, the CDP camp's original member. The star gap over Selenium is real, but so is Selenium's two-decade head start in deployed infrastructure.

Pros: direct CDP control on Chrome, a long-standing stealth plugin, and many scraping recipes. Cons: JavaScript only, Chrome-first with Firefox on WebDriver BiDi (some features remain unsupported), and lower-level APIs that require explicit readiness checks. Puppeteer Locators auto-wait for actionability.

Playwright

The microsoft/playwright GitHub repository: framework for Web Testing and Automation covering Chromium, Firefox and WebKit with a single API
Archived microsoft/playwright repository screenshot. The current language families are Node.js, Python, Java and .NET; use the linked official guide for support rather than counting JavaScript and TypeScript as separate bindings.

Pros: locator actionability checks, retrying assertions, browser contexts and Chromium, Firefox and WebKit support across four language families. Cons: managed browser builds add installation work, and engine or connection support still needs verification. Auto-waiting cannot verify application-specific outcomes on its own.

Which should you pick for scraping in 2026?

For a new multi-engine scraper, Playwright is a reasonable starting point because of its contexts and locator checks. Verify the deployed process settings and the fields extracted on your authorized target before adopting it.

For a Node.js workflow needing Puppeteer APIs on a supported Chrome or Firefox build, evaluate Puppeteer. This article has no current comparable capture-speed test.

For Ruby or an established Selenium Grid deployment, evaluate Selenium before migrating. PHP is not one of the official bindings listed here; inspect any third-party binding separately.

The trap to avoid is migrating a working stack for identity reasons. All three scrape competently; switching costs are real; move when a specific limit (language, flake rate, parallelism cost) actually binds, and measure that limit on your own targets before the rewrite, not after.

Which framework should a QA engineer learn?

For a new QA automation learner in 2026, start with Playwright unless your team already standardizes on Selenium or Cypress. Playwright gives one modern API across major browsers, built-in auto-waiting, tracing, and parallel contexts; Selenium remains the better investment when job requirements or an existing Grid require Java, Python, C#, Ruby, or legacy browser coverage.

  • Choose Playwright for a greenfield web suite and learn locators, fixtures, traces, and CI parallelism first.
  • Choose Selenium when cross-language portability, WebDriver-standard tooling, or a large existing estate matters more than a shorter setup.
  • Choose Cypress when its in-browser runner and team workflow are the deciding constraint; it is a separate trade-off, not a drop-in replacement for either framework.

The durable skill is test design: deterministic fixtures, accessible locators, isolation, and useful failure artifacts. Framework syntax changes faster than those principles.

How should each framework handle bot walls and CAPTCHAs?

None of Selenium, Puppeteer, or Playwright reliably defeats Cloudflare, CAPTCHA, WAF, or rate limits, and this comparison does not provide bypass instructions. A block is a signal to reduce load, use an official API or licensed feed, request permission, or stop the collection—not to escalate evasion.

  • Respect robots directives, terms, consent requirements, rate limits, and the site's account policy; identify your client where appropriate.
  • Use backoff, caching, incremental checkpoints, and a bounded request budget for permitted public data.
  • For data behind an account you own, use the supported login flow or an approved export. Do not replay cookies, rotate identities, or spoof fingerprints to evade a control.

Stealth plugins and realistic mouse movement may change a browser fingerprint, but they are not a compliance or reliability guarantee. Treat CAPTCHA and 403 responses as a workflow state that needs a human or a different data source.

How do they handle logins, infinite scroll, and pagination?

All three can handle login flows, infinite scroll, and cursor or page-number pagination, but reliable scraping depends on state modeling rather than a single selector. Wait for a meaningful state change, capture the current cursor or URL, and checkpoint each page before continuing.

  • Login: prefer a dedicated test account or supported session bootstrap, keep secrets outside source control, and stop for MFA or consent instead of attempting to bypass it.
  • Infinite scroll: scroll in bounded increments, wait for new item IDs, deduplicate, and stop when the count or sentinel stops changing.
  • Pagination: persist the next cursor, validate row counts, and make retries idempotent so a timeout does not duplicate records.

Playwright and Puppeteer Locators can reduce element timing errors; Selenium's explicit waits can target the same conditions. Each can be correct when the script checks application state instead of relying on fixed sleeps.

Which process and connection settings must a scraper own?

Choose the library and its deployment contract together. Playwright's current BrowserType reference says chromiumSandbox defaults to false. Its Docker guide says running the browser as root disables the Chromium sandbox and recommends a separate user plus the supplied seccomp profile for crawling untrusted sites. A working container or a visible browser does not prove that process isolation is enabled.

Distinguish the browser sandbox from browser contexts. Contexts separate cookies and storage for jobs; they do not, by themselves, constrain the host Agent's shell, downloads or network. Record the process user, resolved browser build, sandbox setting and network policy with the scraper. Do not turn a documentation check into a claim that a deployment has been penetration-tested.

Connection choice also changes the supported feature surface. Playwright documents connectOverCDP as Chromium-only and significantly lower fidelity than its native connect protocol. Puppeteer supports Chrome and Firefox through WebDriver BiDi, with unsupported operations documented separately; Chrome uses CDP by default and Firefox uses BiDi. Selenium also exposes BiDi capabilities. A protocol label alone cannot rank end-to-end scraping speed.

This November 11, 2026 update reviewed the current vendor documentation without installing browser packages or running a new throughput or sandbox test. Keep the historical waiting examples below separate from current support claims. Compare one authorized workload with the same browser, account, request volume and acceptance checks before paying for a migration.

Configuration questionDocumented boundaryOperator check
Playwright Chromium sandboxchromiumSandbox defaults to falseInspect the actual launch setting and process user
Docker crawlingRoot disables Chromium sandbox; non-root plus seccomp is recommendedVerify the deployed image, user and policy
Existing browser over CDPPlaywright connection is Chromium-only and lower fidelityCheck required features on that connection
Browser coveragePuppeteer supports Chrome and Firefox; Playwright adds WebKitCheck the target build and API, not only the brand

What changes at large scraping scale?

At tens of thousands of pages per day, the framework is only one part of the system. Queue URLs, enforce per-domain budgets, reuse browser contexts safely, persist checkpoints, and separate extraction workers from validation and export so one slow site cannot stall the fleet.

  • Use Playwright when browser contexts, tracing, and modern parallelism are the priority; use Selenium Grid when your organization already operates a broad, multi-language fleet.
  • Use Puppeteer for a focused Chrome/Node pipeline when its direct CDP model and team expertise outweigh cross-browser needs.
  • Add observability for queue age, success rate, bytes, browser minutes, retries, block responses, and cost per valid record.

Scale is not permission to ignore a site's controls. If a target cannot support the requested rate, negotiate access or choose an API, feed, or licensed provider instead of adding more parallel browsers.

Which framework fits each scraping use case?

Pick the framework that matches the dominant constraint: Playwright for three-engine workflows, Puppeteer for a compatible Node.js browser integration, and Selenium for supported bindings or an established Grid. Keep a stable incumbent until a measured limit justifies migration.

Use caseGood defaultWatch out for
Modern web app, several browsersPlaywrightBrowser version pinning and target-site limits
Node extraction with a supported Puppeteer browserPuppeteerNo official non-JS API parity
Java/Python/C# estate or GridSeleniumDriver operations and explicit readiness conditions
Your own logged-in portalSupported session or ego(lite)Consent, account policy, and isolation

How do you keep browser tests and scrapers stable?

Stability comes from waiting on observable state, using resilient locators, isolating data, and retaining a trace or equivalent artifact for every failure. Do not hide flake by increasing global timeouts or blindly retrying a mutating scrape.

  • Prefer role, label, test-id, or stable data attributes over generated CSS classes and screen coordinates.
  • Use bounded retries only for known transient failures, and mark a test flaky when retries conceal a real regression.
  • Pin browser and framework versions, run a small smoke flow after upgrades, and retain traces, console logs, and response status for diagnosis.

For scrapers, add schema validation and duplicate detection after extraction. A page that loaded successfully but returned an empty or shifted schema is a data-quality failure, not a successful run.

How do you survive browser and anti-bot updates?

Treat browser updates, layout changes, and new access controls as expected drift. Keep selectors and extraction rules in a small versioned layer, monitor representative pages, and canary upgrades before rolling them across the fleet.

  • Pin dependencies for reproducibility, but schedule security and browser updates; verify release notes and run smoke tests before unpinning.
  • Alert on sudden changes in status codes, item counts, field null rates, latency, and CAPTCHA or consent frequency.
  • When a site changes its policy or markup, pause the job, review permission and contract terms, then update the parser or source—not a stealth workaround.

A maintained fallback can be an official API, export, or licensed dataset. Reliability is the ability to recover transparently, not the ability to keep scraping after every control changes.

What about your own logged-in sources?

For account-dependent work, all three can use deliberately configured authorized state. A fresh browser starts empty; persistent profiles and saved authentication are separate choices with expiry and protection requirements. A visible window does not supply a login.

ego(lite) is an interactive browser execution option when the job benefits from an authorized local session, visible inspection and user takeover. Verify the active account and current permission scope before acting. It does not remove session expiry, grant access through challenges or replace the three libraries' CI and batch infrastructure.

That route is measured on Real-World Bench, a public 31-task benchmark against live production sites, and the tasks look like the scraping work teams actually assign rather than demo pages: cross-comparing Metacritic's highest-rated Action RPGs on PS5 against PC, five games per platform with Metascores, or collecting the top posts about index funds from r/personalfinance sorted by top over the past year. Selenium and Puppeteer were not among the tools measured; the harness and dataset are open atcitrolabs/ego-browser-benchmark-framework.

The following archived transcript records an earlier public-page read and its then-current API. It is not a current setup recipe or a new November 11 trial. For a new run, read the installed ego-browser CLI guide, use one TaskSpace and verify the returned fields against the intended page.

ego-browser nodejs <<'EOF'
const task = await egoBrowser.newTaskSpace('evidence-egobrowser-hn')
console.log({ taskSpaceId: task.id })

await task.page.goto('https://news.ycombinator.com/', { waitUntil: 'load', timeout: 20000 })
const title = await task.page.title()
const topStory = await task.page.locator('.athing .titleline > a').first().innerText()
const points = await task.page.locator('.subtext .score').first().innerText().catch(() => null)
console.log({ title, url: task.page.url(), topStory, points })
EOF

# Real output
{
  "taskSpaceId": 13
}
{
  "title": "Hacker News",
  "url": "https://news.ycombinator.com/",
  "topStory": "Qwen 3.8 27B",
  "points": "412 points"
}

Use that interactive route only when its session and takeover behavior fits the task. For unattended batch scraping or CI, keep the library and process configuration that meets the workload; the archived transcript does not compare their speed or reliability.

See ego(lite) vs Selenium, or download ego(lite) for Mac, free.

FAQ

Which is better, Selenium or Puppeteer?

For Chrome-first scraping in a JavaScript codebase, Puppeteer offers direct CDP control on Chrome and WebDriver BiDi on Firefox. For other languages, legacy browsers, or existing Grid infrastructure, Selenium may fit better. Match the choice to the actual targets and existing infrastructure.

What are Playwright vs Selenium pros and cons in short?

Playwright offers locator actionability checks, contexts and three browser engines. Selenium offers supported language bindings, Grid and WebDriver plus BiDi capabilities. Both require application-level readiness and result checks; keep an existing deployment until a measured limitation justifies changing it.

Is Selenium too slow for web scraping?

This article has no current comparable throughput test that establishes Selenium as too slow. Driver placement, network waits, readiness conditions and extraction work all contribute. Measure the same authorized workload before deciding that a protocol change is worth a migration.

Which of the three works best with AI coding agents?

Playwright, by infrastructure: its MCP server and CLI are first-party and maintained, while Puppeteer's reference MCP server is deprecated and Selenium's is community-run. Playwright's locator assertions reduce manual wait code; Puppeteer Locators also auto-wait, and neither proves an application write succeeded. Selenium shops can still connect agents through the community mcp-selenium; our Selenium MCP guide covers that setup.

Is Playwright replacing Selenium?

In new-project defaults, largely yes; in installed base, no. Selenium's language breadth, Grid infrastructure, and two decades of institutional knowledge keep it running enormous estates that have no reason to move. The accurate statement is that the frontier shifted, not that the incumbent died.

Do any of the three handle CAPTCHAs or bot walls automatically?

No framework grants access through a CAPTCHA or site restriction. Use an official API, export or explicitly supported account workflow; stop and seek permission when challenged. Third-party plugin claims do not establish permission, reliability or safety.

Keep reading