Selenium WebDriver vs Playwright in a Nutshell
A quick comparison in the context of Web Test Automation
Update (2024â01): Check out my fatherâs Correcting Wrong âPlaywrightâs Advantage over Seleniumâ series. The first article was awarded âBoostâ status by Medium Curators.
General
đ Selenium
đ Playwright
đ W3C WebDriver Standard
đ Not a W3C WebDriver Standard
đ Native support by all major browser vendors
đ No, downloads own browsers and does not include Chrome (has Chromium instead)
đ Established, first released in 2011, still active, the latest version 4.11 released three week ago (2023â07â31)
đ Released in 2020
đ Well-proven, used at Facebook and Google, was regarded the most popular web automation framework
đ It quickly overtook Cypress as the most popular web automation framework in the JS community.
đ Selenium is versatile, and works well for nearly all major programming languages. 5 official ones (Ruby, Python, Java, C# and JavaScript)
đ Playwright is limited to JavaScript, Python, Java and C#
đ Supports all major test syntax frameworks, such as RSpec, PyTest, JUnit and Mocha.
đ Playwright comes with its own test runner. In my opinion, it is less flexible.
Features/Limitations
Selenium is based on the W3C standard, so it is more feature complete and future proof.
Plawright does not work with Chrome (the clientâs Chrome browser). Instead it installs itâs own Chromium version to use. Playwright guarantees functionality only for itâs downloaded browsers, and while executable paths can be defined to use clientâs Chrome, it is generally not recommended.
Selenium can use the Ruby language, a long-time and proven good scripting language for Test Automation (since Watir, ~2003). According to the authors of the classic âAgile Testing Bookâ (Lisa Crispin and Janet Gregory), Ruby is the best language for scripting automated tests â and I agree.
Also, âRuby is the most in-demand skillâ, according to Hiredâs 2023 State of Software Engineers Report.
Playwright cannot clear text fields. It seems like a simple function to just clear a text-field. However, it does not currently exist in Playwright. Current solutions include: pressing backspace input valueâs length number of times, triple clicking then deleting (source) or passing an empty string (documentation).
await page.evaluate( () => document.getElementById("inputID").value = "")Playwright also has an auto test retry feature. If any test fails, Playwright will rerun the entire script file. I think this feature is unnecessary since it falls under the responsibility of the Continuous Testing Server rather than the automation framework.
Speed
A benchmark test running the AgileTravel login and flight tests.
Note that I ran the tests once first to warm up the test site and browser applications before I began recording the timing. Each combination was run 3 times each and approximate average is shown below.
On a Mac
Playwright with Mocha running on Chromium averaged 9 seconds.
Selenium with RSpec running on Chrome averaged 10.4 seconds.
On Windows 10
Playwright with Mocha running on Chromium averaged 9 seconds.
Selenium with RSpec running on Chrome averaged 12.2 seconds.
More about Speed
From the above tests, Playwright is quite a bit faster than Selenium Ruby. This results was different for another benchmark my father did over six months ago. It is not surprising, different test suite and different time.
âthere is no value to debate the test execution speed, such as the set-up for this benchmark test, if reaching the top limit of useful range (Selenium and Playwright both did) in functional testing perspective. Still, we want to shorten the overall execution time, the only practical way is via parallel execution in a proper wayâ
I agree it is not much value debating the which framework is faster, given the different is minor. By running three build agents, I saved ~65% off the total execution time, plus other benefits.
Donât Compare Apples vs Oranges
I do notice some people sing the following for Playwright over Selenium, such as:
- Playwright has test runner
this is provided by test syntax frameworks, such as RSpec and Mocha. - Playwright has built-in HTML reporters
same as the above, this is test syntax framework, not automation frameworkâs responsibility. - Playwright API Testing
this is done via the scripting language, using âhttpclientâ alike libraries. - Playwright Auto-Waiting
this is really a marketing thing. Selenium WebDriverâs waiting is fine, otherwise, how could Facebookâs E2E Test Automation enabled âRelease twice a dayâ. I first heard of Auto-Wait is from TestCafe, which is no longer relevant. The next is Cypress, quite big bragging about it, the result, Cypress.io is dying.
The best waiting mechanism is my fatherâ invention, check out, Test AJAX Properly and Efficiently with Selenium WebDriver, and Avoid âAutomated Waitingâ - Playwright parallel execution support
this should be done in CT server, not automation framework.
These are not the feature of a web automation framework.
Updated (2024â01): two recent independent surveys showed a similar result. Selenium WebDriver is still the choice for web test automation, leading by a huge gap.
Playwright leads in JS automation framework at the price of Cypress, according to JavaScript Rising Stars.
