Jan 7

5 min read

Playwright vs Selenium Speed Comparison

Selenium WebDriver is slightly faster than Playwright.

Update 2023–01–09: Annotated version by my father.

In the previous article, I compared the test syntax of Playwright and Selenium WebDriver. One feedback I have heard is that “Selenium is slower”. Both are fast in terms of test execution from my experience.

Today, I did a benchmark to verify the timing difference.

Test Setup

Test Machine: iMac (2015, macOS 12.6.2)
I chose a relatively slow machine purposefully.
Target website: https://travel.agileway.net
This a simple test site created by my father for training purposes.
Execution method
I developed both versions in TestWise. To remove any factors, I timed and ran them both from the command line.
Framework versions
- Selenium WebDriver: 4.7.1 (Ruby 3.0.2)
- Playwright 1.18.1 (Node v18.10)
Browser versions
- Chrome: 108.0.5359.124 (Official Build)
- Playwright’s Chromium: 99.0.4812.0 (Developer Build)
Test Scenarios
24 test steps covers:
- Launching and Closing the Chrome/Chromium browser
- Various control types, such as Link, Text Field, Checkbox, Radio, Select List, and Button
- Different locators, such as ID, Name, Tag, Attribute, CSS and XPath.
- Assertion

Test Script

TestWise supports Mocha too, i.e. JavaScript frameworks, such as Playwright. But using Ruby is much more fun!

Raw Selenium WebDriver (Ruby) in RSpec syntax framework

it "End-to-End Selenium Raw" do
    driver.find_element(:id, "username").send_keys("agileway")
    driver.find_element(:id, "password").send_keys("testwise")
    driver.find_element(:id, "remember_me").click
    driver.find_element(:id, "username").submit
    expect(driver.find_element(:tag_name, 'body').text).to include("Signed in")
    
    driver.find_element(:xpath, "//input[@name='tripType' and @value='oneway']").click
    Selenium::WebDriver::Support::Select.new(driver.find_element(:name, "fromPort")).select_by(:text, "Sydney")
    Selenium::WebDriver::Support::Select.new(driver.find_element(:name, "toPort")).select_by(:text, "New York")
    Selenium::WebDriver::Support::Select.new(driver.find_element(:id, "departDay")).select_by(:text, "02")
    Selenium::WebDriver::Support::Select.new(driver.find_element(:id, "departMonth")).select_by(:text, "May 2016")
    driver.find_element(:xpath, "//input[@value='Continue']").click
    
    # now on passenger page
    driver.find_element(:name, "passengerFirstName").send_keys("Bob")
    driver.find_element(:name, "passengerLastName").send_keys("Tester")
    driver.find_element(:name, "passengerLastName").submit
    
    # on payment page
    driver.find_element(:xpath, "//input[@name='card_type' and @value='master']").click
    driver.find_element(:name, "holder_name").send_keys("Bob the Tester")
    driver.find_element(:name, "card_number").send_keys("4242424242424242")
    Selenium::WebDriver::Support::Select.new(driver.find_element(:name, "expiry_month")).select_by(:text, "04")
    Selenium::WebDriver::Support::Select.new(driver.find_element(:name, "expiry_year")).select_by(:text, "2016")
    driver.find_element(:xpath, "//input[@value='Pay now']").click
  end

Raw Playwright in Mocha test syntax framework

it('End to End Playwright', async function() {
    this.timeout(5000)
    await driver.locator('text=Login').click()
    await driver.fill("#username", "agileway")
    await driver.fill("#password", "testwise")
    await driver.click("#remember_me")
    await driver.click("input:has-text('Sign in')")
    await driver.textContent("body").then(function(body_text) {
      //console.log(body_text)
      assert(body_text.contains("Signed in"))
    });
    
    const trip_radio = await driver.$$("input[name=tripType]");
    await trip_radio[1].check();
    await driver.selectOption("select[name='fromPort']", "New York");
    await driver.selectOption("select[name='toPort']", "Sydney");
    await driver.selectOption("select[name='departDay']", "02");
    await driver.selectOption("#departMonth", "052021");
    await driver.click("input:has-text('Continue')");
    
    await driver.fill("input[name='passengerFirstName']", "Bob");    
    await driver.fill("input[name='passengerLastName']", "Tester");
    await driver.click("input:has-text('Next')");
    
    const card_type_radio = await driver.$$("input[name='card_type']");
    await card_type_radio[1].check();
    await driver.fill("input[name='holder_name']", "Bob the Tester");
    await driver.fill("input[name='card_number']", "4242424242424242");
    await driver.selectOption("select[name='expiry_month']", "04");
    await driver.selectOption("select[name='expiry_year']", "2016");
    await driver.click("input:has-text('Pay now')");
  });

Benchmark Results (raw)

1. Raw Selenium WebDriver, 2.21

Raw Playwright: The best of 3 runs, the other two: 2.23, 2.50

2. Playwright, 2.34

Raw Playwright: The best of 3 runs, the other two: 2.43, 2.44

Refactored Versions

After getting the test steps right, as a habit, I immediately refactor based on the Maintainable Automated Test Design, using Page Object Model, a well-known design pattern in test automation. Because the raw test steps are hard to read and maintain.

Selenium WebDriver using POM

it "End-to-End Selenium POM" do
    login_page = LoginPage.new(driver)
    login_page.enter_username("agileway")
    login_page.enter_password("testwise")
    login_page.check_remember_me
    login_page.click_sign_in
    expect(driver.find_element(:tag_name, 'body').text).to include("Signed in")
    
    flight_page = FlightPage.new(driver)
    flight_page.select_trip_type("oneway")
    flight_page.select_depart_from("Sydney")
    flight_page.select_arrive_at("New York")
    flight_page.select_depart_day("02")
    flight_page.select_depart_month("May 2016")
    flight_page.click_continue
    
    passenger_page = PassengerPage.new(driver)
    passenger_page.enter_first_name("Bob")
    passenger_page.enter_last_name("Tester")
    passenger_page.click_next
    
    payment_page = PaymentPage.new(driver)
    payment_page.select_card_type("master")
    payment_page.enter_holder_name("Bob the Tester")
    payment_page.enter_card_number("4242424242424242")
    payment_page.enter_expiry_month("04")
    payment_page.enter_expiry_year("2016")
    payment_page.click_pay_now
  end

Playwright using POM

it('E2E Playwright POM', async function() {
    this.timeout(5000);
    
    let login_page = new LoginPage(driver);
    await login_page.enterUsername("agileway");
    await login_page.enterPassword("testwise");
    await login_page.checkRememberMe();
    await login_page.clickSignIn();
    await driver.textContent("body").then(function(body_text) {
      assert(body_text.contains("Signed in"));
    });
    
    let flight_page = new FlightPage(driver);
    await flight_page.selectTripType("oneway");
    await flight_page.selectDepartFrom("New York");
    await flight_page.selectArriveAt("Sydney");
    await flight_page.selectDepartDay("02");
    await flight_page.selectDepartMonth("052021");
    await flight_page.clickContinue();

    let passenger_page = new PassengerPage(driver);
    await passenger_page.enterFirstName("Bob");
    await passenger_page.enterLastName("Tester");
    await passenger_page.clickNext();
    
    let payment_page = new PaymentPage(driver);
    await payment_page.selectCardType("master");
    await payment_page.enterHolderName("Bob the Tester");
    await payment_page.enterCardNumber("4242424242424242");
    await payment_page.enterExpiryMonth("04");
    await payment_page.enterExpiryYear("2016");
    await payment_page.clickPayNow();
});

Astute readers will conclude that putting the differences in language conventions aside, these two versions are quite similar. Yes, that’s the beauty of Page Object Models.

Benchmark Results (POM)

1. Raw Selenium WebDriver using POM design, 2.23

Raw Selenium POM: The best of 3 runs, the other two: 2.24, 2.49

2. Playwright using POM design, 2.34

Raw Playwright: The best of 3 runs, the other two: 2.35, 2.35

Findings

For this simple test scenario, Selenium WebDriver is about 5% faster than Playwright for both the raw and Page Object Model versions. The difference is minor, after taking out variances (network speed at the time and web page content), I would say they are on par, in terms of test execution speed.

This finding was similar to the one by Giovanni Rago two years ago (2021–01): Playwright was slightly faster than Selenium.

With Selenium 4 and the ongoing refinement of ChromeDriver, I have noticed a speed improvement in Selenium test execution. So, I am not too surprised by the result: Selenium is faster than Playwright.

Why I did not include Cypress?

I just did a google search of “Playwright vs Cypress”, and the results page shows, if there is any judgement, favour Playwright.

Speedwise, Playwright is faster than Cypress, according to this post.

Giovanni Rago did a thorough test comparing the performance of tools under different scenarios: Cypress vs Selenium vs Playwright vs Puppeteer speed comparison. The test result: not much difference in terms of speed for the four frameworks, however, Cypress was the slowest in most categories.

So I did not bother timing Cypress; even though they claimed to be much, much faster on their web page.