Case Study: User Login Tests in Selenium WebDriver

A super simple automated test case that some might still get wrong.

Testing user login is usually the first testing task for an app. This article will go through typical user login testcases in Selenium WebDriver and highlight a common pitfall in automated testing.

Example site, Agile Travel (https://travel.agileway.net/login).

AgileTravel login page

Test Design

For login tests, there are usually at least two test cases.

1. One happy case: Login successfully
2. One alternative case: Login failed from bad credentials

Test Case 1: Login OK

The test script (Selenium + RSpec) below is quite straight-forward, it consists of four steps; just like how it is done manually.

it "[1] Can sign in OK" do
  driver.find_element(:id, "username").send_keys("agileway")
  driver.find_element(:id, "password").send_keys("testwise")
  driver.find_element(:id, "username").submit   
  expect(driver.page_source).to include("Signed in")
end

Run this test case.

It passes! Let’s move on to the alternative case now.

Test Case 2: Login Failed

The steps are the same, except with different input (wrong password) and assertions.

expect(driver.page_source).to include("Invalid email or password")

Run this test.

This test case passes as well!

Refine the Whole Test Script

However, the job is not done yet. While both tests passed individually, we want to make sure the whole test script (including these two together) passes as well. To do that in TestWise (a testing IDE), click the “Run Test Script” button (big blue triangle) on the toolbar.

TestWise’s “Run all tests in the file” button

The result: only the first one (login OK) passes, and the second alternate case fails.

Why?

Analyse and Fix it

After the execution of the first test case, the user is logged in successfully in the browser. The second test tried to continue from there, but it was expecting the login page. That’s why it failed on the first line (line 29) of the second test case.

To fix this, we want a clean state before each test case. The easiest way to do this is logout at the end of the login OK case.

Click “Sign off” to logout after the login OK case

This is the easiest way to make sure the second test case starts on the login page.

This works, but if there are more alternate test cases in this login test script file, it might make more sense to extract it to the after(:each)block.