Case Study: User Change Password Automated Test

This article will provide four approaches to my father’s One Test Automation Scenario Interview Question that Most Candidates Failed.

“Change a User’s Password”

Be aware of Side-Effect of Test Execution

Unlike Unit Testing, functional testing may have side effects, that is, changing the state of the application or certain data. This test case falls into that category.

After a user changed his password successfully, we cannot use the old password to log in again. The below video (animated GIF) shows the execution of running a change password test twice, then failing the second time.

Run the user-change-password twice. first time: pass; second time: failed

Here I will present four approaches.

1. Change the password back

Once the password has been changed, immediately change it back to the previous password.

Test Script:

# try running this case twice
it "[1] User can change password, Change it back" do
  sign_in("james@client.com", "test01")
  click_avatar_menu("Edit Profile")
edit_user_page = EditUserPage.new(browser)
  edit_user_page.enter_password("newpass")
  edit_user_page.enter_confirm_password("newpass")
  edit_user_page.click_save
relogin("james@client.com", "newpass")
# reset the user's password back
  click_avatar_menu("Edit Profile")
  edit_user_page = EditUserPage.new(browser)
  edit_user_page.enter_password("test01")
  edit_user_page.enter_confirm_password("test01")
  edit_user_page.click_save
  try_for(3, 0.5) { expect(toast_text).to include("User profile has been updated successfully") }
end

Pros:

• Easy to understand

Cons:

• Failures on the second change (set it back) could still leave the data in an invalid state
• Some gap time

Gap Time: I use this term to define the time that the user login was invalid. During that time, other users (or tests) unable to login.
Obviously, the shorter gap time, the better.

Discussion:

A logical question from an experienced tester would be: that is not reliable, as the second change-password steps could fail. Yes, that’s true.

I added a good degree of fault tolerance, using retry in Ruby, in the case, up to 5 attempts.

# reset the user's password back
    attempt_count = 1
    begin
      click_avatar_menu("Edit Profile")
      edit_user_page = EditUserPage.new(browser)
      edit_user_page.enter_password("test01")
      sleep 0.1
      edit_user_page.enter_confirm_password("test01")
      edit_user_page.click_save
      try_for(3, 0.5) { expect(toast_text).to include("User profile has been updated successfully") }
    rescue => e
      attempt_count += 1
      if attempt_count < 5
        retry
      end
    end

The test script shall achieve >99% reliability now. For the rest <1%, I would leave to the Auto-Rerun feature of a Continuous Testing Server, such BuildWise. If someone is fixated on 100% reliability, BuildWise also has a ‘Manual Rerun’ feature.

2. Starting with a newly created user

Use a newly created temporary account as the test user, so we won’t care about changing its password afterwards.

Test Script

it "[2] User can change password, NEW USER EVERY TIME" do
    visit("/sign-up")
    sign_up_page = SignUpPage.new(browser)
    new_email = Faker::Internet.email
    puts new_email
    try_for(2) { sign_up_page.enter_email(new_email) }
    sign_up_page.enter_password("test02")
    fail_safe{ sign_up_page.enter_captcha("wise") }
    sign_up_page.click_create_account
    expect(page_text).to include("Please check your email to activate your account.")
    
    activate_url = nil
    try_for(10, 5) {
      open_email("Account activation") { |email_body_html|
        activate_url = Nokogiri::HTML(email_body_html).at_css("a#activate-link")["href"]
      }
      driver.get(activate_url)
      try_for(3, 0.5) { expect(toast_text).to include("Account activated, you can sign in now.") }
    }
    
    # Now new change password steps...
    sign_in(new_email, "test02")
    expect(page_text).to include("Find Business")
    
    click_avatar_menu("Edit Profile")
    
    edit_user_page = EditUserPage.new(browser)
    edit_user_page.enter_password("newpass")
    edit_user_page.enter_confirm_password("newpass")
    edit_user_page.click_save
relogin(new_email, "newpass")
    try_for(2) { expect(toast_text).to include("You have signed in successfully") }
  end

Pros:

• Guaranteed working (if user registration works)
• No issues with gap time, as nobody is aware of this new temp user.

Cons:

• May leave many temporary users created in the system
  (if doing proper CI/CD, running a whole suite of automated tests multiple times a day, that means quite a lot)
• Slow
• Dependent on user registration
• Maybe incur extra costs (for example, one admin told me that MS Dynamic 365 accounts costs money)

Discussions:

Some would write delete the new user in after(:all) or tearDown() . This means extra test steps: longer running time and possible failures as well.

after(:all) do 
  // add logic to find and delete his newly created user
driver.quit
end

3. Save the password-change status

Store test users’ credentials somewhere, such as a shared folder, so that you can read them from the test scripts. The automated script will get the password dynamically from this special file.

After changing one user’s password successfully, an automated test script must update this special file.

Test Script

# create a users.json file on a shared folder, e.g. /Users/Shared/user.json
  it "[3] User can change password, read password from external file" do
    users_login = JSON.parse(File.read('/Users/Shared/user.json'))
    
    # get current password from the JSON
    password = users_login["james@client.com"]
    
    # create a new password
    new_pass = Faker::Internet.password(min_length:6, max_length:6)
    
    # Login
    sign_in("james@client.com", password)
    click_avatar_menu("Edit Profile")
    
    # Change the password
    edit_user_page = EditUserPage.new(browser)
    edit_user_page.enter_password(new_pass)
    edit_user_page.enter_confirm_password(new_pass)
    edit_user_page.click_save
    try_for(2) { expect(toast_text).to include("User profile has been updated successfully.") }
    
    # if change successful, update users.json
    users_login["james@client.com"] = new_pass
    
    relogin("james@client.com", new_pass)
    try_for(2) { expect(toast_text).to include("You have signed in successfully") }
File.write('/Users/Shared/user.json', JSON.dump(users_login))
  end

Pros:

• Quick, no need to revert (second changing it back).
• Short gap time

Cons:

• Dependent on an external file (access, permission, … )
• Extra care needs to consider when executing tests in different environments, such as in a CT server or Build Agents.
• Need to check the special file before performing manual testing
• The file might be modified by humans or saved in a wrong format

4. Database Reset

Invoke a utility feature that reset the data in the database. For more, stay tuned for my father’s “The Simpsons’ Pattern” article. For now, check out “Free Test Automation Practice Site with Database Reset”. You can try out the database reset utility now.

Test Script:

it "[4] User can change password, DB RESET" do
  reset
sign_in("james@client.com", "originalpass")
  click_avatar_menu("Edit Profile")
    
  edit_user_page = EditUserPage.new(browser)
  edit_user_page.enter_password("newpass")
  edit_user_page.enter_confirm_password("newpass")
  edit_user_page.click_save
relogin("james@client.com", "newpass")
  try_for(2) { 
    expect(toast_text).to include("signed in successfully") 
  }
end

Pros:

• Simple
• Fast, if done properly
• Reliable
• Short gap time (if we call reset ) after the test execution

Cons:

• Development assistance is required.
• Database reset needs to be very quick.
  Below is the timing for WhenWise’s quick reset.

Reset [Quick] took  0.38085 seconds