Jan 20

6 min read

Case Study: Mechanize Performance/Load Testing User Login with CSRF Token Protection

How to use Mechanize for a website’s performance and load testing

Mechanize is a library for automating interactions with websites, without a browser involved. Mechanize is implemented in many scripting languages, such as Perl (WWW::Mechanize), Python (mechanize) and Ruby. In this exercise, I will use it for performance/load testing in Ruby.

If you are new to Ruby, don’t need to worry about learning the Mechanize syntax, just run the sample scripts. The purpose here is to get you a feel of scripting a performance/load test using Mechanize.

Preparation

Mechanize is a Ruby library, besides Ruby, you will need to install the Mechanize gem as well.

gem install mechanize

Login

Below is a Mechanize script that logs into a website.

require 'mechanize'
@browser = ::Mechanize.new
@browser.get("https://whenwise.agileway.net")
@browser.get("https://whenwise.agileway.net/sign-in")
login_form = @browser.page.forms.first
login_form.fields_with(:name => "session[email]").first.value = "driving@biz.com"
login_form.field_with(:name => "session[password]").value = "test01"
@browser.submit(login_form, login_form.button_with(:id => "sign-in-btn"))

When you run it, the program ends after about 1 second. Did it actually work?

Add assertion

We can add an assertion to verify that our Virtual User actually landed on the dashboard page.

# verify the specific text only shown to logged in user
raise "Not logged in" unless @browser.page.body.include?("Current Plan")

Debugging

Still not sure, we can save the current page to a file and open it in a browser to inspect.

File.open("/tmp/a.html", "w").write(@browser.page.body)
# then open /tmp/a.html in Chrome

It looks like this.

The page is rendered without CSS and JavaScript, we could modify this off-line HTML to use the site’s web resources (such as CSS, images and JavaScript). But it is not necessary, we only care about the actual page content in load testing.

Add measurements

The above automated script works functionally. To make it a performance test, we need to add some measurements.

In functional testing, the measurement of execution time is simple: end_time minus start_time. For performance/load testing, the measurement is on the time it took for the server to handle individual user operations. For a typical web request:

A. User initiates an action, e.g. click a link
B. The request arrives at the server
C. The server sends the response (HTML) back D. The response arrives at the client’s browser
E. The browser finishes the rendering of the page

The correcting timing for performance/load testing is B→C. However, the network transferring times (A→B and C→D) is negligible (and hard to measure consistently) nowadays. Therefore, we normally just take the timings of A→D.

Below is basic timing in Ruby.

start_time = Time.now
# operation here ..
duration = Time.now - start_time
puts "took #{duration} seconds"

As the above is going to be used often, I will extract them into a reusable function, log_time:

def log_time(msg, &block)
  start_time = Time.now
  begin
   yield
  ensure
    end_time = Time.now - start_time
    puts("#{msg} | #{end_time}")    
  end
end

A sample usage:

log_time("Visit home page") { 
   @browser.get("https://whenwise.agileway.net") 
}

Performance Testing

Record the timings of each operation, then we get a performance test script.

require 'mechanize'

site_url = "https://whenwise.agileway.net"
@browser = ::Mechanize.new

log_time("Visit home page") { @browser.get(site_url) }
log_time("Visit login page") { @browser.get(site_url + "/sign-in") }

login_form = @browser.page.forms.first

log_time("enter data") {
  login_form.field_with(name:"session[email]").value = "driving@biz.com"
  login_form.field_with(:name=> "session[password]").value = "test01"
}

log_time("Login") {
  @browser.submit(login_form, login_form.button_with(:id=> "sign-in-btn"))
}

The log_timefunction is predefined. Here is what an output looks like.

|Visit home page  |1.088529|
|Visit login page |0.208664|
|enter data       |0.000036|
|Login            |1.043211|

These timings are true performance testing results (compared to when you do these operations in Chrome), as no browser rendering is required.

The timing of “enter data” in the above example is unnecessary, as there is no interactions with the server. Those two steps simply prepare the form data for submission.

Simple Load Testing

Next, we will generate load with a number of concurrent users.

One way is to run the command multiple times such as:

ruby load_test.rb &; ruby load_test.rb &; ruby load_test.rb &

Of course, this way of generating load is no good, as it is hard to manage and unable to aggregate the timings. However, this gives engineers a quick taste of load testing.

Generate load

In protocol-based testing, concurrency is typically implemented using threads.

Threads, in software programming, are a way for a program to split itself into two or more simultaneously running tasks.

require 'mechanize'

@browser = ::Mechanize.new
virtual_user_count = 5 # may change concurrent users count here
threads = []

virtual_user_count.times do |idx|
  threads[idx] = Thread.new do
    Thread.current[:id] = idx + 1
    @browser.get("https://whenwise.agileway.net")
    @browser.get("https://whenwise.agileway.net/sign-in") 
    login_form =  @browser.page.forms.first
    login_form.field_with(:name => "session[email]").value = "driving@biz.com"
    login_form.field_with(:name=> "session[password]").value = "test01"
    @browser.submit(login_form, login_form.button_with(:id=> "sign-in-btn"))
    raise "Not logged in " unless @browser.page.body.include?("Current Plan")
  end
end

threads.each {|t| t.join; } # wait for all threads to complete

There will be no output from executing the above script (because there is no timing yet), however, if you monitor the log of your app, you will see a lot of activity there.

Measure timings

By combining the performance test script and load generation, we get a basic load testing script.

# ...
threads[idx] = Thread.new do
 Thread.current[:id] = idx + 1
  log_time("Visit home page") {   
     @browser.get("https://whenwise.agileway.net") 
  }
  log_time("Visit login page") { 
    @browser.get("https://whenwise.agileway.net/sign-in") 
  }
  login_form =  @browser.page.forms.first
  log_time("enter_data") {
    login_form.field_with(:name => "session[email]").value = "driving@biz.com"
    login_form.field_with(:name=> "session[password]").value = "test01"
  } 
  log_time("Login") {
    @browser.submit(login_form, login_form.button_with(:id=> "sign-in-btn"))
  }
  raise "Not logged in " unless @browser.page.body.include?("Current Plan")
end

threads.each {|t| t.join; }

Output:

1|Visit home page|1.069316
3|Visit home page|1.031282
2|Visit home page|1.088884
1|Visit login page|0.232194
1|enter_data|3.5e-05
3|Visit login page|0.247188
3|enter_data|3.1e-05
2|Visit login page|0.248053
2|enter_data|0.000107
1|Login|3.244234
3|Login|3.417211
2|Login|3.5701

Here are the individual timings (in seconds) of each of the 10 Virtual Users.

Compare Load Test Results

Change the virtual_user_count in the load script to 1, 5, 10, 20, 50 and 100, then run the test and get the average timings of the operations. We get a simple yet meaningful load test report.

Failure to load test AJAX

Protocol-based load scripts can handle standard HTTP Requests, but not AJAX, like the one below.

The ‘Pay now’ button uses AJAX to load the receipt on the same page.

From the testing perspective, an AJAX operation immediately ‘completes’ after the mouse/keyboard action (such as clicking the ‘Pay now’ button), and no page reload is observed. After the server finishes processing the request, seconds or even minutes later, some part of the web page may be updated.

The below is an attempt to invoke a typical AJAX operation in Mechanize.

# ...
@browser.get("https://travel.agileway.net/flights/passenger/1")

passenger_form = @browser.page.forms.first
passenger_form.field_with(name: "passengerLastName").value = "Wise"
@browser.submit(passenger_form)  # standard HTTP request, OK

payment_form =  @browser.page.forms.first    
File.write("/tmp/a.html", @browser.page.body)

@browser.submit(payment_form)    # a AJAX operation
#...

It will fail with an error.

502 => Net::HTTPBadGateway for https://travel.agileway.net/payment/confirm — 
unhandled response (Mechanize::ResponseCodeError)

Some will say Mechanize is not useful, as AJAX is commonly used in modern web apps. That’s correct. Some vendors have come up with workarounds for testing AJAX with protocol-based test scripts, but they are unreliable or too complicated. Using real browsers is a more practical and intuitive way to load testing modern web apps.

I will write about a practical approach to load test AJAX operations in the next article.

Having said that, Protocol-based performance/load testing can still be useful. Because even in many modern web apps, protocol-based scripts can test the following mostly visited pages or features:

Home Page
Sign in and sign off
Sign up
Password reset
etc.