Running Playwright Mocha tests in BuildWise CT Server

I am currently undertaking an internship, where I have been using Microsoft’s Playwright test automation framework for automated UI testing. I completed the test suite in raw Selenium WebDriver + RSpec first which was much easier. Once I started writing a couple of tests in Playwright, I have been working on getting it set up on a Continuous Testing (CT) server.

BuildWise is an open-source Continuous Testing Server available online for free. See this article for why Continuous Testing is a crucial process in software development:

Playwright Test Runner vs Playwright with Mocha

Playwright with PlaywrightTest (Playwright’s default syntax framework) cannot be set up in a CT server yet (from my understanding). As I was not able to find any solutions online that enabled this.

Luckily, Playwright supports other test syntax frameworks (called third-party runners in their documentation). To run Playwright in BuildWise, you should use the Mocha test syntax framework. Mocha is one of the most popular JavaScript syntax frameworks.

This article will cover how to set up a Playwright Mocha project to run in BuildWise, a CT Server.

Installing BuildWise

First follow this article to install BuildWise and familiarise yourself with the navigation.

Creating a Project

Fill in details for your new project (I am using the sample Playwright + Mocha one in the above guide’s GitHub repo). Importantly, set the test framework to Mocha (JavaScript) on the bottom right.

New Project creation using sample tests. Note selection of test framework as Mocha on bottom right

Now, we want to verify whether the test repository is checked out correctly. So let’s kick off a build first. Click ‘Build Now’.

Empty Project. Click Build Now to start a build (execute a set of tests)

For mine, it ran for a while without doing anything, so cancel the build (I cancelled it at 120 seconds, feel free to cancel it sooner than that).

Click on the project name to edit the configuration (in my case, AgileTravel Build Playwright Mocha).

If a config error states that project sources were not checked out like the image below, then manually do so.

Config error message. Clone your git repo into the given path

In a new command window, go to C:/agileway/tmp/.buildwise/<your project identifier>.

Clone your test repository under the name sources.

In sources, do a quick test execution on command line to ensure everything is working as intended.

Within the path, I cloned the repo and ran the tests successfully on command line :)

My tests are running which is a good sign 😊. Head back to BuildWise, and trigger another build.

Viewing a Build

Again, click ‘Build Now’ on your project.

Your build should start, checkout the latest commit from the repo then begin running the test. You should see some Chrome browsers pop up and run the tests. Here is an in-progress screenshot I took of the build executing:

Playwright Mocha project build in progress

Viewing a Build’s Failures

Here were the results of my build, with one failure.

Playwright Mocha project build completed with 1 failure

I can see that 02_flight_spec.js failed, if you click on the red Failure on the Result column, you can see the error in more detail (and a screenshot). For my build, I can see that it failed due to a timeout error.

Flight Spec error message with option to copy error line to clipboard

From the error output, now you can go into your tests and fix the error (in my case, I can increase the time out or simply rerun the build to see if it always times out). It is very helpful to copy the error line to your clipboard to quickly get to the troublesome area.

BuildWise will automatically pull your latest commit automatically when the build begins, so you can focus on committing your test changes.

I hope this article was able to help you set up a CT server to run Playwright with Mocha E2E automated UI tests. Happy testing 😊



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store