Testing GraphQL APIs, Part 2: Run frequently in a Continuous Testing Server

Stabilising the tests and setting them up in a Continuous Testing server.

6 min readNov 27, 2022

In Part 1, I created five basic GraphQL API tests in Ruby. They worked but were not good enough as test execution is 100% reliable.

I used specific IDs in test scripts, which might work fine the first time around. However, the situation change with upcoming executions, the ID might be taken (duplicate error for CREATE), or no longer exist (causing READ, UPDATE and DELETE operations to fail). Also, I didn’t clean up after my tests in Part 1.

Furthermore, as a habit, I run automated tests in a Continuous Testing server. To do that, I must stabilize my tests first: can run them successfully multiple times. In this article, I will revise the tests written in Part 1 and set up running them in a CT server, as regression testing.

Table of Contents
· Stabilize Tests
 ∘ 1. LIST all records 
 ∘ 2. READ one record
 ∘ 3. CREATE a new record
 ∘ 4. UPDATE a new record
 ∘ 5. DELETE a new record
· Run all tests in BuildWise CT Server
 ∘ Run all tests locally a couple of times
 ∘ BuildWise Setup

Stabilize Tests

1. LIST all records

No changes from Part 1:

it "List all continents" do
    uri = URI.parse("https://countries.trevorblades.com/")
    request = Net::HTTP::Post.new(uri)
    request.content_type = "application/json"
    request.body = JSON.dump({
      "query" => "{ countries { code name } } ",
    })
    req_options = {
      use_ssl: uri.scheme == "https",
    }
    response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
      http.request(request)
    end
    expect(response.code).to eq("200")
    expect(response.body).to include("Antarctica")
    json_response_data = JSON.parse(response.body)
    expect(json_response_data["data"]["countries"].count).to eq(250)
end

2. READ one record

No changes, maybe. For this case, I was using a read-only end-point and assumed that the EU continent always exists. If you are querying something that changes a lot (e.g. stock inventory), the test can start to fail. We can stabilize this by invoking CREATE and then using READ on the newly created one (see UPDATE or DELETE for more).

it "Get a particular continent" do
    uri = URI.parse("https://countries.trevorblades.com/")
    request = Net::HTTP::Post.new(uri)
    request.content_type = "application/json"
    request.body = JSON.dump({
      "query" => '{ continent(code: "EU") { code name countries { name emoji } } }',
    })
    req_options = {
      use_ssl: uri.scheme == "https",
    }
    response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
      http.request(request)
    end
    expect(response.code).to eq("200")
    expect(response.body).to include("Europe")
    expect(response.body).not_to include("America")
    expect(response.body).to include("Belgium")
end

3. CREATE a new record

If the ID is not provided, the service might assign a unique ID for a newly created record. Luckily, in this service, this is the case.

This means we will never have to worry about the duplicate ID scenario. However, we still need to look at the duplicate field scenario.

For the Signup User API, the email field has a unique constraint

How can we make sure our creation request passes every time? Supply a unique email each time the tests are run. This is very easy to do with Ruby Scripting. So I used a random email address with the Faker gem.

request.body = JSON.dump({
  "query" => "mutation {
    signupUser(data: { 
      name: \"#{Faker::Name.name}\", email: \"#{Faker::Internet.email}\" }) {
      id
    }
  }",
  "variables" => "{}"
})

After verifying a successful creation, it is a good idea to delete this record by invoking DELETE. For the example project I chose, there is no way to delete users, so I won’t — but keep in mind that deleting the new test record is good practice.

Full Test Script

it "Sign up a new user" do
    uri = URI.parse("http://localhost:4000/")
    request = Net::HTTP::Post.new(uri)
    request.content_type = "application/json"
    request.body = JSON.dump({
      "query" => "mutation {
        signupUser(data: { 
          name: \"#{Faker::Name.name}\", email: \"#{Faker::Internet.email}\" }) {
          id
        }
      }",
      "variables" => "{}",
    })
    puts "request body: #{request.body}"
    req_options = {
      use_ssl: uri.scheme == "https",
    }
    response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
      http.request(request)
    end
    expect(response.code).to eq("200")
    puts response.body
end

4. UPDATE a new record

We cannot just update an existing record in an automated test, because the test won’t be valid after the first run. Check out this article, “One Test Automation Scenario Interview Question that Most Candidates Failed”.

Therefore, the test design will be as below:

Create a brand-new record
Update it
Verify the update
Delete it

it "Toggle Published status" do
    # create a new post first
    post_id = create_post()

    # update the newly created post
    uri = URI.parse("http://localhost:4000/")
    request = Net::HTTP::Post.new(uri)
    request.content_type = "application/json"
    request.body = JSON.dump({
      "query" => "mutation TogglePublishPost($togglePublishPostId: Int!) {
        togglePublishPost(id: $togglePublishPostId) {
          id
          published
        }
      }",
      "variables" => "{ \"togglePublishPostId\": #{post_id} }",
    })
    puts "request body: #{request.body}"
    req_options = {
      use_ssl: uri.scheme == "https",
    }
    response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
      http.request(request)
    end
    expect(response.code).to eq("200")
    puts response.body
    expect(response.body).to include("true")

    # delete post
    delete_post(post_id)
end

There are two reusable helper functions I created: create_post() and delete_post(post_id).

5. DELETE a new record

Like UPDATE, we shall not delete an existing record. Create a new one first.

it "Delete a post" do
    post_id = create_post()
    uri = URI.parse("http://localhost:4000/")
    request = Net::HTTP::Post.new(uri)
    request.content_type = "application/json"
    request.body = JSON.dump({
      "query" => "mutation DeletePost($deletePostId: Int!) {
        deletePost(id: $deletePostId) {
        id
        title
      }
    }",
      "variables" => "{ \"deletePostId\": #{post_id} }",
    })
    puts "request body: #{request.body}"
    req_options = {
      use_ssl: uri.scheme == "https",
    }
    response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
      http.request(request)
    end
    expect(response.code).to eq("200")
    # Assertion, read record
    read_response = read_post(post_id)
    expect(read_response.code).to eq("200")
    expect(read_response.body).to include("null")
end

I added another reusable helper function here, read_post(post_id).

Run all tests in BuildWise CT Server

For automated end-to-end tests, if we don’t run them often in a Continuous Testing Server (not CI/CD server), those automated tests will be outdated quickly.

Run all tests locally a couple of times

Before I put the tests in a Continuous Testing Server to run as regression, I will try to run them multiple times locally first.

Once these five tests seem reliable, we can check in (Git repository) and run them on a CT server, such as BuildWise.

BuildWise Setup

You can check out this article: “Set Up a Continuous Testing Server to Run Selenium Tests in Minutes” for setting up a new BuildWise CT server instance. Or, run a BuildWise CT server in Docker.

Once the BuildWise Server is up, it is very easy to create a new API Testing project to run these tests. For any CI/CD/CT, we need to check in code/tests into a source repository, Git mostly. If you are not familiar with Git, check out the 10-Minute Guide to Git Version Control for Testers.

The CT build project setup is very easy, it is exactly the as the Selenium one in the guide above. The basic steps are:

1. Check the tests into a Git repository

2. Update the Rakefile to run your tests
The project template created by TestWise IDE is BuildWise CT ready. I just need to update Rakefile to specify the test file to be included in a sequential build (running tests one by one on the server)