Case Study: Broken Link Checker

How to check a website for broken links using Selenium WebDriver

While Selenium WebDriver is mainly used for web test automation, don’t forget Selenium is an automation framework, which can be used for other automation tasks, not limited in testing. In this case study, I will show a simple way to scrape a website’s links and check if they are broken, using Selenium WebDriver with C#.

This article will take this a tiny step further and specifically check for broken links with a little bit of programming know-how.

Note: This will be a simple unoptimised script just for demonstration purposes.

Test Design

1. Create a list of URLs to check (to_check_list), starting with a site’s home URL.
2. Retrieve one URL from the to_check_list
3. Check whether that URL is broken.
4. If it is a valid URL, visit the URL using Selenium WebDriver
5. Get a list of hyperlinks from the new page, and add them to the to_check_list (unless it is already there)
6. Repeat Steps 2–5, until all URLs in the to_check_list have been checked.
7. Print out all the broken links.