Automating Shadow DOM with Selenium WebDriver

How to drive web elements inside a shadow root on a web page

A Shadow DOM is a self-contained web component in a web page. Google uses shadow DOMs extensively (link to their Shadow DOM introduction). By default, a standard automation driver is unable to driver web elements inside a shadow DOM.

The latest Selenium WebDriver v4 supports Shadow Roots (HTML elements for a shadow DOM). In this article I will show a quick and easy way to drive web elements inside a Shadow DOM using Selenium WebDriver.

An Example

The task: Add a new list item to the Editable List of the demo Fiddle site.

A typical Selenium script looks like this:

# add new list element
driver.find_element(:xpath, "//input[@class='add-new-list-item-input']").send_keys("Buy milk and eggs")
driver.find_element(:xpath, "//button[@class='add-new-list-item-input']").click
# verify new element is there
expect(page_text).to include("Buy milk and eggs")

But it won’t work on the demo site. Why? The editable list is inside a Shadow DOM and inaccessible — for now.

Finding the Shadow DOM

Open Inspect Element in Chrome (or equivalent for other browsers).

Looking at the page, there is a stand-out tag: <luigi-wc-2f77632f6c6973742e6a73> (highlighted in the screenshot below).

Underneath it, it has one element #shadow-root (open) and that is how you can identify it. It is a bit like a frame (has a title tag …) ,except Selenium unable to find any elements within it.

Testing in a Shadow DOM

The approach:

1. Find the shadow root element

Go to the shadow root’s parent tag and get it using Javascript.

Note: XPaths do not work in Shadow DOM finding, so instead, use Tag names or CSS Selectors.

For our example, the shadow root tag is in the format of luigi-wc- followed by a hex.

# regex this if you wish
shadow_element_wrapper_tag = "luigi-wc-2f77632f6c6973742e6a73"
elem = driver.find_element(:tag_name, shadow_element_wrapper_tag)
# get the shadow root using Javascript
shadow_root = driver.execute_script("return arguments[0].shadowRoot", elem)

The hex is actually hardcoded on this page. For dynamic ones, we can use Ruby scripting to extract it out (see below).

Use execute_script to retrieve the shadow root, and save it to a variable, shadow_root .

2. Using the shadow root as a driver to interact with elements inside it.

We now use shadow_root the same way as driver for the web elements inside it.

input_elem = shadow_root.find_element(:tag_name, "input")
input_elem.send_keys("Mario here I come!")
shadow_root.find_elements(:tag_name, "button").last.click

To verify the text outside the shadow root, switch back to the driver .

expect(driver.find_element(:tag, "body").text).to include("...")

And that’s how you can access shadow roots for automated tests in Selenium WebDriver.

Complete Test Script

 # retrieve using regex for a dynamic shadow root
 def retrieve_shadow_root
    elem = driver.find_elements(:xpath, "//div[contains(@class, 'wcContainer svelte-')]").first
    puts "ELEM: #{elem}"
    
    elem_html = driver.execute_script("return arguments[0].outerHTML;", elem)
    puts elem_html
    if elem_html =~ /<luigi-wc-([\d\w]+)>/
      shadow_element_wrapper_tag_name = "luigi-wc-" + $1
      elem = driver.find_element(:xpath, "//div[contains(@class, 'wcContainer svelte-')]/#{shadow_element_wrapper_tag_name}")
      @shadow_root = driver.execute_script("return arguments[0].shadowRoot", elem)
    end
 end
shadow_root = retrieve_shadow_root()
input_elem = shadow_root.find_element(:tag_name, "input")    input_elem.send_keys("Mario, here I come")
button_elems = shadow_root.find_elements(:tag_name, "button")   add_button = button_elems.last    
add_button.click

The above works, but won’t be easy to maintain. The refactored (based on Maintainable Automated Test Design) version can be accessed on this GitHub repository (link).

Happy testing!