Use Advanced User Interactions in Selenium WebDriver to drive keyboard and mouse

How to use Selenium WebDriver’s ActionBuilder for automated testing

  • click
  • click_and_hold
  • context_click
  • double_click
  • drag_and_drop
  • drag_and_drop_by
  • move_by
  • move_to
  • release
  • key_down
  • key_up
  • send_keys

The usage

driver.action. + one or more above operations + .perform

Mouse Over

When a mouse is moved over the email field below,

<input id="email" name="email" type="email" style="height:30px; width: 280px;" data-toggle="tooltip" data-placement="right" title="must contains @"> + "/html5.html")
elem = driver.find_element(:id, "email")
sleep 1
expect(page_text).to include("must contains")
driver.action.move_to_location(0, 0).perform

Double Click

When you double click the text “Quick Fill”,.

<input type="password" name="password" id="pass">
<span id="quickfill" ondblclick="quick_fill()">Quick Fill(double click)</span>
function quick_fill() {
</script> + "/text_field.html")
quick_fill_elem = driver.find_element(:id, "quickfill")
# double click to fill
sleep 0.2
expect(driver.find_element(:id, "pass")["value"]).to eq("ABC")

Click and Hold

Select the item boxes 6 to 8 in the following grid.

Screenshot selecting item boxes 6, 7 and 8 in one mouse hold. Screenshot from website.
<ol id="selectable" class="ui-selectable">
<li class="ui-state-default ui-selectee">1</li>
<li class="ui-state-default ui-selectee">2</li>
<li class="ui-state-default ui-selectee">3</li>
<li class="ui-state-default ui-selectee">12</li>
sleep 1
driver.find_element(:link_text, "Display as grid").click
sleep 1
list_items = driver.find_elements(:xpath, "//ol[@id='selectable']/li")
sleep 0.5

Drag and Drop

Drag Item 1to the Trash block.

<div id="trash" class="ui-droppable over">
<div id="items">
<div class="item ui-draggable ui-draggable-handle" id="item_1" style="position: relative;">
<span>Item 1</span>
<div class="item ui-draggable ui-draggable-handle" id="item_2" style="position: relative;">
<span>Item 2</span>
<div class="item ui-draggable ui-draggable-handle" id="item_3" style="position: relative;">
<span>Item 3</span>
</div> + "/drag_n_drop.html")
drag_from = driver.find_element(:id, "item_1")
target = driver.find_element(:id, "trash")
driver.action.drag_and_drop(drag_from, target).perform


Right-clicking will bring up the context menu, where additional actions can be performed.

<input type="password" name="password" id="pass">
elem = driver.find_element(:id, "pass")driver.action.context_click(elem).perform

Move Mouse by Offset

You can move the mouse to a specific coordinate relative to a web element by providing an offset coordinate.

<img src="images/agileway_software.png" border="0" width="400" id="agileway_software" usemap="#agileway_software_map">
<map name="agileway_software_map" id="agileway_software_map">
<area shape="rect" coords="13,16,120,42" href="" alt="testwise" title="">
<area shape="rect" coords="13,73,127,100" href="" alt="buildwise" title="">
<area shape="circle" coords="220,30,27" href="" alt="whenwise">
elem = driver.find_element(:id, "agileway_software")
driver.action.move_to(elem, 210, 30).click.perform
expect(driver.title).to eq("WhenWise - Booking Made Easy")

Send Key Sequences

Select all the text in a text area and delete it.

<textarea id="comments" name="comments"></textarea>
driver.find_element(:id, "comments").send_keys("Multi\r\n Line\r\n Comment")
elem = driver.find_element(:id, "comments")
# use Command key for macOS, Control key otherwise
ctrl_key = RUBY_PLATFORM.include?("darwin") ? :command : :control
elem.send_keys([ctrl_key, "a"])



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store