Lessons Learned:Migrating Tests To Selenium v2

September 20, 2011Roger Hu (rhu@hearsaycorp.com)

Hearsay Social

• Enterprise leader in social media management for regulated B2C organizations

• Enables reps to compliantly capitalize on LinkedIn, Twitter, and Facebook

• Backed by Sequoia, NEA, Steve Chen, FB execs

• Visionary founders and management team from GOOG, SFDC, MSFT, AMZN

“If 2011 is the year of social media for business, Hearsay [Social] may have some say about it.”

Overview of Talk

• Reasons to Switch• Overview of Selenium v2• Issues/Workarounds in Selenium v2 • Q/A

Reasons to Switch

• Selenium v2 more closely approximates the user experience.

• Internet Explorer (IE7/IE8) especially runs faster with Selenium v2 (> 20%?)

• Easier to write tests. WebDriver reflects how other JavaScript frameworks work.

Selenium v2 Approach

Example:driver = webdriver.Firefox()driver.find_element_by_xpath(“//a[text()=‘Create New Ad’”).click()driver.find_element_by_id(“ads_headline”).send_keys(“Selenium Testers Wanted”)

• Selenium server launches without needing proxy-based approach.

• Browser driver generates native keyboard/mouse clicks to more closely simulate user behavior.

• WebDriver API (REST-based) simpler.

WebDriver APIBrowser-specific driver

i.e.: IEDriver.dll (IE), webdriver.xpi (Firefox)

+

Complexities of Moving to Selenium v2

• Your tests have to be rewritten to leverage the WebDriver API. – Record/playback less useful option? (Selenium IDE plug-in for Firefox)– Synchronization issues: implement more implicit/explicit wait conditions

(Selenium v2 sometimes acts like an impatient user) – Hidden/non-visible elements can no longer be clicked.

• Specific WebDriver extension for each browser.• Cross-browser issues with mouse click/keyboard events.• CSS3 selectors now more dependent on the browser (unlike Selenium v1).• Not all WebDriver extensions are fully baked (i.e. Chrome, Android, iPhone)

• Keeping up with the release cycles.– Release Candidates: toggle()/select() removed from API.– We do a lot of pip –U selenium. Make sure you’re using the latest version of

your preferred language binding!– Documentation is sparse, and hard to debug without digging through

Selenium source code (Java, C++, and JavaScript).

Visibility Matters

i.e. driver.find_element_by_id(“cms_toolbar_icon”).click()• The X/Y coordinates of the element location are used to determine where to click.

– Hidden elements can no longer be acted upon until they are visible.– The element gets focused and clicked. Some quirky issues with elements at

top or bottom of the screen exist.– CSS z-index takes precedence now; the top-most element will be the one that

receives the mouse event. • If you use any type of debug overlay tool (i.e Django Debug Toolbar, Rails

Footnotes, etc.) disable them before running your Selenium tests!

• To set cookies in IE, Protected Mode must be disabled for all zones. IE8 doesn’t appear to throw any exceptions if cookies can’t be set.

Using Cookies• In the Django Debug Toolbar case, we

can set a cookie called ‘djdt’ to disable the overlay.

• You have to open a connection to the page before you can set the cookie. Exceptions usually get thrown otherwise.

• Contrary to the documentation, setting cookies actually requires passing in a dictionary of name, value, and secure parameters.

driver.add_cookie({"name" : "djdt", "value" : "true", "path" : "/", "secure" : False})

(name, value, secure parameters required)

Implicit vs. Explicit Waits

Implicit Waits vs. Explicit• Implicit waits just need to be set once

during setup (server-side polling)

• Use implicit waits for most page loads, assuming you have a unique DOM element present.

• Implicit waits complete once element present in DOM (click and other mouse events require visible elements).

Explicit Waits• Use explicit waits for all other cases (i.e.

popup dialogs that are already rendered in the DOM but aren’t shown, waiting for a DOM element to disappear, etc.):

• Use WebDriverWait’s anytime you need to avoid JS/database race conditions (client-side polling)

• Selenium v2.6.0+: ExpectedConditions class to handle often-used cases (clickable elements, text present, etc.)

driver.find_element_by_id(“myelement”)driver.find_element_by_css_selector(“#myelement”)driver.find_element_by_xpath(“//div[@id=‘myelement’]”)

WebDriverWait(driver, timeout=30).until(lambda x: driver.execute_script(“return jQuery.active === 0”) (wait for Ajax calls to complete)

Remember wait_for_page_load() and wait_for_condition() in Selenium v1?

from selenium.webdriver.support.ui import WebDriverWait

WebDriverWait(driver, timeout=30).until(lambda driver: driver.find_element_by_css_selector("div.popup.content ul.tabs").is_displayed))

driver.implicitly_wait(30) (sets timeout for finding elements)

Demo

• So how does this implicit/explicit wait stuff work?• Let’s play Hearsay Social Says...http://hearsaysocialsays.appspot.com

(Selenium v2 source code posted on the site)def play_level(): driver.find_element_by_name("b").click()

WebDriverWait(driver, timeout=10).until(lambda driver: driver.find_element_by_name("s").get_attribute("value") == "Player's Turn") pcclicks = driver.execute_script("return pcclicks;")

for pcclick in pcclicks[1:]: ele = "pl%s" % pcclick driver.find_element_by_name(ele).click()

driver.find_element_by_name("b").click()

# Alert box shows up. Let's dismiss it. driver.switch_to_alert() Alert(driver).accept()

for i in xrange(20): play_level()

Native vs. Synthetic Events• Selenium v1 relied entirely on generating events through JavaScript, while Selenium v2

tries to do it through the operating system.

Selecting a dropdown:

driver.find_element_by_xpath("//select[@id=‘myselect']/option[text()=‘Seattle']").select() (deprecated)driver.find_elements_by_css_selector("#myselect option")[2].click() # IE7/IE8 won’t click (JS issue)

from selenium.webdriver.common.keys import Keysdriver.find_element_by_css_selector("#myselect").send_keys(Keys.DOWN) (IE7/IE8 have different behavior for disabled elements)

One workaround:driver.execute_script(“$(‘#myselect option’][2].change()’);”) (use jQuery to trigger select-box changes)You can always bypass some cross-browser issues related to native events by reverting to JavaScript-based events, though it’s not ideal.

<select id=“myselect”> <option value="1">Seattle</option> <option value="2“ disabled=disabled>Boston</option> <option value=”3”>San Francisco</option> <option value=“4">Washington D.C.</option> </select>

Hover states, drag-and-drop, motion-based gestures, HTML5….

• Want more complex sequences of keyboard/mouse actions (i.e. right clicks, double clicks, drag/drop)? Use the ActionChains class.– Creating hover states: also done

through ActionChains class.

– Remember, the elements still have to be visible for them to be chained!

– Many advanced user interactions still somewhat experimental, so expect issues.

from selenium.webdriver import ActionChains

driver.get(‘http://www.espn.com’)menu_mlb = driver.find_element_by_id("menu-mlb")chain = ActionChains(driver)chain.move_to_element(menu_mlb).perform()

scores = driver.find_elements_by_css_selector("#menu-mlb div.mod-content ul li")[1] scores.click()

chain.move_to_element(menu_mlb).click(scores).perform() (scores is not visible until MLB is selected)

• Earlier blog posts about Selenium encourage using CSS selectors instead of XPath: div:contains(“Click here”) vs. //div[text()=“Click here”)

•Matching by inner text? You still may need to use XPath.•CSS selectors supported in Selenium v1 are not standard (i.e. contains())

– Selenium v2 relies on document.querySelector() for modern browsers (IE8+, Firefox 3.5+).

– If document.querySelector() isn’t supported, the Sizzle CSS engine is used.– Even if you’re using a CSS3 selector, check whether it’s supported in IE.

driver.find_element_by_css_selector(“#ad-summary:last-child") (breaks in IE7)driver.find_element_by_css_selector(“#ad-summary.last") (workaround: explicitly define the last element when rendering the HTML)

CSS Selectors in Selenium v2

Other Tips & Tricks…• Use Firebug with Selenium v2.

– Create a profile and installing the extension:firefox -ProfileManager --no-remote (launch profile manager and install Firebug extension)

– Specify profile directory:profile =

FirefoxProfile(profile_directory=“/home/user/.mozilla/firefox/60f7of9x.selenium”)driver = webdriver.Firefox(firefox_profile=profile)

• Test on local browsers first, use SSH reverse tunneling when testing on remote dev servers:– Install Selenium server on host machine:

• java –jar selenium-server.jar (default port 4444)– Create a SSH tunnel into remote machine:

• ssh -nNT -R 9999:<IP address of server>:4444 username@myhost.com (clients connect to port 9999)

– Login to remote machine and connect via webdriver.Remote()• driver = webdriver.Remote(command_executor=http://localhost:9999/wd/hub)

• Using SauceLabs? – Make sure your max-duration: option is set if your test runs exceed 30 mins. – Can’t keep up with the Selenium v2 release cycles? Fix your version with selenium-version:

option.– Use exit signals handlers to issue driver.quit() commands to see your test results sooner.

Summary

• Selenium v2 gets closer to simulating the user experience with an entirely new architecture (WebDriver API + browser plug-in/extension).

• Your tests may start failing on clicking on hidden/non-visible elements, so disable your debug overlays and be aware of CSS z-index issues.

• Bugs/issues with generating events still persist, and you may encounter many browser quirks (i.e. dropdown boxes).

• WebDriver may run faster, though you may need to add more synchronization checkpoints with implicit/explicit waits.

• When using CSS selectors, use the ones supported across all browsers (especially IE).

Q/A

Questions/Comments?rhu@hearsaycorp.com