Embed Size (px)
Citation preview
How To Use Selenium, Successfully
by Dave Haeffner, @TourDeDave
http://www.wpclipart.com/geography/features/chasm.png.html
http://en.wikipedia.org/wiki/Optimal_solutions_for_Rubik's_Cube
Write business valuable tests that are reusable, maintainable and resilient across all relevant browsers.
Then package and scale them for you & your team.
Selenium Overview
• What it is — the Reader’s Digest version
• What it is and is not good at
• IDE vs. Local vs. Remote
• Slow, brittle, and hard to maintain?
Step 1 Define a Test Strategy
Test Strategy1. How does your business make money?
2. What features of your application are being used?
3. What browsers are your users using?
4. What things have broken in the app before?
Outcome: What to test and which browsers to care about
Step 2 Pick a Programming
Language
Programming Language
• Same language as the app?
• Who will own it?
• Build a framework or use an existing one?
• http://bit.ly/seleniumframeworks
Step 3 Use Selenium fundamentals
Selenium Fundamentals
• Mimics human action
• Uses a few common actions
• Works with “locators”
Locators tell Selenium which HTML element to interact with
Common Actions
• get();
• findElement();
• click(); //or submit();
• sendKeys();
• isDisplayed();
Locator Strategies• Class
• CSS selectors
• ID
• Link Text
• Partial Link Text
• Tag Name
• XPath
Good locators are: • unique • descriptive • unlikely to change
That rules a few of these out
Locator Strategies• Class
• CSS selectors
• ID
• Link Text
• Partial Link Text
• Tag Name
• XPath
Good locators are: • unique • descriptive • unlikely to change
That rules a few of these out
Locator Strategies• Class
• CSS selectors
• ID
• Link Text
• Partial Link Text
• Tag Name
• XPath
Good locators are: • unique • descriptive • unlikely to change
That rules a few of these out
Start with IDs and Classes
Locator Strategies• Class
• CSS selectors
• ID
• Link Text
• Partial Link Text
• Tag Name
• XPath
Good locators are: • unique • descriptive • unlikely to change
That rules a few of these out
Start with IDs and Classes
Use CSS or XPath (with care)
Locator Strategies• Class
• CSS selectors
• ID
• Link Text
• Partial Link Text
• Tag Name
• XPath
CSS vs XPath http://bit.ly/seleniumbenchmarks http://bit.ly/cssxpathexamples
Finding Quality Locators• Inspect the page
• Verify your selection
• e.g., FirePath or FireFinder
• http://bit.ly/verifyinglocators
• Learn through gaming
• http://bit.ly/locatorgame
• Conversation
Step 4 Write your first test
Good Test Anatomy
• Write for BDD or xUnit test framework
• Test one thing (atomic)
• Each test can be run independently (autonomous)
• Anyone can understand what it is doing
• Group similar tests together
A Login Example
1. Visit the login form
2. Find the login form’s username field and input text
3. Find the login form’s password field and input text
4. Find the submit button and click it
1. or, find the form and submit it
Now to find an assertion
1. Login
2. Inspect the page
3. Find a locator
4. Verify it
5. Add it to the test
Exception Handling• org.openqa.selenium.NoSuchElementException:
Unable to locate element: {"method":"css selector","selector":".flash.error"}
• Most common ones you’ll run into: NoSuchElement and StaleElementReferenceError
• A list of all WebDriver exceptions: http://bit.ly/se-exceptions-java
Step 5 Write reusable and
maintainable test code
Page Objects
Application Under Test
Test 1 Test 2 Test 3 Test 4 Test 5Test 1 Test 2 Test 3 Test 4 Test 5
Need to update EVERY test :-(
Application Under Test
Page Object(s)
Test 1 Test 2 Test 3 Test 4 Test 5
Need to update JUST the page object :-D
Let’s look at a page object for login
And here’s what the test looks like when using it
Page object helpers: http://bit.ly/po-html-elements http://bit.ly/po-page-factory
Base Page Object
Selenium Commands
Page Object 1
Page Object 2
Page Object 3
Page Object 4
Page Object 5
Base Page Object
Page Object 1
Page Object 2
Page Object 3
Page Object 4
Page Object 5
Selenium Commands
• Global reuse • More readable • Insulates you from
Selenium API changes http://bit.ly/se-upgrade
Let’s take a look at a Base Page Object
And here it is implemented
How everything fits together
Test TestTest
Page Object
Page Object
Base Page
Object
Tests use page objects
Page objects inherits the base page object
The base page object wraps your Selenium commands
Step 6 Make your tests resilient
Waiting
Thread.sleep(); Implicit wait Explicit waits
Thread.sleep(); Implicit wait Explicit waits
Explicit Waits
• Specify an amount of time, and an action
• Selenium will try repeatedly until either:
• The action is completed, or
• The amount of time specified has been reached (and throw a timeout exception)
In the Base page object
In the DynamicLoading page object
Browser Timing Considerations
Step 7 Prep for use
Test Harness• Simple organizational structure
• Central setup and teardown
• Configurable at run-time (with sensible defaults)
• Reporting & Logging
• Parallelization
• Test Grouping
Folder structure
Simple config with defaults
Reporting & Logging
• Machine readablee.g., JUnit XML
• Human readable e.g., screenshots, failure message, stack trace
Fantastic Test Report Tool http://bit.ly/se-reporter (Allure Framework)
Parallelization• In code
• Through your test runner
• Through your Continuous Integration (CI) server
#protip Enforce random order execution of tests http://bit.ly/junit-random-order
Recommended approach: http://bit.ly/mvn-surefire
Test Grouping• Metadata (a.k.a. Categories)
• Enables “test packs”
• Some category ideas
• wip
• shallow
• deep
• story number
More info: bit.ly/junit-categories
Step 8 Add in cross-browser
execution
Locallyhttp://bit.ly/se-chromedriver http://bit.ly/se-firefoxdriver http://bit.ly/se-iedriver http://bit.ly/se-operadriver (12.16) http://bit.ly/se-safaridriver
Chrome
Grid
Grid Hub
Browser
Tests
All done with the Selenium Standalone Server Just requires additional runtime flags
Grid Node
Grid Node
Grid Node
Browser
Browser
GridHub
Node(s)
Grid
More on Selenium Grid http://bit.ly/se-grid-docs http://bit.ly/se-grid-post http://bit.ly/se-grid-extras http://bit.ly/se-grid-scaler
Sauce Labs
Sauce Labs BrowserTests
Sauce Labs
Additional Considerations - Test name - Pass/Fail status - Secure tunnel
More on Sauce: http://bit.ly/sauce-platforms http://bit.ly/sauce-post http://bit.ly/sauce-tutorial-java
Step 9 Build an automated
feedback loop
Feedback loops• The goal: Find failures early and often
• Done with continuous integration and notifications
• Notifications e.g., remote: Email, chat, SMSin-person: audio/visual, public shaming
Code Committed
Unit/Integ. (pass?)
Deploy to autom. test
server (success?)
Run automated
tests (pass?)
Deploy to next env.
yes
yes
yes
Notify team if no
Code Promotion
Bonus points: stop the line
Simple CI configuration1. Create a Job
2. Pull In Your Test Code
3. Set up Build Triggers
4. Configure Build steps
5. Configure Test Reports
6. Set up Notifications
7. Run Tests & View The Results
8. High-five your neighbor
Step 10 Find information on
your own
http://bit.ly/se-info-slides
http://bit.ly/se-info-video
Steps to solve the puzzle1. Define a Test Strategy
2. Pick a programming language
3. Use Selenium Fundamentals
4. Write Your First Test
5. Write re-usable and maintainable test code
6. Make your tests resilient
7. Package your tests into a framework
8. Add in cross-browser execution
9. Build an automated feedback loop
10. Find information on your own
Write business valuable tests that are reusable, maintainable and resilient across all relevant browsers.
Then package them and scale them for you & your team.
–Dave Haeffner
“You may think your puzzle is unique. But really, everyone is
trying to solve the same puzzle. Yours is just configured
differently — and it’s solvable”
https://seleniumguidebook.com/
Selenium Tips & Tricks
by Dave Haeffner @TourDeDave
etc.
Headless w/ XvfbTip 38
Headless w/ GhostDriverTip 38
Step 2 - Start PhantomJS w/ WebDriver flag
Step 1 - Download PhantomJShttp://phantomjs.org/download.html
Step 3 - Connect Your Test to PhantomJS using Selenium Remote
NOTE: You can also connect PhantomJS to a Selenium Grid
HTTP Status CodesTip 17
Configuration
• Use a proxy server to capture the traffic from your Selenium test(s)
• Find the status code for the action you’re interested in (e.g., visiting a URL)
• Assert that the status code is what you expect
Selenium Browser Proxy Server
Application Under Test
BlacklistingTip 66
Configuration
• Use a proxy server to manipulate the traffic from your Selenium test(s)
• Identify third-party resources that are slow to load (which could negatively impact your tests) and blacklist them (e.g., make it so they don’t load)
Load TestingTip 68
Configuration
• Use a proxy server to capture the traffic from your Selenium test(s)
• Convert the HTTP Archive into a JMeter JMX file
• Run the new JMX file with JMeter to enact load on your application (modify as needed)
Code adapted from ruby-jmeter example from flood.io
Broken Image CheckingTip 67
Option 1 - Proxy Server
Option 2 - HTTP Library
Option 3 - JavaScript
Option 1: Proxy Server
Option 2: HTTP Library
Option 3: JavaScript
Forgot PasswordTip 43
Configuration• Use Selenium to trigger the forgot password
workflow (which sends an e-mail to a Gmail account) and keep the session active
• Retrieve the e-mail and it’s contents (e.g., a URL) through the Gmail API
• Launch the URL from the e-mail using the Selenium session
A/B TestingTip 12
Example Explanation
• There are 3 states available in the example on the-internet. Each state has different header text.
• The control has the text ‘A/B Test Control’
• The variation has the text ‘A/B Test Variation 1’
• Outside of the split test, the text says ‘No A/B Test’
Configuration
• You can easily opt-out of A/B tests by forging a cookie or appending a query to the URL
• This way you get a known state of the page which isn’t likely to change without your knowledge
Downloading a FileTips 2, 8, & 15
Two approaches• Configure Selenium to download to local disk, and
delete the file when done
• Use an HTTP library, perform a HEAD request, and check the headers for the correct content type & length.
A HEAD request with an HTTP library is an order of magnitude faster than downloading with Selenium since you’re just checking the headers. And, it can be used in
tandem with Selenium.
With Selenium
With an HTTP library
With an HTTP library (for secure files)
Highlight ElementsTip 65
Growl NotificationsTip 55
Visual Testing
Some Write-ups
http://bit.ly/se-visual-1 Getting Started
http://bit.ly/se-visual-2 False Positives part 1
http://bit.ly/se-visual-3 False Positives part 2
http://bit.ly/se-visual-4 Add Visual Testing To Your Existing Tests
http://bit.ly/se-visual-5 Add Visual Testing To Your BDD Tests
A Robot On Every Desk ™
Tapsterbot
http://bit.ly/tapster-build
http://bitly.com/tapster-buy
Build Your Own
Buy One Fully Assembled
Get in touch
@TourDeDave
DaveHaeffner.com
