View
1.576
Download
0
Category
Preview:
DESCRIPTION
Presentation given at GR8Conf in Copenhagen, Denmark on May 22nd, 2013
Citation preview
Writing an Extensible Web Testing
Framework ready for the Cloud Presented by Mike Ensor May 22, 2013 – Presented at GR8Conf in Copenhagen Denmark
©2013 Acquity Group, LLC. All rights reserved.
Introductions
2
• What is this talk about?
• Curious about automated browser testing
• Testing a Reference application and/or SaaS
• Expanding Geb’s inheritance model
• Who is Mike!?
• Developing software for 16+ years
• Started working with Groovy and Grails in 2007
• Attended GR8Conf last year
• Acquity Group
• Blending strategy, creative and technology
• Currently a principal architect; Intershop and Demandware platform
• Introduced Geb and Spock to Demandware practice
©2013 Acquity Group, LLC. All rights reserved.
Useful Links
3
Demandware testing framework
• http://mikeensor.bitbucket.org/demandware-testing-framework
Spock feature override extension
• https://github.com/mike-ensor/spock-feature-override-extension
Presentation
• http://www.slideshare.net/MikeEnsor/
Twitter @mikeensor
• http://www.twitter.com/mikeensor
Blog
• http://www.ensor.cc
Web Testing
©2013 Acquity Group, LLC. All rights reserved.
Why Test?
5
Raise Confidence
Safety Net
Prove Functionality
Ensure Quality
Lower Costs
©2013 Acquity Group, LLC. All rights reserved.
Cost Savings
6
$0
$1,000
$2,000
$3,000
$4,000
$5,000
$6,000
Developer Unit Tests IntegrationTests
Environment
Cost per Defect
Avg Cost
Google Study presented at XP Days London 2009 – Mark Streibeck, Google Inc
©2013 Acquity Group, LLC. All rights reserved.
SaaS Access
Acceptance
Functional
Integration
Unit tests
Levels of Testing
7
Com
ple
xity Q
ua
ntity
©2013 Acquity Group, LLC. All rights reserved.
History of Web Testing
Manual Click Testing
•Time consuming
•Error prone
Screen Scrape
•Custom scripts
•Maintenance nightmare
HTTPUnit
•Does not work with JavaScript
•Tests HTML structure only
Canoo WebTest
•Developing in XML*
•Duplicate code
Selenium RC
•Cut-Copy-Paste scripts
•Duplication of code
•Mouse driven
WebDriver
•Very low level framework
•Easy to create brittle tests
Geb
•Abstraction on top of WebDriver
•Designed for code reuse
8
©2013 Acquity Group, LLC. All rights reserved.
What is Geb?
9
• Groovy based abstraction of Selenium Web Driver with
jQuery-like syntax for content selection
• Designed for robust yet simplistic syntax
• Simplified DSL describing Pages and Modules
• Words like
• url
• content
• base
• at
• Created with DRY principal in mind
• Use of highly configurable reusable modules
• Abstraction of WebDriver’s Page Model
©2013 Acquity Group, LLC. All rights reserved.
Geb Core
10
Page Objects
Modules Geb
©2013 Acquity Group, LLC. All rights reserved.
Page Objects
11
• What is a “Page”?
• Object representing a generic type of
webpage on a site
• Domain Specific Language
• url - Appended to configurable ‘baseUrl’
• Assists in navigating to page
• at
• closure used to determine if browser
is ‘at’ the page
• content
• Variables representing page content
with CSS selectors
class HomePage extends Page {
static url = "default/Home-Show"
static at = {
title.trim() == "Welcome"
}
static content = {
pageTitle {
$("header h1.title")
}
}
}
...
<header>
<h1 class=“title”>Main Page Title</h1>
</header> ...
©2013 Acquity Group, LLC. All rights reserved.
Building multiple Pages
12
class HomePage extends Page {
static url = "default/Home-Show"
static at = {
title.trim() == "Welcome"
}
static content = {
companyTitle {
$("header h1.title")
}
footerCopywrite {
$("footer span.copy")
}
}
}
class ProductPage extends Page {
static url = "default/Product-Show"
static at = {
title.trim() == "Products"
}
static content = {
companyTitle {
$("header h1.title")
}
footerCopywrite {
$("footer span.copy")
}
productTitle {
$("#product h2.title")
}
}
}
©2013 Acquity Group, LLC. All rights reserved.
Code Duplication
13
©2013 Acquity Group, LLC. All rights reserved.
Dangers of Code Duplication
14
• Small change in underlying structure = many changes
• Long sections of nearly identical code
• Conceal purpose of code
• File size
• Code Maintenance
©2013 Acquity Group, LLC. All rights reserved.
Modules
15
• What is a ‘Module’?
• Re-usable encapsulation
consisting of content references
• Domain Specific Language
• base
• Optional content closure acting as a
base to start content selection
• content
• Variables representing page content
with CSS selectors, optionally
originating from base closure
class HeaderModule extends Module {
static base = { $("header") }
static content = {
companyTitle {
$("h1.title")
}
login { $("a.login") }
}
}
...
<header>
<h1 class=“title”>Page Title</h1> <a href=“/login”>Login</a>
</header> ...
©2013 Acquity Group, LLC. All rights reserved.
Geb: Pages + Modules
16
class HomePage extends Page {
static url = "default/Home-Show"
static at = { title.trim() == "Welcome" }
static content = {
footer { module FooterModule }
header { module HeaderModule }
}
}
class ProductPage extends Page {
static url = "default/Product-Show"
static at = { title.trim() == "Products" }
static content = {
productTitle { $("#product h2.title") }
footer { module FooterModule }
header { module HeaderModule }
}
}
class HeaderModule extends Module {
static base = { $("header") }
static content = {
companyTitle { $("h1.title") }
login { $("a.login") }
}
}
©2013 Acquity Group, LLC. All rights reserved.
Now what?
17
Not Logged In Logged In
Extending Geb Objects
©2013 Acquity Group, LLC. All rights reserved.
Scenario
Your Mission:
You have been asked to create a special Product page for when a customer has logged in
The page looks and functions exactly like the Product page, but has an extra header
19
©2013 Acquity Group, LLC. All rights reserved.
Extending Page Objects
20
• Extend our page object
• Re-define url, and at
• Inherits content
• Overwrite/add content
elements
class PersonalProductPage extends ProductPage { static url = "default/MyProduct-Show" static at = { title.trim() == "Personalized" } static content = { // header inherited from HomePage // footer inherited from HomePage productTitle { $("section#personal h2.title") } } }
©2013 Acquity Group, LLC. All rights reserved.
• Extend module object
• Re-define base
• Inherits content
• Overwrite/add content
elements
Extending Modules Objects
21
class AuthHeaderModule extends HeaderModule {
static base = { $("header section#personal") }
static content = {
// companyTitle inherited from HeaderModule
// login inherited from HeaderModule
logout { $("a.logout") }
}
}
©2013 Acquity Group, LLC. All rights reserved.
Drawbacks & Best Practices
• Watch out for
• Overriding content segments happen at runtime
• No name spacing
• Possible to unintentionally overwrite content
• Possible solutions
• Document well (use JavaDoc style comments)
• API Groovy Doc
• Write a test to explicitly prove the inheritance
• Tests should fail if the inheritance fails
• Subscribe to user@geb.codehaus.org
22
©2013 Acquity Group, LLC. All rights reserved.
With great power
comes great responsibility -- Uncle Ben
23
Using Geb with Spock
©2013 Acquity Group, LLC. All rights reserved.
Spock
• Specification based testing framework designed to describe
the intention of the test in business friendly terms
• Provides keywords for structure
• when
• Action
• then
• Expectation
• Geb provides GebReportingSpec
• Can be extended to change functionality
• Provides extension model
• Control specification lifecycle
25
©2013 Acquity Group, LLC. All rights reserved.
Creating a Specification
• Extend GebReportingSpec
• Describe a feature
• aka - “Test method”
• Browse to
• Verify at
• Verify content
• Defined in ProductPage
• Verify module content
• Defined in HeaderModule
26
class ProductPageSpec extends GebReportingSpec {
def "user can view the product page"() {
given:
to ProductPage
expect:
at ProductPage
and:
productTitle.text() == "My Cool Product"
and:
header.companyTitle.text() == "Company X"
}
}
Extensible Web Testing
©2013 Acquity Group, LLC. All rights reserved.
Scenario
Your Mission:
You work on a SaaS platform where each implementation is very similar in features and structure.
Design a testing framework where you can leverage common functionality inherent in all
implementations of the SaaS platform.
28
Reference Application Implementation
• Size & Color Filters
• Breadcrumbs
• Category Navigation
• Mini-Cart
• Wish List
• 3-in-row Product Grid
• Size & Color Filters
• Breadcrumbs
• Category Navigation
• Mini-Cart
• Wish List
• 4-in-row Product Grid
©2013 Acquity Group, LLC. All rights reserved.
Common Functionality: Reference Application
29
Category Filter
Color Refinement
Size Refinement
Mini-Cart
Wish List
Breadcrumbs
Pagination
Price Refinement
©2013 Acquity Group, LLC. All rights reserved.
Common Functionality: Implementation Application
30
Category Filter
Price
Refinement
Color
Refinement
Mini-Cart
Wish List
Breadcrumbs
Pagination
©2013 Acquity Group, LLC. All rights reserved.
Inheriting Test Functionality
• Add dependency on
Reference App’s Test JAR
• Extend target specification
• Inherits all features from parent specification
• Foresee problems?
• Overwrite inherited features?
• Use derived Page objects?
• Investigate these one-by-one
31
class HomePgSpec extends GebReportingSpec {
def "user can view the home page"() {
// trimmed for clarity
given:
to HomePage
}
def "user can login"() {
// omitted for simplicity
}
}
class ClientPageSpec extends HomePgSpec {
// Inherit "user can view the home page“ & "user can login"
def "user can open wishlist"() {
// trimmed for clarity
given:
to ClientHomePage
}
}
©2013 Acquity Group, LLC. All rights reserved.
Inheriting Test Functionality: First Run
32
class HomePgSpec extends GebReportingSpec {
def "user can view the home page"() {
given:
to HomePage
when:
header.login.click()
then:
at MyAccountPage
}
def "user can login"() { /*obmitted for clarity*/ }
}
class ClientPageSpec extends HomePgSpec {
def "user can open wishlist"() {
given:
to ClientHomePage
}
}
• Run ClientPageSpec
• Spock determines list of features
• “user can view the home page”
• “user can login”
• “user can open wishlist”
• Run “user can view the home page”
• Navigate “to” HomePage ????
©2013 Acquity Group, LLC. All rights reserved.
Plan: overwrite inherited features
• Look for a feature in the parent
class with the same name
• Replace the inherited feature
with the provided feature
• Utilize Spock Extension features
to modify test execution
33
class ClientPageSpec extends HomePgSpec {
def "user can login"() {
given:
to ClientHomePage
when:
header.login.click()
then:
at ClientMyAccountPage
}
def "user can open wishlist"() {
// omitted for clarity
}
}
©2013 Acquity Group, LLC. All rights reserved.
Spock Extensions
Add runtime functionality
Implements Visitor Pattern • Allow for modifications to the
Specification feature sequence
Ignore inherited features if implemented
in Child
Triggered by Annotation
• @OverrideFeatures
34
©2013 Acquity Group, LLC. All rights reserved.
Overwrite Inherited Features
• Look for a feature in the Parent
class with the same name
• Create a marker @Annotation
• Tell Spock to use this feature, not
the Parent by using a Spock
Extension
• spock-feature-override-extension
• Open Sourced on github
35
@OverrideFeatures
class ClientPageSpec extends HomePgSpec {
def "user can login"() {
given:
to ClientHomePage
when:
header.login.click()
then:
at ClientMyAccountPage
}
def "user can open wishlist"() {
// omitted for clarity
}
}
©2013 Acquity Group, LLC. All rights reserved.
Now what?
• Able to override inherited features
• Override every feature?
• Why inherit?
36
class HomePage extends Page {
static url = "default/Home-Show"
static at = {
title.trim() == "Welcome"
}
static content = {
header { module HeaderModule }
footer { module FooterModule }
}
}
class ClientHomePage extends HomePage {
static url = "default/Home-Show"
static at = {
title.trim() == "Company Home"
}
static content = {
nav { module NavigationModule }
}
}
• What do we want?
• Dynamically swap sub-classed Page
objects
©2013 Acquity Group, LLC. All rights reserved.
Dynamically Swap Page Objects
• Page objects are managed by Browsers in Geb
• Extend existing Browser
• Create a “Browser” that knows about Page object relationships
37
geb.Browser
---------------------------------------------
Browser()
Page at(Class<? extends Page>)
Page to(Class<? extends Page>)
PageSelectingBrowser extends Browser
----------------------------------------------------------
PageSelectingBrowser(Map pageMap)
Page at(Class<? extends Page>)
Page to(Class<? extends Page>)
Page createPage(Class<? extends Page>)
©2013 Acquity Group, LLC. All rights reserved.
PageSelectingBrowser
38
class PageSelectingBrowser extends Browser {
private final Map<Class<? extends Page>, Class<? extends Page>> overridePages
/** Other methods omitted **/
@Override
Page createPage(Class<? extends Page> possibleParent) {
Class<? extends Page> type = possibleParent
if (overridePages.containsKey(possibleParent)) {
type = overridePages.get(possibleParent)
if (!possibleParent.isAssignableFrom(type)) {
throw new IllegalArgumentException("${type} not a ${possibleParent}")
}
}
type
}
}
©2013 Acquity Group, LLC. All rights reserved.
Using PageSelectingBrowser
39
• Browsers are created/used in GebSpec or GebReportingSpec
• Extend GebReportingSpec
• Override createBrowser() and return new PageSelectingBrowser
• Create a default empty Page object Map
• Each specification can overwrite Page object Map
class OverridableGebReportingSpec extends GebReportingSpec {
@Override
Browser createBrowser() {
new PageSelectingBrowser(createConf(), getChildPageMap())
}
Map<Class<? extends Page>, Class<? extends Page>> getChildPageMap() {
[:]
}
}
©2013 Acquity Group, LLC. All rights reserved.
OverridableGebReportingSpec
40
class HomePageSpecification extends OverridableGebReportingSpec {
def "user can view the home page"() {
given:
to HomePage
expect:
at HomePage
}
}
class ClientHomePageSpecification extends HomePageSpecification {
// inherit all features from HomePageSpecification
@Override
Map<Class<? extends Page>, Class<? extends Page>> getChildPageMap() {
[(HomePage.class): ClientHomePage.class]
}
}
Dynamically replace
ClientHomePage for HomePage
©2013 Acquity Group, LLC. All rights reserved.
Final Example
41
class HomePageSpec extends OverridableGebReportingSpec {
def "user can view the home page"() { // omitted for clarity }
def "user can login"() { // omitted for clarity }
}
@OverrideFeatures
class ClientHomePageSpecification extends HomePageSpecification {
def "user can login"() {
when:
to ClientHomePage
header.login.click()
then:
at ClientMyAccountPage
}
def "user can open wishlist"() { /* omitted for clarity */ }
@Override
Map<Class<? extends Page>, Class<? extends Page>> getChildPageMap() {
[(HomePage.class): ClientHomePage.class]
}
}
Inherit Features
Override Feature by Name
Create new features
Dynamically replace Page objects
©2013 Acquity Group, LLC. All rights reserved. 42
Testing in the Cloud
©2013 Acquity Group, LLC. All rights reserved.
Testing on Continuous Integration
44
©2013 Acquity Group, LLC. All rights reserved.
Running without a Browser?
45
• What is RemoteWebDriver?
• Remotely sends driver commands to Selenium Grid
• Allows for same code to run on multiple browsers
©2013 Acquity Group, LLC. All rights reserved.
Sauce Labs
46
• SauceLabs – In-cloud based Remote WebDriver
• Removes infrastructure requirements
• Records video of remote run
• Can operate behind firewalls via SSH tunnel
©2013 Acquity Group, LLC. All rights reserved.
Parallelization
47
• Initial approach
• Modify GebSpec to create multiple browsers
• Cons
• Spock Extensions are not thread safe
• Browsers run at different speeds
• Remote WebDriver runners difficult to determine completion
• Reports not easy to collate
• Solution*
• Poor man’s Parallel
• Kick off multiple builds on CI with different configuration
• Possible parallelization with Gradle
• Fork per JVM
Questions?
Recommended