Test Automation with Twist and Sahi

May 2014

Mattersight Corporation

1

Eric Blackburn, Chris Stanush, Peter Zejda

Agenda

2

• Why• Sahi• Sahi vs Selenium• Twist• Design Suggestions• Demo Video• Questions

Save Questions Till The End

3

• We will stick around

Agenda

4

• Why• Sahi• Sahi vs Selenium• Twist• Design Suggestions• Demo Video• Questions

Why

5

• Functional Test Automation – hard

• GUI Functional Test Automation - especially hard− DOM changes occur often− Complex HTML controls − Support multiple browsers− Record and playback is fragile – doesn’t support change− Need for separating the “What” from the “How”

• Intent (the “What”) doesn’t change as often as the testing method (the “How”)

• Business side needs to write/maintain scenarios, not developers

• Automated testing needs to be faster than manual

• Testing tools can be expensive

• Need ability to “check in” / version Test Cases with code

• Need a full language and enterprise software architecture / design to create an effective test framework (and make part of ConDel)

Agenda

6

• Why• Sahi• Sahi vs Selenium• Twist• Design Suggestions• Demo Video• Questions

7

What is Sahi

• Sahi is the proxy between Java and the Browser− Java -> Sahi -> Javascript -> Browser− Java <- Sahi <- Javascript <- Browser

• Learn more about Sahi: http://sahi.co.in/

Agenda

8

• Why• Sahi• Sahi vs Selenium• Twist• Design Suggestions• Demo Video• Questions

9

Sahi vs Selenium

• Selenium− Positives

• Run JavaScript directly against the browser• Widely used• Different flavors

− Selenium 2.0− Geb− Helium

− Negatives• Lack of auto-waits for Ajax calls

− http://www.thoughtworks.com/products/docs/twist/current/help/how_do_i_handle_ajax_in_selenium2.html

• Wait for driver updates• For built in functions, different experiences with different drivers.

• Sahi− Positives

• Run JavaScript directly against the browser• Automatically waits for page to load and for any Ajax calls to finish• For built in functions, same experience for all browsers.• Supports concurrent testing (batch runs)

− Negatives• Sometimes proxy doesn’t initiate properly.

Agenda

10

• Why• Sahi• Sahi vs Selenium• Twist• Design Suggestions• Demo Video• Questions

11

Twist http://www.thoughtworks.com/products/twist-agile-testing

• BDD interface built into Eclipse

• Cost − $100 a year per developer

• Free updates

− Able to export libraries and run on test harness systems/servers for free, as many test harness servers as desired.

12

Twist - Behavior-driven Development BDD [1]

• Domain Specific Language (DSL)• Business can write tests

− Non-formal English− Specify intent− Abstracted step from how it is implemented

• Code changes, but the sentence (intent) remains the same.

− Prior to code being implemented• BDT

[1] http://en.wikipedia.org/wiki/Behavior-driven_development

13

Twist

• GUI and Non-GUI functional testing• Contexts

− Setup and Teardown steps always run, when added to a scenario

• Tags− Used to organize scenarios− “Not IE7”, “Disabled”, “Active”

• Steps from multiple Classes• Comments• Manual steps• Table driven steps• No browser focus needed• Auto snapshots for failures• Auto generated HTML result reports• Jenkins slave runs tests via ANT or Maven

− Windows needs VNC service running (browser needs GUI rendering). Real VNC free edition works fine.

− Reportedly Linux can run headless

14

Twist

• Table driven steps−Can fail sections of the table using Asserts

• Run results as HTML Report• More info about how to use various features is available at

− http://www.thoughtworks.com/products/docs/twist/current/help/whats_new.html

Agenda

15

• Why• Sahi• Sahi vs Selenium• Twist• Design Suggestions• Demo Video• Questions

16

Design Suggestions – Browser Adapter

• Problem− Dependency on the web interaction

tool (Sahi)

• Solution− Abstract browser interaction

• Portability• Identify elements via JavaScript

chaining (pass around strings)

− Browser Adapter class • All of code inherits added or changed

classes • Doesn’t help when a class is removed

− Create own API• Not a big deal

− 15 methods for Action− 7 methods for State

• Better API than original

17

Design Suggestions – Control Object Pattern

• Problem− Many web controls

• Each with specialized find, set and verify functionality

− Control interfaces can be very complicated

− Control interfaces do change

• Solution− Follow the “Information Expert”

design pattern[1] • Encapsulate logic for

− Find and Set methods − Verify state of control

[1] http://en.wikipedia.org/wiki/GRASP_(object-oriented_design)#Information_Expert

18

Design Suggestions – Pre-build JavaScript/jQuery

• Problem− Calls to the browser are

slow

• Solution− Move logic into JavaScript

• Browser is able to process complex logic fast

• Sometimes the algorithm needs to be next to the data

19

Design Suggestions – Jsoup http://jsoup.org/

• Problem− Need for complex inspection of DOM and calls to the browser are

slow.

• Solution− Use Jsoup to cache the DOM

• Return DOM content as a String and parse directly in code• Complex processing of the DOM is fast • Syntax is easy to use

20

Design Suggestions – Find Elements

• Problem− Identify web content based on what the scenario writer can see

• Solution− public String findNthVisibleElement(FieldTypeEnum fieldType, String

parentSelectorValue, String elementIdentifier, int index)

• A central function that is able to find elements. Elements can be identified by their text content, id, name, aria-label, etc.

− Identify an element by passing around a String that can be used to scope up and down• Use JavaScript or jQuery syntax

21

Design Suggestions – Find Elements – Helper Methods

• List out what controls can exist as

• List out what identifies are used for each HTML element type

22

Design Suggestions – Find Elements – Helper Methods

• Build comma delimited selector using the identifier and the list of element types− e.g. “#userinput, .userinput, [aria-label=‘userinput’]”

23

Design Suggestions – Find Elements – Helper Methods

• ContainsExact(Case) – Find element that only contains• Problem - :Contains(‘some of text’) doesn’t work

• Matches both− <span>some of text</span>− <span>text and some of text and more text</span>

• Solution - :ContainsExact(Case)(‘some of text’) works• Matches only

− <span>some of text</span>

Agenda

24

• Why• Sahi• Sahi vs Selenium• Twist• Design Suggestions• Demo Video• Questions

• Concerns, Ideas, Requests, Champagne?

Questions?