View
227
Download
0
Category
Preview:
Citation preview
K2 Keynote 5/4/16 10:00
Open Source Test Automation: Riding the Second Wave
Presented by:
David Dang
Zenergy Technologies
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073 888-‐-‐-‐268-‐-‐-‐8770 ·∙·∙ 904-‐-‐-‐278-‐-‐-‐0524 -‐ info@techwell.com -‐ http://www.stareast.techwell.com/
David Dang Zenergy Technologies For more than seventeen years, David Dang has been a leader in the test automation industry. As VP of automated solutions for Greensboro, NC-‐based Zenergy Technologies, David spearheads the development of advanced frameworks that emphasize reusability and reduce maintenance efforts. He is an expert in all major commercial automation tools as well as open source tools such as Selenium and Jenkins. On the mobile front, David uses advanced concepts to design optimal frameworks using mobile automation toolsets including Perfecto and Appium. In addition to his high-‐level consulting engagements for Zenergy’s clients, David is in high demand as a presenter at major software quality assurance and testing conferences. Read more about David and Zenergy at zenergytechnologies.com.
Open Source Test Automation: Riding the
Second Wave
StarEast 2016 Presented By: David Dang
A Testing Partner You Can Trust
Overview
What is an open source test automation framework/tool?
Why is it gaining in popularity?
First wave of open source frameworks/tools
Current open source frameworks/tools
Strategies for success
Should you catch the second wave?
Q & A
What is Open Source?
Software for which the original source code is made freely available and may be redistributed and modified
Open source code is meant to be shared within the community. Enhancements, bug fixes, branching of the code are made by the community
Currently on www.opensourcetesting.org/functional.php, there are around 135 open source test automation tools/frameworks
Why is Open Source Test Automation Gaining Popularity?
FREE! Companies have to do more with less
More applications are being converted to web-based
Development teams are pushing for open source test automation
Industry trends include more open source in all areas (Eclipse, Java, Jenkins, OpenStack, Drupel, etc.)
First Wave of Open Source Test Automation Tools/Frameworks
FitNesse: Operates below the user interface level. The user provides various inputs to the application and determines if the correct results are returned
Ruby Watir: Ruby libraries used to automate web browsers. It allows users to automate clicks, fill in forms, press buttons, etc.
Selenium RC: Writes automated web application UI tests. RC comes in two parts: server acts as an HTTP proxy for web, and client libraries interact with the web elements
Why the First Wave Tapered Off
Tools were too fragmented, difficult to integrate into complete solutions.
Level of instrumentation increased the technical challenge and decreased reliability
Learning curve tended to be higher due to overly complex and less refined solutions
IT landscape wasn’t ready to embrace open source at the time
Why the First Wave Tapered Off
FitNesse Does not run at the UI level; runs at the source code level
Fixtures must directly access the source code
Watir Only available for Ruby
Each browser had a different library
Selenium RC: Required heavy instrumentation
Had a large, clunky API
Effectiveness and reliability limited by use of Javascript injection
Current Open Source Test Automation Tools/Frameworks
Selenium WebDriver: Drives a browser directly using its built in support for automation
Cucumber: Runs automated acceptance tests written in a behavior-driven development (BDD) style. It uses Ruby to create step definition
Robot Framework: Generic test automation framework based on keyword-driven testing approach. The core framework is implemented using Python
Why Second Wave Tools are More Popular
WebDriver laid foundation for numerous additional tools
Increased support from well-known organizations WebDriver
Developed by Thought Works
Microsoft is developing a driver for their new Edge browser
W3C has adopted it as the standard for browser automation
Appium – Sauce Labs
Supports popular methodologies (Ex. Cucumber BDD)
Cleaner abstractions and more concise APIs
Less instrumentation required
Better integration
Selenium RC vs Selenium WebDriver: Architecture
Selenium RC Server
Test Code (e.g. Java, C#, Python,
etc.)
Web Browser
Selenium RC
Selenium Core
Application Client
Application Server
Browser Launcher
HTTP Proxy
1
2
3
4
5
Selenium WebDriver
Selenium WebDriver
Test Code (e.g. Java, C#, Python,
etc.)
Web Browser (IE, Firefox,
Chrome, etc.) Application Server
WebDriver: The Foundation of the Second Wave
As the industry standard tool for
browser automation, countless tools are being created that extend or
integrate with Selenium WebDriver
Tools like Appium and Protractor
extend WebDriver for use in other technologies
Tools like Cucumber integrate with WebDriver so it can be used with
methodologies such as BDD
Application Under Test
Selenium WebDriver
Selenium vs Packaged Tools: Feature Comparison
Feature Packaged Tools Selenium
Environment Integrated Development IDE (Eclipse, IntelliJ, RubyMine)
Browser Compatibility Limited (e.g., Firefox, IE, Chrome)
Firefox, IE, Chrome, Safari, Opera, Edge, HtmlUnit, PhantomJS, and others
Language support Limited (e.g., VBScript only or .NET only)
Java, Javascript, C#, Python, Ruby, and others
Reporting Included Separate package (ReportNG, Allure) or custom
Test Management Integrated Custom
Selenium vs Packaged Tools: Feature Comparison
Feature Packaged Tools Selenium
Continuous Integration Integrated Separate package (Jenkins, Bamboo)
Support 800 number The Community…
Object Management Included Separate package or custom
Test Control Included Separate package (TestNG, phpUnit, RSpec)
Strategies for Success
People
Requires more technical resources
Resources must be comfortable with object-oriented programming
Resources must understand framework/tool configuration and environment setup
Resources must be able to diagnose and fix technical issues without vendor support
May have to identify and even fix bugs in the tool
May need to contribute to the community with new features
May need to work more closely with development team
Strategies for Success
Technology
Open source requires more time and effort to get started and maintain
No official support, only the community
Components are not built-in; they must use other libraries or be developed from scratch
Some technologies are not well supported by open source tools
Some technologies have features that lack support from otherwise useful open source tools
Strategies for Success
Process
Requires more dedicated resources
Tool should fit into your existing process
For e.g., if you are a .NET shop, you will often find less of a learning curve with a tool that offers .NET as an option
A tool like Cucumber will have a greater return if it is adopted early in the SDLC
Requires a detailed plan for implementation and maintenance
Framework will be required including standards and page object models
Framework Development Process
Gather Information Architecture
Design
Core Framework
Implementation
Ongoing
Development
Ongoing Maintenance
Should You Catch the Second Wave?
People Is your team comfortable with object-oriented programming?
Are they comfortable dealing with technical challenges with minimal support?
Are they able to contribute to the tool with bug fixes and enhancements?
Do you have a plan in place for training and mentoring?
Should You Catch the Second Wave?
Process Do you have the resources to dedicate toward ensuring
automation is successful?
Do you have a plan for creating a stable, maintainable framework?
Does open source make sense from a budget standpoint?
Is open source a good fit for your overall IT strategy?
Should you Catch the Second Wave?
Technology Have you chosen a tool that is appropriate for your platform?
How active is the community? Is there somewhere to turn for help?
Is the tool updated frequently and responsive to platform changes?
Does investing in open source fit into your organization’s technology road map?
Are you comfortable supporting multiple tools sets?
Summary
WebDriver is the foundation of the current wave of open source automation
People, process and technology are the keys to success Moving to open source is not a decision to take lightly
Recommended