StickyMinds.com and Better Software magazine presents…
Avoid Throwaway Test Automation Sponsored by Cognizant
Non-streaming participants should call 1-866-761-8643International Non-streaming participants should call 1-904-596-2362
Setting the Context
What we mean by “automated testing” Other types of tool-assisted testing Principles will apply to other types Many topics deserve more attention Automated testing is software
development
What Typically Goes Wrong
Create automated tests that don’t run anymore
Spending too much time babysitting and maintaining the automation
Automated tests are too brittle Tools don’t work in environment Automated tests not providing value
Common Mistakes
No plan for implementation No buy-in from staff or management No training for automators No time allotted to automate No time allotted for maintenance No framework for reusability Good intentions, poor execution
Why Automate?
Sounds cool Boss said so Can’t keep up Lots of repetitive tests Lots of data driven tests Reduce time spent on regression testing
Develop an Automation Plan
Why you will automate What to automate When to automate Who will automate, execute, maintain How to automate – Framework How to report results Where to run tests
What to automate
Smoke tests Repetitive tests Can run autonomously Big risks Take less time to automate than execute
Should run 3-5 times without changing Data intensive tests
When to automate
When you’re ready Depends on development lifecycle
Waterfall – may wait until the end Agile – may need continuous automation
Not too early, not too late Need stable-ish UI Before you have to regression test
Plan time for automation and maintenance Manage as part of regression testing time
Selecting the Right Tool
Define your tool requirementsWhat you need it to doCompatibility with your applicationCompatibility with your skillsets
Try it out Beware of the hype
“Record and Playback” rarely is that simple
Open Source Tools
Free to acquire, not to use More time required for implementation
Installation and configurationLearning to use the product
More technical skills required http://opensourcetesting.org
Develop a Framework
Organization of artifacts Aim for reusability
Across features, product versions, and products Separate interface from functionality
Dealing with common activities Object recognition Navigation Data validation
Dealing with Change
Plan for changes in UI How to respond to test failures Flexible object recognition
Make tests data independent Reduce dependencies between tests Set up test data in cleanup scripts Script tests to use dynamic data
Enlist help of developers to ease automation
© 2008, Cognizant Technology Solutions. All Rights Reserved. The information contained herein is subject to change without notice.
Automation Assessment
Testing ServicesTesting Services
14
ObjectiveObjective
BackgroundObjective
• Automation Assessment Approach •Process•Infrastructure•Tools•Framework•Operating Model•Best Practices
15
Assessment Scope
1515
Infrastructure
Pro
cess
Fra
mew
ork
Best
Pra
ctices
Tools
OperatingModel
Automation Prioritization Planning & Strategy Approach Estimation Model Documents Guidelines Review procedures & Checklists Metrics Collection Configuration management Audits/Assessments Change Management Environment management Process Defect management Maintenance Approach
Scripting Standards User Guides Maintenance Process Handbook Dynamic Script Allocation Automation Review Tool KR portal Integration of automation
scripts to test management tools
Environment Management Test bed creation and maintenance
Architecture & Type Test Data Management Reporting Mechanism Error & Exception
Handling Folder Structure Scalability Reusability Function Library Object Repository Database Testing Batch Execution
Functional Automation Test Management Configuration
Management Defect Management License Management
Roles & Responsibility Governance Model Organizational SLAs Project structuring Communication
FocusAreas
Assessment Scope
16
Process
Communication and Collaboration – (BA’s, Developer’s, Manual Tester’s, etc)
Identification and Prioritization Planning and Estimation Change Management Maintenance Approach
17
Tools
Functional Automation Test Management Configuration Management Defect Management Open Source
18
Framework
Architecture and Type Test Data Management Reporting Mechanism Reusability Maintainability Object Repository Database Testing
19
Infrastructure
Environment Management Test Bed creation and management
20
Best Practices
Scripting Standards User Guides Maintenance Process Handbook Automation Review Tool KR portal
21
Operating Model
Roles and Responsibilities Organizations SLA’s Project Structuring
22
Assess current automation capabilities
Identify the ideal automation tool
Set up communication model and status reporting
Structured methodology automation testing
Define governance model
Customized metrics framework
Definition of automation framework
Highlights
Approach - HighlightsApproach - Highlights
Organization – Organization structure, TrainingMethodology – Automation approach, Functional automation, Metrics and reporting
Use of reusable automation scripts
Well defined independent and peer review procedures in place
Defined communication and workload processes for onsite - offshore co-ordination
Fully customized metrics framework for implementation across application
Consolidation of automation tool
Well defined Organization structure and Governance model in place
Benefits
Focus Areas
23
Testing Services Practice Overview
IV&VINDEPENDENCE
Over 70% of testing performed against code provided by client or third-party vendors.
ALLIANCESEstablished alliances with leading tool vendors like Mercury, Borland & IBM Rational
CLIENTS200+ Clients with 10+Deep client engagements with over 100 people
CENTER OF EXCELLENCE
Invested in focused groups around tools & frameworks to provide client value-adds
PEOPLETeam of over 5000 dedicated SQA professionals
2001 & 2002 2003 2004 2005 2006 2007
Offered as a distinct service offering to customers
Established onsite-offshore model for testing
End-to-end IV&V services provided
Brought in domain alignment (Domain Product Testing and BA/QA Offering)
Launched to provide specialized functional testing services to existing Cognizant customers
Integrated with other value added services such as Performance testing
Engaged with clients to setup Managed Test Centers
Commenced new client engagements with Test Consulting
Focused on Automation and Mainframe CoEs
75
Independent Verification & Validation Service (IV&V)
Enhanced service offerings such as compliance testing, package testing, White-box testing as well as Domain/ Product Testing (VisionPLUS, FACETS & POS)
BFS Insurance Communications Manlog Technology Healthcare Life Sciences IME
?
$
Retail
OUR DOMAIN FOUNDATIONIntegrated BA / QA Offering in collaboration with domain practices
170
8500 (E)
Delivery excellence through deployment of innovative methodologies.
Expand Global footprint
2400
5000
800
24
Value Adds
AHEAD
CRAFT
WS TestProfessional
DataXpress
CRAFT 2.0
CRAFT defines the method for scripting of business functionalities as reusable libraries that are repetitive among test cases
Bulk uploads QTP scripts, attachments and folder structure to Quality Center
SOA testing solutions to test business logic. It enables client to execute data-driven web service testing without any programming knowledge
CRAFT 2.0 is a tool which streamlines the test execution activity during test automation, it dynamically executes the test cases in multiple machines in a distributed environment
DataXpress is a automated test generation tool which enables to streamline the test data preparation activity
25
QC2bugzilla
ROI Calculator
Watir Win2Pro
Selenium test manager
It integrates and synchronizes the defects management module of Quality Center with that of Bugzilla.
Converts Winrunner scripts to QTP automatically
It is an automation functional test tool developed for web automation
Return on investment details to have maximum transparency to client before an automation engagement.
Web Application Testing in Ruby (WATIR) is an open source function testing framework to test any web application built on ASP, .NET, J2EE or PHP
Value Adds
© 2008, Cognizant Technology Solutions. All Rights Reserved. The information contained herein is subject to change without notice.
Thank you
Testing ServicesTesting Services
Have a question for the speakers? Ask now.
Q & A