Embed Size (px)
DESCRIPTION
"TaaS" is an open-source product that allows you do achieve the "correct" way of doing integration testing across a variety of products via Test Automation. Typically in organizations, there are multiple projects / products. Many organizations like to have a common Test Automation solution across these products in an effort to standardize the framework. However, this is not a good idea! Each product should be tested using the tools and technologies that are "right" for it. Yet - these different products talk with each other and you need to test the integration between them in an automated way.
Citation preview
Automate across Pla.orm, OS,
Technologies with TaaS
Anand Bagmar So#ware Quality Evangelist
Principal Consultant, @BagmarAnand
Organiza@on Objec@ve
Time
Quality
Money / Value
Reality – Organiza@ons are spread across the World
What is a prac@ce that makes Teams successful?
Test Automa=on
What is a prac@ce that makes Teams unsuccessful?
Test Automa=on
Is Test Automa@on treated as 1st class ci@zen in your organiza@on? • Value – To the team? – From the team?
• Quality – Provide feedback? – Built well?
Where is Test Automa@on placed in the Food Chain of SDLC?
Principles & Prac@ces for Building a
Test Automa@on Framework
Principles for building an Automa=on Framework
Code Quality
Design PaDerns
Abstrac=on Layers
Refactor Pairing
Extend
Evolve
Prac=ces for building an Automa=on Framework
No Copy-‐Paste
Configurable
Test Data
Tools & U=li=es Logging
Screenshots, Video
Recording
Con=nuous Integra=on (CI)
Audience Survey
How many of you are part of a large organiza@on?
Audience Survey
What are the average number of products in your organiza@on’s porVolio?
… large organiza,on?
Audience Survey
What are the average number of projects across all these products?
… large organiza,on? … number of products?
Audience Survey
What is the typical technology stack of these products?
… large organiza,on? … number of products and projects?
Audience Survey
How many of you are part of merged / acquired companies?
Do you have the same technology across the “new” products?
… large organiza,on? … number of products and projects? … same technology stack?
How many of you work in distributed teams?
Audience Survey … large organiza,on? … number of products and projects? … same technology stack? … merged companies – same technology stack?
Audience Survey
Have you heard of a Common Test Automa@on framework?
Good? Bad?
Easy? Difficult?
… large organiza,on? … number of products and projects? … same technology stack? … merged companies – same technology stack? … distributed teams?
Case Study
Outlook Outlook (Windows)
Outlook (Mac)
Outlook Web Access (OWA) (Browser) Outlook
Sync (Android)
…
How will you Automate the Tes@ng for Outlook?
Outlook for Windows • White / QTP
Outlook for Mac • Automator
Outlook Web Access (OWA) • Selenium / Ruby
Outlook Sync for Android • Robo@um / Java
For a user who has installed and configured Outlook on – Windows OS – desktop version, and, – Android device – na@ve mobile app – …
An email draTed in one product is reflected in the other products
How will you Automate the Integra@on Test ….
End-‐2-‐End Integra-on
Tests
(Test Framework
for) Outlook on Windows
(Test Framework
for) Outlook Sync on Android
Create & Save as DraE
1
Verify DraE 4
Verify DraE present 2
Modify DraE
3 Orchestrator
Can you use the same technology stack for automa=ng the tes=ng
of each variant of Outlook?
The Problem
Outlook for Windows • White / QTP
Outlook for Mac • Automator
Outlook Web Access (OWA) • Selenium / Ruby
Outlook Sync for Android • Robo@um / Java
My Original Problem
The Solu=on
What is TaaS?
“TaaS” is a product that allows you to validate the integra-on across a variety of products via Test Automa-on
• The “correct” way • Pla.orm / OS agnos=c
• Implementa=on Language Independent
What is TaaS?
Agnos-c PlaVorm / OS
Implementa@on Language
What is TaaS?
Contract Decoupling of technology barriers
Timeout
What is TaaS?
Simple Parameters Input & Output
What is TaaS?
Developed in Ruby using Sinatra
TaaS is Not a …
“Tool” for integra@on different products -‐ Lack of Security
-‐ Probably not as robust
TaaS is Not a …
Load Tes@ng Tool
How does TaaS work?
End-‐2-‐End Integra-on
Tests Framework
TaaS Client
Test Framework
for Outlook on Windows
TaaS Server
Test Framework for Outlook Sync on Android
TaaS Server
Create & Save as DraE
1
Verify DraE 4
Verify DraE present 2
Modify DraE
3 Orchestrator
Service Providers
Demo
How to use TaaS? TaaS Server -‐> Ruby project
TaaS Service provider -‐> Cucumber-‐jvm Project (Java)
Client – command line invoca@on
TaaS Server Implementa@on
What is a Contract?
Specified in a simple yml file
Contract Defini@on
Contract Defini@on
Implement the Contract
• Update tests to -‐ – Handle input parameters
• From Environment variables
– Execute the “test”
Implement the Contract
Parse Input Parameters
Return the Results
• Output parameters – As console output between special markers
• Console logs • Excep@ons (if any) • All return values are in “json” format
Collect Output Parameter Values
Return Output Parameter Values
Run TaaS Server
• Dedicated machine • Provide IP address / host name + port to poten@al clients
TaaS Client Implementa@on
Setup TaaS Client
Invoke TaaS Contract
TaaS Technical Summary
TaaS Server
• Specify contract details • Implement contracts • Return the results • Run TaaS Server (REST service) • Serve TaaS Client requests – As separate processes
TaaS Client
• Collect the input parameters • Invoke the contract • Consume the result • Con@nue with test orchestra@on
Summary of Features
• Contract – Decoupling of technology barriers – Timeout
• Passing of input parameters as Environment variables
• Result as json – Output parameters – Console logs and errors – Excep@ons
• Developed in Ruby using Sinatra
Why is this a Good Idea?
• Automate the last-‐mile • No code duplica@on • Implementa@on of contract lies with the framework tes@ng that product – Evolves with product changes
• Decoupling of technologies
Why is this a Good Idea?
• Helps in Manual Tes@ng (setup of data) • Anyone can use it • Each product is tested in the “best” possible way
How can you use TaaS?
• Open source (apache2 license) • Available on github! • Available on rubygems.org ! • Refer sample project (taas-‐server) to get started
Steps to Follow
• Nego@ate contract details • Setup & configure TaaS Server – Implement contracts – Return the results – Run TaaS Server (REST service)
• Configure TaaS Client
Does it really work?
>1.2K downloads
Before you use TaaS …
Iden@fy – • Is TaaS the right thing for you?
• Error codes for the service • Service Discovery • Security • Create a Java / .Net version of TaaS Server • Asynchronous requests
How can You help TaaS?
What TaaS did for me?
• Be crea@ve • Find Innova@ve solu@on to the problem • Another open source contribu@on
