Leveraging Automated Testing to Improve Application Qualityfiles.meetup.com/18247475/Leveraging Automated... · WHY AUTOMATE API TESTING? •Each scenario could have many variations

Leveraging Automated Testing to Improve Application Quality

Copyright © 2016 Deliveron Consulting Servicesw w w . d e l i v e r o n . c o m

INTRODUCTIONS

Matt Michaelsen

Tim McKeighan Misty Skudler

Kim Dietz


AGENDA

• Automation Challenges

• Defining an Automation Strategy

• Automated Testing Demos

– Automated UI Testing

– API Testing

– Load Testing

• Quality in the Software Delivery Pipeline

• Monitoring Quality

• Q&A


BUSINESS DRIVERS TODAY

Reduce time to market

Drive more customers to our software

Innovate from customer feedback

Every company is a software company

Improve application quality and performance


SOFTWARE DELIVERY PIPELINE

Dev Test Staging Prod


Defining an Automation StrategyDefining an Automation StrategyResponsive UI TestingAPI TestingLoad TestingQuality in the Software Delivery PipelineMonitoring Quality

AUTOMATION CHALLENGES


AUTOMATION CHALLENGES

• Identifying an automation strategy

• Determining where to start with automation

• Ensuring test execution efficiencies

• Benefiting from automation in the delivery cycle

• Providing visibility to the quality


Automation Challenges

Defining an Automation StrategyResponsive UI TestingAPI TestingLoad TestingQuality in the Software Delivery PipelineMonitoring Quality

DEFINING AN AUTOMATION STRATEGY


TESTING TYPES

Unit UI

Integration (API)

Load and Performance

Security

Exploratory Usability

UAT

AUTOMATED MANUAL

Unit UI

Integration (API)

Load and Performance

Security


AUTOMATION APPROACH

• Which test cases should we automate?– Business critical test cases

– High risk functionality (must work)

– Test cases that are repetitive

– Test cases that time consuming

• Which are not the best candidates?– Newly designed test cases

– Requirements that are changing frequently

– Exploratory testing or ad-hoc testing

– Low value or minimal ROI test cases


AUTOMATION ALLOCATION

• UI Testing– Highest cost

– Most fragile

– Slow

• API/Integration– Least costly

– High reliability

– Faster

• Unit Test– Low cost

– Requires ongoing maintenance

– Fastest

UI

API/Integration

Unit


TOOL CONSIDERATIONS

• Skillset of testers on the team

– Coding experience

– Manual testers

• Support for software platforms and configurations

– Desktop

– Mobile

• Integration with Agile Project Management tools


TESTING TOOLS

Microsoft VS ALMManual Testing

Team Foundation

Server

Exploratory Testing

Selenium UI Testing

Load Testing

API Testing

Automated UI Testing

Test Case DefinitionMicrosoft Test

Manager

Visual Studio

Unit Testing

Business Analyst/ Tester

Developer

Other ALM Tools

LogiGearTestArchitect


AUTOMATED TESTING DEMOS


DELIVERMORE WELLNESS SITE

• Health and Wellness application

• Releases after each two week sprint

• Responsive design

– PCs

– Mobile Devices

• Integrations with other systems/applications

• High user load


DELIVERMORE WELLNESS SITE


Automation ChallengesDefining an Automation Strategy

Responsive UI TestingAPI TestingLoad TestingQuality in the Software Delivery PipelineMonitoring Quality

UI TESTING


UI TESTING

Service Layer(Business Logic)

Database Layer(Database)

UI Testing Performed HerePresentation Layer

(User Interface)Presentation Layer

(User Interface)


WHY AUTOMATE UI TESTING?

• Multiple Platforms and Configurations

– Desktop

– Mobile Devices

– Operating Systems

– Browsers

• Manual Execution is very time consuming

• Consistency of test execution

• Who likes to test the same thing over and over and over….?


TYPES OF UI AUTOMATION

• Scripting

• Keyword-based / Action-based


TESTARCHITECT ARTIFACTS

• Interfaces



• Actions



• Test Cases


UI TESTING DEMO

• Create an automated UI test case

– User can log in and navigate to the Contact Page

• Show how one test can be used for a Responsive UI

• Run test case on desktop and mobile device


Automation ChallengesDefining an Automation StrategyResponsive UI Testing

API TestingLoad TestingQuality in the Software Delivery PipelineMonitoring Quality

API TESTING


API TESTING



API Testing Performed Here

Presentation Layer(User Interface)



WHY AUTOMATE API TESTING?

• Each scenario could have many variations of inputs and outcomes

• Service could have multiple calling applications

• Quickly Test network connectivity across application enterprise


API TESTING DEMO

LogiGearTestArchitect

Test Harness(C#)

Calculate Wellness

Web Service

Calculate Wellness Business Objects


Automation ChallengesDefining an Automation StrategyResponsive UI TestingAPI Testing

Load TestingQuality in the Software Delivery PipelineMonitoring Quality

LOAD TESTING


LOAD TESTING



Load Testing Performed Here




Load Testing Performed Here


WHY AUTOMATE LOAD TESTING?

• Difficult to simulate actual expected loads manually

• Measure Performance Impact of changes


TYPES OF LOAD TESTS

• Smoke

• Stress

• Performance

• Capacity Planning


MEASURING THE IMPACT OF CHANGES


LOAD TESTING DEMO

Visual Studio

Calculate Wellness Web

Service

Delivermore Wellness Web

Server


Automation ChallengesDefining an Automation StrategyResponsive UI TestingAPI TestingLoad TestingQuality in the Software Delivery Pipeline

Monitoring Quality

QUALITY IN THE PIPELINE


SOFTWARE DELIVERY PIPELINE


MONITORING QUALITY

• Testing Results for Automated and Manual Testing

• Iteration/Sprint Level

• Regression

– Configuration

• Browser

• Device

– Environment


SUMMARY

• Defining a strategy and implementing automated testing for your application

• Demos of Automated UI, API & Load Testing

• Incorporating quality within your software delivery pipeline

• Monitoring quality for your project


LINKS

• Deliveron

– http://www.deliveron.com/

– https://www.linkedin.com/company/deliveron-consulting-services

• Logigear TestArchitect

– http://testarchitect.logigear.com/

– If you would be interested in a TestArchitect demo or trial, please contact Keith Holt at Deliveron

• 402.238.1399 ext. 105

• [email protected]

http://www.deliveron.com/

https://www.linkedin.com/company/deliveron-consulting-services

http://testarchitect.logigear.com/

mailto:[email protected]

We believe in HELPING our CUSTOMERS create software SOLUTIONS in a better way

Documents

Leveraging Automated Testing to Improve Application Qualityfiles.meetup.com/18247475/Leveraging Automated... · WHY AUTOMATE API TESTING? •Each scenario could have many variations