Leveraging Automated Testing to Improve Application Qualityfiles.meetup.com/18247475/Leveraging...

Leveraging Automated Testing to Improve Application Quality

INTRODUCTIONS

Matt Michaelsen

Tim McKeighan Misty Skudler

Kim Dietz

AGENDA

• Automation Challenges

• Defining an Automation Strategy

• Automated Testing Demos

– Automated UI Testing

– API Testing

– Load Testing

• Quality in the Software Delivery Pipeline

• Monitoring Quality

• Q&A

BUSINESS DRIVERS TODAY

Reduce time to market

Drive more customers to our software

Innovate from customer feedback

Every company is a software company

Improve application quality and performance

SOFTWARE DELIVERY PIPELINE

Dev Test Staging Prod

Defining an Automation StrategyDefining an Automation StrategyResponsive UI TestingAPI TestingLoad TestingQuality in the Software Delivery PipelineMonitoring Quality

AUTOMATION CHALLENGES

• Identifying an automation strategy

• Determining where to start with automation

• Ensuring test execution efficiencies

• Benefiting from automation in the delivery cycle

• Providing visibility to the quality

Automation Challenges

Defining an Automation StrategyResponsive UI TestingAPI TestingLoad TestingQuality in the Software Delivery PipelineMonitoring Quality

DEFINING AN AUTOMATION STRATEGY

TESTING TYPES

Unit UI

Integration (API)

Load and Performance

Security

Exploratory Usability

AUTOMATED MANUAL

Unit UI

Integration (API)

Load and Performance

Security

AUTOMATION APPROACH

• Which test cases should we automate?– Business critical test cases

– High risk functionality (must work)

– Test cases that are repetitive

– Test cases that time consuming

• Which are not the best candidates?– Newly designed test cases

– Requirements that are changing frequently

– Exploratory testing or ad-hoc testing

– Low value or minimal ROI test cases

AUTOMATION ALLOCATION

• UI Testing– Highest cost

– Most fragile

– Slow

• API/Integration– Least costly

– High reliability

– Faster

• Unit Test– Low cost

– Requires ongoing maintenance

– Fastest

API/Integration

TOOL CONSIDERATIONS

• Skillset of testers on the team

– Coding experience

– Manual testers

• Support for software platforms and configurations

– Desktop

– Mobile

• Integration with Agile Project Management tools

TESTING TOOLS

Microsoft VS ALMManual Testing

Team Foundation

Server

Exploratory Testing

Selenium UI Testing

Load Testing

API Testing

Automated UI Testing

Test Case DefinitionMicrosoft Test

Manager

Visual Studio

Unit Testing

Business Analyst/ Tester

Developer

Other ALM Tools

LogiGearTestArchitect

AUTOMATED TESTING DEMOS

DELIVERMORE WELLNESS SITE

• Health and Wellness application

• Releases after each two week sprint

• Responsive design

– PCs

– Mobile Devices

• Integrations with other systems/applications

• High user load

DELIVERMORE WELLNESS SITE

Automation ChallengesDefining an Automation Strategy

Responsive UI TestingAPI TestingLoad TestingQuality in the Software Delivery PipelineMonitoring Quality

UI TESTING

Service Layer(Business Logic)

Database Layer(Database)

UI Testing Performed HerePresentation Layer

(User Interface)Presentation Layer

(User Interface)

WHY AUTOMATE UI TESTING?

• Multiple Platforms and Configurations

– Desktop

– Mobile Devices

– Operating Systems

– Browsers

• Manual Execution is very time consuming

• Consistency of test execution

• Who likes to test the same thing over and over and over….?

TYPES OF UI AUTOMATION

• Scripting

• Keyword-based / Action-based

TESTARCHITECT ARTIFACTS

• Interfaces

• Actions

• Test Cases

UI TESTING DEMO

• Create an automated UI test case

– User can log in and navigate to the Contact Page

• Show how one test can be used for a Responsive UI

• Run test case on desktop and mobile device

Automation ChallengesDefining an Automation StrategyResponsive UI Testing

API TestingLoad TestingQuality in the Software Delivery PipelineMonitoring Quality

API TESTING

API Testing Performed Here

Presentation Layer(User Interface)

WHY AUTOMATE API TESTING?

• Each scenario could have many variations of inputs and outcomes

• Service could have multiple calling applications

• Quickly Test network connectivity across application enterprise

API TESTING DEMO

LogiGearTestArchitect

Test Harness(C#)

Calculate Wellness

Web Service

Calculate Wellness Business Objects

Automation ChallengesDefining an Automation StrategyResponsive UI TestingAPI Testing

Load TestingQuality in the Software Delivery PipelineMonitoring Quality

LOAD TESTING

Load Testing Performed Here

WHY AUTOMATE LOAD TESTING?

• Difficult to simulate actual expected loads manually

• Measure Performance Impact of changes

TYPES OF LOAD TESTS

• Smoke

• Stress

• Performance

• Capacity Planning

MEASURING THE IMPACT OF CHANGES

LOAD TESTING DEMO

Visual Studio

Calculate Wellness Web

Service

Delivermore Wellness Web

Server

Automation ChallengesDefining an Automation StrategyResponsive UI TestingAPI TestingLoad TestingQuality in the Software Delivery Pipeline

Monitoring Quality

QUALITY IN THE PIPELINE

SOFTWARE DELIVERY PIPELINE

MONITORING QUALITY

• Testing Results for Automated and Manual Testing

• Iteration/Sprint Level

• Regression

– Configuration

• Browser

• Device

– Environment

SUMMARY

• Defining a strategy and implementing automated testing for your application

• Demos of Automated UI, API & Load Testing

• Incorporating quality within your software delivery pipeline

• Monitoring quality for your project

• Deliveron

– http://www.deliveron.com/

– https://www.linkedin.com/company/deliveron-consulting-services

• Logigear TestArchitect

– http://testarchitect.logigear.com/

– If you would be interested in a TestArchitect demo or trial, please contact Keith Holt at Deliveron

• 402.238.1399 ext. 105

• keithholt@Deliveron.com

We believe in HELPING our CUSTOMERS create software SOLUTIONS in a better way

Leveraging Automated Testing to Improve Application Qualityfiles.meetup.com/18247475/Leveraging...

Documents

Automated testing 101

Automated UI Testing

Lecture 19: Automated Testing Automated Testing Strategy

Automated Testing Tools

Automated Testing vs Manual Testing

Automated Infrastructure Testing

Automated Acceptance testing by Developers & Automated ... · Automated Acceptance testing by Developers & Automated Functional Testing by Testers. ... Case Study –An ... • Web

Leveraging Azure for Performance Testing

Automated Website Testing

Automated Performance Testing

Automated Testing Plan

Automated Testing Tool - Automated Regression Testing for ... · Automated Testing Tool For POS applications Connie Driscoll CEO CDA RHISCOM’s Representative connie@cdriscollassociates.com

Leveraging Project Testing Accelerators - SAPfm.sap.com/pdf/10170/10_23/10-23_130_BT3_Willaims_Leveraging... · Leveraging Project Testing Accelerators ... SAP uses Mercury internally:

Software testing techniques Automated software testing

Agile/Automated Testing

Automated Browser Testing

Automated Testing Nathan Weiss April 23, 2007. Overview History of Testing Advantages to Automated Testing Types of Automated Testing Automated Testing

Biomedical Automated Testing

Automated Testing

Feuerstein Automated Testing