Mobile Application Dev and QA Testing with Simulated Environments

Parasoft Proprietary and Confidential 1

2015-04-23

Mobile Application Dev and QA Testing with Simulated Environments

Mobile DEV + TEST- April 23rd 2015

Parasoft Proprietary and Confidential 2Parasoft Proprietary and Confidential 2

Fixed Fixed

Testing is A Barrier to Speed

Plan Develop Build Test Release

1. “Test” is Currently a Time-Boxed Event

2. With the “Need for Speed” Testing Must be an in-line event

Plan Develop

Build

Test

Release

Build

Test

Build

Test

Build

Test

Build

Test

Build

Test

Build

Test

Build

Test

Build

Test

Build

Test

Build

Test


Next Generation Software Testing


Complexity Impacts End User Experience

Increased complexity, means multiple points of performance failure

Major ISP

3rd Party

Cloud Services

Content

Networks

WiFi

Browsers

Phones

Tablets

Mainframe

DB

Servers

Web

Services

App

Servers

Web

Servers

Load

Balancers

Mobile

Components

Service VirtualizationMobile Network Simulation


Complexity Impacts Process Velocity

Mobile Network

Simulation

App Behavior

Simulation

Mainframe

3rd Party

Services

ERP

Internal

App

Client

Simulation

Mobile

Application

Isolation


Software Drives Innovation

The Cost of Quality

associated with software

has shifted dramatically

Software has shifted

from process enabler to

business differentiator

SDLC Speed will be the

difference between a first

mover and a follower


Software Failures = Headlines

-$2.3 Bn-3.75%

The day of the announcement companies lost an average

of shareholder value

Software failures make headline news—eroding customer confidence, shareholder value and brand equity



821,0001,080,000

451,000

1,400,000

896,000

1,550,000

5,500,000

650,000

3,490,000

2,240,000

0

1,000,000

2,000,000

3,000,000

4,000,000

5,000,000

6,000,000

Cloud Services Airline Bank Software Retail

1st Event 2nd Event

Market does not forget— news about failure increase and average of 167%



Financial Airlines/Aero Government Media Internet/Tel

Automotive Technology Retail

http://aws.amazon.com/

http://aws.amazon.com/


The Cost of Software Quality - Sony

http://www.reuters.com/article/2011/04/26/us-sony-stoldendata-idUSTRE73P6WB20110426

http://www.digitaltrends.com/gaming/sony-fined-almost-400000-for-2011-playstation-security-breach/

Mark

et

Valu

e

Event 15 Days 30 Days

-22% -33% -30%-19% -11% -12%

Cumulative Loss = $18B

http://www.reuters.com/article/2011/04/26/us-sony-stoldendata-idUSTRE73P6WB20110426

http://www.digitaltrends.com/gaming/sony-fined-almost-400000-for-2011-playstation-security-breach/


User Experience Counts


Fixed Fixed

Impact of Test in the Evolving SDLC

Plan Develop Build Test Release

1. Test is Currently Time-Boxed Event

2. With the “Need for Speed” Testing Must be an in-line event

Plan Develop

Build

Test

Release

Build

Test

Build

Test

Build

Test

Build

Test

Build

Test

Build

Test

Build

Test

Build

Test

Build

Test

Build

Test


Testing Considered a Bottleneck

Basic Tasks

Skilled Tasks

Process Oriented

Business Views

Fully Automated Provisioning

End-to-End Tasks

Data Analytics

Organic Systems

Automation is Incremental

Effi

cien

cy

Software Quality Processes


What’s the View from Your Seat?

Actions

Strategy

Tasks

Process

Risk

Governance

Business

Portfolio

Application

Release

Iteration

Code


Process Intelligence

Process Intelligence aggregates “siloed” information throughout the SDLC and identifies business risks early and automatically

Correlate disparate data across: Dependent teams

Dependent components

Infrastructure systems

Testing practices and tools

Apply advanced statistical analysis From pre-build patterns

From Marketplace

With Open APIs

Advantages Defect remediation

Exception-based notifications


Transforming the Identification of Risk

From To

Causal Observations Probabilistic

Defect Documentation Most Likely Causes

Only Structured Data Structured and Unstructured

Filters Business Policies

Ad Hoc Automated

Tool Dependent Tool Independent


Next Generation Test Construction


The State of Continuous

Continuous delivery

Continuous development


Continuous – what’s missing

Continuous delivery

Continuous development

Continuous testing


Getting from “Automated” to “Continuous”

There is a big leap moving from “Automated” to “Continuous”

Business expectations clearly defined Business risks identified

Per application, team, release

Defects automatically prioritized versus business drivers No defect left behind

Mitigate business risks

Clear ownership and workflow for remediation Collaborative risk reduction

Collapse the remediation cycle time

Feedback loop for defect prevention Pattern definition

Defect prevention practices enhanced


“Shift-Left”


“Shift-Left”


“Shift-Left”


Continuous Tests

Logically componentized

Correlated with business requirements

Incremental

Repeatable

Deterministic

Maintainable within a process

Process isprescriptive based on results

Continuous

Testing


From Automated to Continuous

Continuous testing accelerates the SDLC by

managing quality expectations and actionable tasks

Requirements

Defined

Policy Management

Development

Defect Prevention

Development

Development Testing

Static Analysis

Unit/Component

Peer Review

Automated Tests

Integration Testing

API/Service Tests

Smoke Test

Security Tests

Automated Tests

System Testing

Functional Tests

Scenario Tests

Performance Tests

CI Build

Defect Remediation Tasks

Go

Release Path

No GoBusiness

Decision

Service Virtualization – Test Environment Access


30

Environment Access?

The average number ofdependent applicationassociated with theSystem Under Test (SUT)

BUT, Dev/QA only have “trustworthy”

access to SIXof the applications

302928272625

242322212019

181716151413

121110987

654321


Test Environment Configuration

The average percentage of time spent configuring the test environment.

30%

50%The average percentage of the test plan ableto complete once configured.


Survey Results…

When testing with a dependent application only need access to a small percentageof the applications functionality

20%


Lack of reliable access to arealistic test environmenthampers software quality

Conclusion


Service Virtualization delivers a simulated dev / test environmentallowing an organization to test

anytime or anywhere

Service Virtualization


So, What are you Virtualizing?

• Mainframe

• SOA / Web services

• Java

• .NET languages

• XML

• WSDL

• WADL

• UDDI

• WSIL

• SOAP

• PoX (Plain XML)

• REST

• JSON

• BPEL

• Web Applications

• Mobile Interfaces

• AJAX

• JSP

• JavaScript

• HTML

• CSS

• WS-*

• Standards

• MTOM(XOP) / MIME / DIME

• OAuth

• TCP/IP

• webMethods Broker

• webMethods IS

• HL7

• FTP

• HTTP 1.0

• HTTP 1.1

• HTTPS

• JMS

• IBM WebSphere MQ

• Sonic MQ

• RMI

• EJB

• SMTP

• Tibco Rendezvous

• .NET WCF

• ISO 8583

• EDI

• SAP

• custom

AUT


Stubs Vs Service Virtualization

Stubs Inside-out approach that disassociates a test case

with a dependent systems Brittle Limited reuse Static response

Service Virtualization Creates an environment in which to run “rich” test scenarios Virtualized assets represent real system behavior All virtualized-assets are reusable Virtualized assets are programmable and extensible

Service Virtualization provides a complete environment for developing and

testing versus complex, dependent systems


Access to Dev / Test Systems

Test Access

Simple Difficult

Configura

tion

Complex

Internal

Service

Internal

Application 3rd Party

Service

ERP

Mainframe

External

Application

External

Database

Internal

Database

Cloud

App

Message

Queue

ESB

Web

Server


Service Virtualization & Dependencies


Virtual Lab

Configura

tion

Access

Access Dependent applications difficult Scheduling conflicts High access fees Geo-political boundaries 3rd party or partner applications

Configuration Complex to configure No control Limited variability Consumes test time

DifficultSimple

Ea

sy

Com

ple

x


The Total Cost of System Access

Service Virtualization dramatically reduces the cost of delivering

a dev / test environment

Hardware Savings

Access EfficiencyStaged w/ Virtual Test Lab

Staged Application Instance

System Behavior

Cost Only manage the functionality

you need Only manage data you need Low-Risk “disposable instance”

that can be accessed at anytime Dramatically reduce test

environment setup and management



Service Virtualization Examples

Mobile Application development and extension

Agile/Parallel development limited by system dependencies

Capacity Constrained staged environments

Limited access to mainframes, ERPs, or 3rd party systems

Test data management for complex transactions

Parasoft Virtualize is used throughout the SDLC to speed access to

and reduce the costs of managing development and test environments


The Test Environment Challenge

Test environment access is outside the control of development and test leaving

gaps in the process

IT OperationsParallel development delays… Need simple, realistic access to dependent components…

Too much time waiting for access…Need reliable test data

Need a realistic testenvironment easy to maintain

Scheduling

Configuration

Access Limits

DependentApplications

Staged Assets

3rd Party Assets Virtual Environments

Hyper Visor

App App App


The Test Environment Challenge

Test environment access is outside the control of development and test leaving

gaps in the process

IT Operations

Scheduling

Configuration

Access Limits

DependentApplications

Staged Assets

3rd Party Assets Virtual Environments

Hyper Visor

App App App

Create, Manage, Provision


Environment – “Passive” or “Active”

Parasoft’s architecture is built for “passive” or “active” environment modeling and maintenance

PVAEndpoint

or

App Component

Protocol

Correlated

Tests

AssociatedEnvironments

Message

Payload

Response

Logic

Performance

Profile

Test

Data

Management

• Native Support

• Extension API

• Marketplace

• Environment Manager

• ALM APIs

• CR/CD APIs

• Marketplace

• Environment Manager

• Marketplace

• Native Support

• Extension API

• Marketplace

• Modeling UI

• Log Files API

• Marketplace

• Native Support

• APM API

• Marketplace

• Modeling UI

• TDM API

• Marketplace


Value of Service Virtualization

Increase Access to “Trustworthy” Test Environments Provide a complete test environment Provide access 24/7 Flexibility provides a platform for extending test scenarios, coverage

Reduce Operational Expenditures Significantly reduce test environment configuration time Simplify test environment access Remove the bottleneck of test environment setup

Reduce Capital Expenditures Software assets can be run anywhere Significantly reduce bandwidth demand Reduce dependency on staged environments

Service Virtualization drives better software quality

while reducing CapEx and OpEx



Parasoft Service Virtualization

Wayne Ariola

Chief Strategy Officer

Parasoft Corporation

[email protected]

Questions?

mailto:[email protected]