Automated Testing and Test Data Management – using Service VirtualizationKishore Bhatia- DevOps Practice Lead

2

Kishore Bhatia, DevOps/Engineering Practices

Developer, SOA Architect with a passion for driving agility and efficiency in development and operations teams across the Application Lifecycle management stack.

Technology: Agile practitioner and Enterprise Dev Engineering lead with a proven track record leveraging agile concepts to develop value and consistently deliver results for Fortune 500 clients, start-ups and the community. Actively practicing TDD and creating Automation frameworks (continuous Build, Test, deploy, monitor and measure) to enable Agile development.

Industry expertise: Financial Services, Telecommunications, Healthcare, High Tech.

Community: NY Tech Community, Users group (Java, Google), CD/DevOps and iMentor

www.linkedin.com/in/kishorebhatia/

Twitter - @BhatiaKishore

3

Agenda

Test Automation and Test Data Management challenges

Test Automation with Service Virtualization

Test Data Management with SV and Datafinder

MainframeClient/Server

(including packagedApps such as SAP, Siebel, Oracle…)

Composite(Custom Applications,

SOA, Cloud)

1980 … 1985 … 1990 … 1995 … 2000 … 2005 … 2010 … 2015

Constraints

Complexity

Costs

Changes in software developmentChanges in software development

5

6

The Product is the entire brand and customer experience

Service oriented products are delivered late, over budget and with questionable quality…WHY?

Innovate or die

7

8

“I can’t do anything until I have everything… and I never have everything!”

INCOMPLETEDEVELOPMENT

SYSTEMUNAVAILABLE

INVALID DATA

ACCESS FEES

The big problem: constraints

Problems Solved:

1. Shift-Left 2. Infrastructure Availability 3. Performance Readiness 4. Test Scenario Management

Solution: service virtualization

10

UPSETCUSTOMERS

SERVICEDOWN

PRODUCTIONISSUES

“Too many things go into production without adequate testing”

Problem: change & unintended consequences

11

Reduced issues in QA and reduce effort in integration release cycle

Solution: continuous validation

12

1. Unintended Consequences: Individual components and technologies are changing at their own pace and create ripple effects across the architecture

2. Environment Management: Dependency on constrained internal and external systems. Access and capacity constraints on key systems reduces agility, and increases dependencies and costs between teams

3. Lack of Automated Testing: Traditional testing techniques cannot automate testing against component and middleware-based workflows

4. Lack of Control: Governance techniques to dictate standards are useless due to inability to enforce policies

Challenges Summary

13

CA LISA: A Complete, Integrated Solution

LISA Framework

Continuous Monitoring

LISA Validate LISA Virtualize

LISA Test

LISA Product Suite

Pre-packagedIntegrations

LISASDK

PerformanceEnvironment

Capture & Simulation

Test DataAutomation

Load & Performance

Advanced UI Testing

Functional& Regression

SOA Policy Validation

QualityConsole

PathfinderWorkflow Transparency

Defect Collaboration

14

Integration Testing with LISAtesting is no longer an event

4

Drive mobile usingLISA Test, MobileCloud

Setup SIT environment with Continuous Delivery

Drive Web with LISA Test

3

Drive headless APIs with LISA Test

2

Eliminate constraints with Service Virtualization

1

Watch for unintended consequences usingContinuous Validation Environment

(CVE)

Integration Lab

15

LISA ContinuousValidation

Service Plan

Internal

Service1Data

TransactionsMDMRegistry

Legacy App

Integration / Services Platform

Business outcomes continuously validated

LISA Continuous Validation

App1 policy test failed: Do not deploy!Internal

Service2

APP 1

APP 2

Prevent unintended consequences through Continuous Validation

16

• LISA Test

• Functional Testing of APIs and Services

• Advanced UI Testing Capabilities with Selenium backend

• Support across platforms like SAP, TIBCO, WebM, IBM, and Oracle

• LISA MobileCloud

• Mobile Testing across all phases of SDLC

• Test with real devices in Cloud

• LISA DataFinder

• Solve Test Data Management challenges in end-to-end environments

• LISA CVE

• Continuous Validation using CVE integrated with various notification mechanisms

LISA Test

Mobile Testing

API Testing

Advanced Web UI Testing

SAP Testing

Continuous Validation

LISA MobileCloud

Take Test Automation level to new heightswith LISA Test for API, Web and Mobile

17

Setup Perf Env withContinuous Delivery

Setup data with Datafinder

1

Drive web load using Cloud Test

3Drive mobile usingLISA Test4

Eliminate constraints with Service Virtualization

2

Performance Testing with LISA

18

Mobile Applications TestingLISA Test, LISA Cloud Test & Service Virtualization

LISA Service Virtualization

SUT

LISA Test

Channels

Live Backend Systems

(CRM, ERP, ...)

Mobile Gateway

Local Messaging

Service

HUB

Applications

Notification Service

Voice Media Gateway

…Load Generator

Load Generator

LISA Cloud Test

Load Generator

Load Generator

19

Test Data – Golden Schema requirements

20

Testable Data

21

Data Obfuscation

Production Development

Data Subsetting

Good way to start a Golden copy. But...

Typical Test Data Management

22

TDM– Myriad use-cases

• Subsetting• Ages almost immediately

• Developers and testers will build interesting data• Bug scenario could be copied in from production

• Requests for a refresh!• Whoops we’ve lost all our lovely juicy data• Manual with schema refresh, additional Inserts/Updates– got it all?

• What about coverage?• Lots of clever combinations! • Add in “Bad” data• Add in performance test data• Min. data max. coverage

• Integration Testing Data• Need to build integration data• Coherent and matches with upstream and down stream systems

23

Test Data Warehouse

24

Test Data Management Solution

25

TestDatabase

Applica

tion U

I

Flat File

API

SOAP

ProductionDatabase

Data DesignTest DesignCoverage

DATA OBJECTS

Data Finder

DATA INHERITANCE

VERSION CONTROL

Test Data Warehouse

Test Data Warehouse with Data Finder

End to End Tests

ERP Database SOA

Test Data Warehouse

Batch

API

Data Coverage Data FunctionsData

InheritanceExpected Results

Version Control

Thanks!

Links –

CA Application Delivery

Demos/Tutorials

Graphics Courtesy-

CA Technologies

Web