Shift Left with Service Virtualization: Rational Test ...gotocon.com/dl/goto-cph-2015/slides/KhurramNizami_Shift...IBM Rational Test Virtualization Server Continuous Testing from build

0 © 2014 IBM Corporation

Shift Left with Service

Virtualization: Rational Test

Workbench & Test Virtualization

Server

Khurram Nizami ([email protected])

1 © 2014 IBM Corporation

© 2015 IBM Corporation 2

IBM Competitive Project Office

Presentation layer

Integrations, data and business logic

Only one-tenth of an iceberg is above water…

Traditional UI testing:

Automated Integration Testing and Service Virtualization:

The majority of risk in modern systems is seldom tested as it is unseen.



Complexity increases as your points of integration increase….

4 systems 5 systems 12 systems 6 interactions 10 interactions 66 interactions

• Types of systems and

interfaces?

• Cost to interact with

the systems?

• Availability of

systems? Collectively?

• Difficulty in setting up

test systems?

“Lets just deal with it later.….”



Today, integration of enterprise applications still follow the “big-bang” approach …

Mobile

component

Web

component

Mainframe

component

Inte

grat

e

typ. 6 Weeks 1 week

Big Bang!

typ. 6 Weeks 1 week

Big Bang!

Inte

grat

e



Software quality fact: errors caught early cost less to fix . . .

Rela

tive C

ost to

Repair

Acceptance Unit Test Coding Maintenance

1.5x

Stage in which Error is Discovered

“30% of project costs are due to rework

and poor execution of requirements”

“50% of outsourced projects are expected

to under perform”

“34% of software projects are deemed successful

costing $300B annually”

Analysis Design

5x 10x

20x

50x

200x

Bugs/failures are exponentially more expensive the longer they go uncaught

Extends to analysis, requirements, development, and everywhere across the application lifecycle

5



Shift Left – focus on testing high risk areas earlier in the development lifecycle

UNIT SERVICE UI

Automated Integration

Testing

Service Virtualization

Traditional UI Tools

Testing only at the UI level provides little confidence for deployment of complex distributed systems

Testing must be shifted left with focus on testing the highest areas of risk

Service virtualization can be applied at all phases – even allowing integration testing to begin in parallel with Unit Testing.

Shift Left

Development Testing Staging Production



With virtual testing, the “big bang” is eliminated from the process

Requirements

Unit Tests

Performance

UAT

OAT

Big Bang

Integration Tests

System Tests

System

Tests

• Accelerates testing

• Reduces costs

• Lowers risk

7



“We could just write up some stub code…” Not a bad idea, for short lived apps, simple scenarios and limited

sets of integrations…

Hybrid Cloud DevOps

public class TestStockAnalyzer {

public void TestMethod1()

{

// Arrange:

var analyzer = new StockAnalyzer();

// Act:

var result = analyzer.GetContosoPrice();

// Assert:

Assert.AreEqual(123, result); // Why 123?

}

}

public class StockAnalyzer {

public int GetContosoPrice(){

var stockFeed = new StockFeed();

return stockFeed.GetSharePrice("COOO");}}

}

public class StockFeed {

//this stuff isn’t ready, isn’t available yet, is expensive to use, or

some other reason…

}

public class TestStockAnalyzer {

public void TestMethod1()

{

// Arrange:

var analyzer = new StockAnalyzer();

// Act:

var result = analyzer.GetContosoPrice();

// Assert:

Assert.AreEqual(123, result); // Why 123?

}

}

public class StockAnalyzer {

public int GetContosoPrice(){

var stockFeed = new StockFeedStub();

return stockFeed.GetSharePrice("COOO");}}

}

public class StockFeedStub {

public float GetSharePrice(string stock) {

return 123.00;

}

}



Shift Left with Service Virtualization

• Service virtualization

can replace

components and

databases in complex

test scenarios.

• Service virtualization

can work with any

type of test – manual,

automated or

performance



Stubs or “virtual” components allow developers and testers to start integration tests early

Incremental Integration Testing

actual component

a stub that replaces a component that does not exist yet or cannot be accessed in dev

and test

The idea of “stubs” is not new

Developers used them for decades to simulate interfaces and behaviour of

the components that were not available for testing

Components are introduced into the continuous integration cycle in a prioritized,

controlled fashion.

Components not yet built or components that cannot be accessed in the

Development can be simulated and tested against.



IBM Rational Test Virtualization Server

supports multiple platforms and protocols

Transports and Formats The building blocks of an integration test or virtualized service

The transport is the technology that carries your message. E.g. HTTP

The format is the type of message that is carried on the transport e.g. XML

The payload is the contents of the message fields

PAYLOAD

Several transports have

advanced support e.g.

sift and pass through

Formats are presented

in an easily readable

and configurable tree

structure

Fields can be

parameterized using

data sources



Testing and Virtualizing Web Technologies

HTTP(s)

XML REST

SOAP JSON

TCP UDP

XSD

WSDL

WADL

RECORD

Accelerator

Accelerator

• Record transactions

• Or use accelerators such as WSDLs to short cut the process

• Build tests and stubs across a range of supported protocols and formats



IBM Rational Test Virtualization Server

supports multiple platforms and protocols Transport/Protocol Message Formats Technologies

• ActiveMQ • Email (SMTP, IMAP) • Files • FTP/S • HTTP/S • WebSphere MQ • Java RMI • JMS • SAP IDoc, BAPI, RFC • Software AG’s IB & IS • Sonic MQ • TCP/IP • TIBCO Rendezvous,

Smart Sockets & EMS • CICS TG • CICS IPIC • IMS Connect • MQTT

• .Net Objects

• Bytes

• COBOL Copybook

• ebXML

• ASC X12

• UN/EDIFACT

• DFDL

• Fixed Width

• Java Objects

• MIME

• SOAP

• Software AG Broker Docs

• SWIFT

• CHIPS

• FedWire

• TIBCO ActiveEnterprise

• HL7 v3

• XML (DTD, XSD, WSDL)

• ISO8583

• Custom

• CentraSite

• Oracle Fusion

• SCA Domain

• Software AG IS, Broker,

BPMS

• WebSphere Application

Server

• WebSphere Message

Broker

• WebSphere MQ

• Websphere SI Bus

• Sonic ESB

• System z (CICS and IMS)

• TIBCO ActiveMatrix

• UDDI

• Web Services

• WebSphere RR

• WSDL

• BPM

• Databases

• Log Files

Supporting technologies the business depends on Protocols, message formats, and technologies



Integration testing is interrelated and connected to your other aspects of application testing

Integration Testing

Release to Production

Testing

Acceptance

System

Performance

Unit Testing

Analysis

Requirements

Functional Testing

Coding

Main

tenance

All of these tests must be planned. Artifacts created by tests need to be

managed, linked to other development artifacts, and analyzed.

14



Rational also offers a complete suite of testing tools

10

Rational Functional Tester

• Record, enhance, and execute automated tests

• Java and/or Visual Studio generated test scripts

Rational Integration Tester

• Web service visual editors • WS-BPEL testing • Code-free testing

Rational Performance Tester

• Root cause analysis • Performance loads • Automated script capture

and playback

Jazz Team Server

Adapters

Test Planning:

Comprehensive test plan, Shared objectives, Scope,

Timeline, Resources, Risk assessment

Test Execution:

Manual test execution, Use test automation tools,

Record test results, Submit & track defects Reporting & Dashboard:

Status & progress tracking, Customizable live

dashboard, Real-time metrics and reports,

Compliance and quality audit

Test Construction:

Requirement-driven testing, Test environments

coverage, Manual test authoring, Test lab

management

Team Collaboration:

Process enactment and enforcement, reviews, task

management, Rapid team member on-boarding

Rational Quality Manager

Rational Test Workbench



Rational Test Virtualization Server

Webservices Databases MQ Mainframes Middleware and others…

Test Management

Automated Deployment


Urban Code

Version Control

APM

Automated Testing

Test Data

Continuous Integration


and others

Jenkins

and others Optim

and others


ITCAM

and others

RTC and

others

and Manual Testing



Orchestrate and automate the deployment and testing of applications,

middleware configuration, and database changes into development,

test and production environments.

Databases Internal Messages

Third-party Services

virtual components

Simultaneously

test across

multiple test

stages

Dev QA

IBM Rational Test

Virtualization Server

Combining continuous testing with Continuous Testing from build to production Reliable, repeatable, & traceable deployment automation

IBM UrbanCode Deploy

IBM Rational

Test Workbench

integrated with

Test Environments

Dynamic Infrastructure

Deploy what is ready,

virtualize the rest

Continuously test in

production-like env.

Deploy private, secure

right sized test data

IBM InfoSphere

Optim Test Data Mgmt

UC Deploy Plugins

IBM Rational

Quality Manager

Manage quality across

the delivery lifecycle



Continuous Deployment Continuous Testing

UrbanCode

Deploy

Automated Integration Testing


1 2

3

1. UrbanCode Deploy enables continuous deployment of available

environments into the targeted infrastructure

2. RTW/RIT can automatically test the deployment and integration of that

environment - do the components communicate correctly?

3. Service Virtualization allows the testers to:

a. Stand up virtual services in place of components that are

unavailable (not built yet, too expensive, resource contention, 3rd

parties etc.)

b. Plan for test conditions, such as negative testing that may not be

possible with the real components.

= RIT/RTVS

= URBANCODE

Continuous testing and continuous deployment



Continuous testing with virtualized services

Automating deployment and testing to production-like environments

Databases Mainframe applications

Third-party Services

Rational Test

Virtualization Server

Application

changes

being tested

Virtualized Services

Dev QA

Rational Team Concert

Jenkins Build

Deploy


Test

IBM UrbanCode Deploy

RTW plugin RTVS plugin



RQM and Service Virtualization Invocation

20

RQM RTCP / RTVS RTW / RIT

Associate Test with Virtual Service

Publish Virtual Service

Start Virtual Service

Run Test Stop Virtual Service

Virtual Service Logs linked to Test Case Result


Rational Test Control Panel

Rational Test Virtualization Server

RTW: Rational Test Workbench

RIT: Rational Integration Tester

Create Test Case Create Virtual Test Service



The Coca-Cola Company decreased production defects by 26-50% with IBM

Business Challenges:

• Quality issues/defects found in production

• Insufficient or unavailable test

environments

The Solution: The Coca-Cola Company

chose IBM’s service virtualization and

automated integration testing solution over

other offerings for the following reasons:

broadest support for the technologies in their

environment; cost of the solution; realizing

increased developer and tester productivity.

Software:

IBM Rational Test Virtualization Server, IBM


26-50% Decreased

Defects in production

26-50% Accelerated

Project delivery

26-50% Improved

Team productivity

26-50% Reduced

Cost of testing

http://www.techvalidate.com/tvid/1B0-FCA-9AD



DEMO



Configure Virtualize Model Deploy Execute

Execute

specified

test cases

Listen and

capture

transactions

Build

negative

test case

Refine ROI

Model

behavior of

transactions

Model

negative test

cases

Model

“perfect

storm” test

cases

Apply

metrics to

virtual

models

Deploy

models to

virtual

environment

Configure

SUT

connections

Execute

specified test

cases

Capture test

results

Capture and

measure

results

Recommend

next steps

Install and

configure

SV software

Configure

transactions

/ protocols

Configure

services

Build

additional

AV test

cases

Identify

testing pain

points

Understand

IT

technologies

Review use

cases

Initiate ROI

analysis

Understand

testing

methodology

Discover

Approach to get started with Virtualization



Documents

Shift Left with Service Virtualization: Rational Test ...gotocon.com/dl/goto-cph-2015/slides/KhurramNizami_Shift...IBM Rational Test Virtualization Server Continuous Testing from build